Skip to main content

Logging strategies

When logging robot data, you'll need to balance between having enough information for debugging and keeping file sizes manageable. This guide covers different logging strategies to help you choose the right approach for your use case.

Lightweight logging​

Lightweight logging focuses on capturing essential information needed to understand what happened to your robot without excessive data overhead.

When to use lightweight logging​

  • Production deployments where storage space is limited
  • Long-duration missions where you need to log for extended periods
  • Bandwidth-constrained environments where upload speed matters
  • Routine operations where you primarily need to detect when something goes wrong

What to include in lightweight logs​

  • Diagnostic messages - Focus on warnings, errors, and fatal codes
  • Key sensor data - Essential sensors like IMU, GPS, wheel encoders
  • System status - Battery levels, temperature, system health
  • Command/response pairs - High-level commands and their outcomes
  • Filtered sensor data - Final outputs of processing chains, skip intermediate steps
  • Throttled high-frequency data - Reduce image and point cloud logging rates

Example lightweight logging configuration​

import foxglove
import time

# Create channels for essential data only
battery_channel = foxglove.channels.Float32Channel("/battery_level")
status_channel = foxglove.channels.StringChannel("/system_status")
pose_channel = foxglove.channels.PoseChannel("/robot_pose")

# Throttle image logging to 1Hz instead of 30Hz
image_channel = foxglove.channels.RawImageChannel("/camera/image")
last_image_time = 0
image_throttle_interval = 1.0  # seconds

with foxglove.open_mcap("lightweight_log.mcap"):
    while True:
        current_time = time.time()

        # Log essential data at full rate
        battery_channel.log(battery_level)
        status_channel.log(system_status)
        pose_channel.log(robot_pose)

        # Throttle image logging
        if current_time - last_image_time > image_throttle_interval:
            image_channel.log(camera_image)
            last_image_time = current_time

        time.sleep(0.1)

Debug logging​

Debug logging captures comprehensive data needed to fully understand and reproduce issues in your robot system.

When to use debug logging​

  • Development and testing phases
  • Troubleshooting specific issues where you need to replay scenarios
  • Algorithm development where you need to iterate on processing pipelines
  • Short-duration experiments where storage isn't a concern
  • Critical missions where you can't afford to miss any data

What to include in debug logs​

  • Raw sensor data - All input sensors at their original rates
  • Intermediate processing steps - Outputs from each stage of your processing pipeline
  • Full image and point cloud data - Unthrottled visual and spatial data
  • Detailed diagnostics - All log levels including debug information
  • System internals - Memory usage, CPU load, network status
  • Algorithm state - Internal state of SLAM, planning, and control algorithms

Example debug logging configuration​

import foxglove
import time

# Create channels for comprehensive data logging
battery_channel = foxglove.channels.Float32Channel("/battery_level")
status_channel = foxglove.channels.StringChannel("/system_status")
pose_channel = foxglove.channels.PoseChannel("/robot_pose")

# High-frequency sensor data
imu_channel = foxglove.channels.ImuChannel("/imu/data")
laser_channel = foxglove.channels.LaserScanChannel("/laser/scan")
image_channel = foxglove.channels.RawImageChannel("/camera/image")

# Processing pipeline intermediates
filtered_laser_channel = foxglove.channels.LaserScanChannel("/laser/filtered")
odom_channel = foxglove.channels.OdometryChannel("/odom")
slam_pose_channel = foxglove.channels.PoseChannel("/slam/pose")

# System diagnostics
cpu_channel = foxglove.channels.Float32Channel("/system/cpu_usage")
memory_channel = foxglove.channels.Float32Channel("/system/memory_usage")

with foxglove.open_mcap("debug_log.mcap"):
    while True:
        current_time = time.time()

        # Log all data at full rate
        battery_channel.log(battery_level)
        status_channel.log(system_status)
        pose_channel.log(robot_pose)

        # High-frequency sensors
        imu_channel.log(imu_data)
        laser_channel.log(laser_scan)
        image_channel.log(camera_image)

        # Processing intermediates
        filtered_laser_channel.log(filtered_laser_scan)
        odom_channel.log(odometry)
        slam_pose_channel.log(slam_pose)

        # System diagnostics
        cpu_channel.log(cpu_usage)
        memory_channel.log(memory_usage)

        time.sleep(0.01)  # 100Hz logging

Hybrid approach​

For many production systems, a hybrid approach works best:

  1. Always log lightweight data for continuous monitoring
  2. Switch to debug logging when issues are detected
  3. Use both strategies with different retention policies

Example hybrid implementation​

import foxglove
import time
from enum import Enum

class LoggingMode(Enum):
    LIGHTWEIGHT = "lightweight"
    DEBUG = "debug"

class HybridLogger:
    def __init__(self):
        self.mode = LoggingMode.LIGHTWEIGHT
        self.issue_detected = False
        self.debug_duration = 60  # seconds
        self.debug_start_time = None

        # Create all channels
        self.setup_channels()

    def setup_channels(self):
        # Essential channels (always logged)
        self.battery_channel = foxglove.channels.Float32Channel("/battery_level")
        self.status_channel = foxglove.channels.StringChannel("/system_status")

        # Debug channels (only when in debug mode)
        self.imu_channel = foxglove.channels.ImuChannel("/imu/data")
        self.image_channel = foxglove.channels.RawImageChannel("/camera/image")

    def check_for_issues(self):
        # Example: switch to debug mode if battery is low
        if battery_level < 20.0:
            self.issue_detected = True
            self.mode = LoggingMode.DEBUG
            self.debug_start_time = time.time()

    def should_log_debug_data(self):
        if self.mode == LoggingMode.DEBUG:
            if self.debug_start_time and time.time() - self.debug_start_time > self.debug_duration:
                self.mode = LoggingMode.LIGHTWEIGHT
                self.issue_detected = False
            return True
        return False

    def log_data(self, battery_level, status, imu_data, image_data):
        # Always log essential data
        self.battery_channel.log(battery_level)
        self.status_channel.log(status)

        # Log debug data only when needed
        if self.should_log_debug_data():
            self.imu_channel.log(imu_data)
            self.image_channel.log(image_data)

        # Check for issues that might trigger debug mode
        self.check_for_issues()

# Usage
logger = HybridLogger()
with foxglove.open_mcap("hybrid_log.mcap"):
    while True:
        logger.log_data(battery_level, status, imu_data, image_data)
        time.sleep(0.1)

MCAP configuration options​

When using MCAP files for logging, you can optimize performance and storage by configuring chunk compression and chunk size. MCAP normally stores messages in chunks, but it's also possible to write MCAP without using chunks, which consumes minimal resources on the robot at the expense of read performance.

Chunk compression​

Enabling chunk compression allows you to spend CPU resources on your robot to gain more recording time before running out of storage. Recording compressed data also saves time when pulling recordings off your robot.

# Enable compression when creating MCAP files
with foxglove.open_mcap("data.mcap", compression="zstd"):
    # Your logging code here
    pass

Chunk size​

The chunk size determines how much data is buffered in memory before writing to disk:

  • Large chunk size (1MB+) - Better compression ratio but more data loss risk if power is lost
  • Small chunk size (less than 1MB) - Less compression but more frequent disk writes, reducing data loss risk
# Configure chunk size for different scenarios
# For lightweight logging (frequent writes, less data loss risk)
with foxglove.open_mcap("lightweight.mcap", chunk_size=512*1024):  # 512KB
    pass

# For debug logging (better compression, more data)
with foxglove.open_mcap("debug.mcap", chunk_size=2*1024*1024):  # 2MB
    pass

Chunking strategies​

For production systems, consider implementing custom chunking based on time intervals:

import time

class TimeBasedChunker:
    def __init__(self, writer, interval_seconds=5):
        self.writer = writer
        self.interval = interval_seconds
        self.last_chunk_time = time.time()

    def should_chunk(self):
        if time.time() - self.last_chunk_time > self.interval:
            self.writer.start_new_chunk()  # Force new chunk
            self.last_chunk_time = time.time()
            return True
        return False

# Usage
chunker = TimeBasedChunker(mcap_writer, interval_seconds=5)
while True:
    # Log your data
    channel.log(data)

    # Check if we should start a new chunk
    chunker.should_chunk()

Best practices​

Storage optimization​

  • Use compression - MCAP files support built-in compression (ZSTD or LZ4)
  • Configure chunk size - Balance between compression ratio and data loss risk
  • Set appropriate retention policies - Delete old lightweight logs, keep debug logs longer
  • Monitor disk usage - Implement alerts when storage is running low
  • Use Foxglove Agent for automated data management

Topic filtering​

  • Exclude unnecessary topics - Don't log topics that don't add debugging value
  • Use regex patterns - Filter out temporary or test topics
  • Configure per-robot - Different robots may need different logging strategies

Performance considerations​

  • Buffer writes - Use buffered I/O for better performance
  • Async logging - Don't block your robot's main control loop
  • Monitor CPU usage - Logging shouldn't impact robot performance