Optimize MCAP files

Bring your own storage and on-premises index-in-place sites read message data directly from your original files. Use the following recommendations to reduce query latency.

Prepare MCAP files so that:

• Chunks use zstd or lz4 compression
• Chunks are 4 MB or less
• High-bitrate topics live in separate files
• Chunk indexes are present (rebuild with mcap recover if missing)
• Metadata is present so Foxglove can group data properly

Optimizing files

Write optimized files from the start

If you control the MCAP writer, configure it for optimized read performance.

Python

import foxglove
from foxglove.mcap import MCAPCompression, MCAPWriteOptions

options = MCAPWriteOptions(
    compression=MCAPCompression.Zstd,  # or MCAPCompression.Lz4
    chunk_size=1024 * 1024,  # 1 MB
)

with foxglove.open_mcap(
    "drive-camera.mcap",
    writer_options=options,
    channel_filter=lambda channel: channel.topic.startswith("/camera/"),
), foxglove.open_mcap(
    "drive-telemetry.mcap",
    writer_options=options,
    channel_filter=lambda channel: not channel.topic.startswith("/camera/"),
):
    # Log messages while both writers are open.
    ...

C++

#include <foxglove/mcap.hpp>

foxglove::McapWriterOptions camera_options = {};
camera_options.path = "drive-camera.mcap";
camera_options.compression = foxglove::McapCompression::Zstd;  // or foxglove::McapCompression::Lz4
camera_options.chunk_size = 1024 * 1024; // 1 MB
camera_options.sink_channel_filter = [](const foxglove::ChannelDescriptor& channel) {
  return channel.topic().starts_with("/camera/");
};

auto camera_writer = foxglove::McapWriter::create(camera_options).value();

foxglove::McapWriterOptions telemetry_options = {};
telemetry_options.path = "drive-telemetry.mcap";
telemetry_options.compression = foxglove::McapCompression::Zstd;  // or foxglove::McapCompression::Lz4
telemetry_options.chunk_size = 1024 * 1024;  // 1 MB
telemetry_options.sink_channel_filter = [](const foxglove::ChannelDescriptor& channel) {
  return !channel.topic().starts_with("/camera/");
};

auto telemetry_writer = foxglove::McapWriter::create(telemetry_options).value();

Rust

let camera_options = mcap::WriteOptions::default()
    .chunk_size(Some(1024 * 1024))  // 1 MB
    .compression(Some(mcap::Compression::Zstd)); // or mcap::Compression::Lz4

let _camera_writer = foxglove::McapWriter::with_options(camera_options)
    .channel_filter_fn(|channel| channel.topic().starts_with("/camera/"))
    .create_new_buffered_file("drive-camera.mcap")?;

let telemetry_options = mcap::WriteOptions::default()
    .chunk_size(Some(1024 * 1024))  // 1 MB
    .compression(Some(mcap::Compression::Zstd)); // or mcap::Compression::Lz4

let _telemetry_writer = foxglove::McapWriter::with_options(telemetry_options)
    .channel_filter_fn(|channel| !channel.topic().starts_with("/camera/"))
    .create_new_buffered_file("drive-telemetry.mcap")?;

Process existing files before upload

If you can't configure the writer, rewrite existing files before upload:

Inspect the file and correct errors with:

mcap info <filename>
mcap du <filename>
mcap list chunks <filename>
mcap doctor <filename>
mcap recover <filename>

Check the uncompressed chunk size (mcap list chunks) and which topics dominate file size (mcap du). Keep uncompressed chunk sizes to 4 MB or less. Large chunks or mixed high-bitrate topics in one file hurt query latency.

Splitting large topics into separate files allows for more selective reading of data and faster queries. Use mcap filter to partition topics:

mcap filter drive.mcap -o drive-camera.mcap \
  --include-topic-regex '^/camera/.*' \
  --compression zstd \
  --chunk-size 1048576  # 1 MB

mcap filter drive.mcap -o drive-telemetry.mcap \
  --exclude-topic-regex '^/camera/.*' \
  --compression zstd \
  --chunk-size 1048576  # 1 MB

If a file remains too large, split it further by time. Use the start / end values from mcap info (RFC3339 or nanoseconds):

mcap filter drive-camera.mcap -o drive-camera-part1.mcap \
  --start 2026-07-14T10:00:00Z \
  --end 2026-07-14T11:00:00Z \
  --compression zstd \
  --chunk-size 1048576  # 1 MB

mcap filter drive-camera.mcap -o drive-camera-part2.mcap \
  --start 2026-07-14T11:00:00Z \
  --compression zstd \
  --chunk-size 1048576  # 1 MB

Add metadata and upload

Use sessions or devices to group split recordings in Foxglove. This example uses the original filename as the session key; use any key that is unique within the project.

Set object metadata in the same operation as the upload:

AWS

SESSION_KEY="drive.mcap"
DEVICE_NAME="robot-1"
PREFIX="s3://your-bucket/robot-1/drives"

for file in drive-camera.mcap drive-telemetry.mcap; do
  aws s3 cp "$file" "${PREFIX}/$(basename "$file")" \
    --metadata "{\"foxglove_device_name\":\"${DEVICE_NAME}\",\"foxglove_session_key\":\"${SESSION_KEY}\"}"
done

Google Cloud

SESSION_KEY="drive.mcap"
DEVICE_NAME="robot-1"
PREFIX="gs://your-bucket/robot-1/drives"

for file in drive-camera.mcap drive-telemetry.mcap; do
  gcloud storage cp \
    --custom-metadata="foxglove_device_name=${DEVICE_NAME},foxglove_session_key=${SESSION_KEY}" \
    "$file" "${PREFIX}/$(basename "$file")"
done

Azure

SESSION_KEY="drive.mcap"
DEVICE_NAME="robot-1"
ACCOUNT="your-storage-account"
CONTAINER="your-container"
PREFIX="robot-1/drives"

for file in drive-camera.mcap drive-telemetry.mcap; do
  az storage blob upload \
    -f "$file" \
    --account-name "$ACCOUNT" \
    --container-name "$CONTAINER" \
    -n "${PREFIX}/$(basename "$file")" \
    --overwrite \
    --metadata \
      "foxglove_device_name=${DEVICE_NAME}" \
      "foxglove_session_key=${SESSION_KEY}"
done

See adding metadata to imports for more info.