Skip to main content

Manage data

Add recordings to your Foxglove Edge Site, import them to your Primary Site, and manage local recording files.

Import to edge​

The edge controller catalogs recordings stored under its storage directory (STORAGE_ROOT) and reports them to Foxglove. Cataloged recordings appear on the Recordings page as not yet imported.

Choose one registration method for each storage directory. The controller allows both modes to run, but use either watch mode or the recordings API to register a file.

Watch mode​

Set WATCH_MODE=true to discover completed MCAP files in the storage directory and its subdirectories. The edge controller uses filesystem notifications and periodic scans, applying the recording filters below.

Watch mode catalogs MCAP files only. Use the recordings API for ROS 1 bag files.

Each MCAP file must include a metadata record named foxglove that names its device with deviceName or deviceId. Add projectId to choose the device's project. The edge controller skips MCAP files that don't name a device. For the metadata keys and how multiple records are merged, see MCAP metadata. For example, using the MCAP CLI:

mcap add metadata your_file.mcap --name foxglove --key deviceName=my-robot --key projectId=prj_...

For device and project resolution, see Adding metadata to imports. If Foxglove can't resolve the device, the edge controller logs skipping submission of followed by the path, and the recording doesn't appear in Foxglove.

The edge controller expects a recording to be complete when it finds it. Write each recording under a name the filters exclude, such as run.mcap.active, and rename it (using mv rather than cp) when it's complete.

note

The edge controller ignores MCAP files that don't contain trailing magic bytes, under the assumption that the file is incomplete. It is still preferable to write incomplete files with a suffix that the filters exclude, and rename them when they're complete.

To upload recordings as soon as they're cataloged, see Automatically import recordings.

Recording filters​

The installation examples select .mcap files. Use these variables to adjust which files watch mode reads:

# A comma-delimited list of file suffixes that the edge controller should treat as recordings
WATCH_INCLUDE_SUFFIXES=.mcap
# A regex of paths that the edge controller should treat as recordings
#WATCH_INCLUDE_REGEX=
# A comma-delimited list of suffixes the edge controller should ignore
#WATCH_EXCLUDE_SUFFIXES=
# A regex of paths the edge controller should ignore
#WATCH_EXCLUDE_REGEX=

A file must match all include filters and none of the exclude filters. With no filter variables, the controller ignores files ending in .active or .json. The deprecated WATCH_IGNORE_SUFFIX applies only when no WATCH_INCLUDE_* or WATCH_EXCLUDE_* variable is set; setting any of those variables replaces both default exclusions.

To rely on periodic scans instead of filesystem notifications, set WATCH_USE_NOTIFY=false. Adjust the scan interval with WATCH_RECONCILIATION_INTERVAL (default 30s).

Import to cloud​

Recordings from your Edge Site initially appear on the Recordings page. To visualize or download them, select the recordings and click "Import". Once the import status shows "Complete", the recordings are available. You can also request import through the Recordings API.

Automatically import recordings​

Set WATCH_AUTO_IMPORT_PATTERN to upload matching recordings as soon as they're cataloged. The glob is matched against each recording's path relative to the storage directory. * matches across directory separators; ** patterns also work for recursive matches.

To automatically import every MCAP file in the storage directory and its subdirectories:

WATCH_AUTO_IMPORT_PATTERN=**/*.mcap

To restrict automatic import to the auto_import/ directory and its subdirectories:

WATCH_AUTO_IMPORT_PATTERN=auto_import/**/*.mcap

Recordings that aren't imported automatically still appear on the Recordings page, where you can import them on demand.

Delete recording files​

You can delete local recording files to manage disk space. Recordings already imported to your Primary Site remain available there after the local file is deleted. Recordings removed from the Edge Site before import are no longer available to import.

In watch mode, deleting a file from the storage directory is sufficient: the controller detects the removal and reports it to Foxglove.

In recordings API mode, manually deleting a file doesn't immediately update its listing. There is no local deletion notification endpoint. The controller discovers the missing file when the recording becomes eligible for retention cleanup; with indefinite retention, the listing remains. Use a retention policy to let the controller delete files and report their removal.

Retention​

The site's "Automatically delete recordings after (days)" setting controls how long the Edge Site keeps recordings. Organization admins can change it on the site's page under Sites settings. Changes apply without restarting the controller.

  • With no retention period, the Edge Site keeps recordings indefinitely.
  • With a retention period, the controller deletes recordings whose file change time is older than the period. It keeps recordings queued for import or waiting to upload until their upload succeeds.

Edge Sites support time-based retention, but don't support automatic deletion immediately after import.