Manage data

Team

Enterprise

Start processing data recordings with your Foxglove Agent.

Adding recording files

Move your completed recordings into the storage directory. The Agent will watch for recordings in this directory.

By default, the Agent registers for native filesystem notifications when files under the storage directory are created, deleted, or modified. Since these notifications are best-effort, the Agent also periodically scans the directory to compensate for missed notifications.

tip

In environments where the storage directory has very frequent filesystem activity, the Agent may exhibit high CPU utilization and/or log throttled warnings about dropped filesystem notifications. In such environments, it may be preferable to disable filesystem notifications by setting WATCH_USE_NOTIFY=false, and rely exclusively on periodic scanning. The scan interval can be configured with WATCH_RECONCILIATION_INTERVAL, which is set to 30s by default.

When the Agent detects a new file under the storage directory, it will attempt to parse it as a recording and summarize its contents. This work is wasted for files that are not recordings, or for files that are incomplete recordings. If the storage directory contains such files, you should configure recording filters so that the Agent will ignore them. For example, you may write recordings with a special suffix like .active to indicate they are incomplete, and then rename them (using mv rather than cp) to .bag or .mcap once they are complete.

note

The Agent will ignore MCAP files that do not contain trailing magic bytes, under the assumption that the file is incomplete. It is still preferable to write incomplete files with a suffix that will be ignored, and rename when the file is complete.

note

The ROS 1 bag writer uses an .active suffix on incomplete files by default. The Agent ignores files with this suffix by default.

Import to cloud

Recordings for your device will initially appear in the Recordings page. In order to visualize and download this data, you can select these recordings and click "Import". Once the import status is displayed as "complete", the recording is available. You can also do this with the Recordings API.

Automatically import recording files

The Agent can be configured to automatically upload recording files that match a configured glob pattern. Set the WATCH_AUTO_IMPORT_PATTERN variable in /etc/foxglove/agent/envfile to a valid glob pattern to enable this functionality. This pattern supports * and ? wildcards.

For example, the following configuration in /etc/foxglove/agent/envfile will cause MCAP files in the auto_import/ subdirectory of STORAGE_ROOT to be imported as soon as they are created:

WATCH_AUTO_IMPORT_PATTERN="auto_import/*.mcap"

Delete recording files

You can delete recording files as needed to manage disk space on your device. Files that were imported will remain available in Foxglove after the original is deleted. Files that were not imported before deletion will no longer appear in the Recordings page.