CompressedVideo

A single frame of a compressed video bitstream

Schema

field	type	description
timestamp	time	Timestamp of video frame
frame_id	string	Frame of reference for the video.
data	bytes	Compressed video frame data.
format	string	Video format.

frame_id

The origin of the frame is the optical center of the camera. +x points to the right in the video, +y points down, and +z points into the plane of the video.

data

For packet-based video codecs this data must begin and end on packet boundaries (no partial packets), and must contain enough video packets to decode exactly one image (either a keyframe or delta frame). Note: Foxglove does not support video streams that include B frames because they require lookahead.

Specifically, the requirements for different format values are:

• h264

  • Use Annex B formatted data
  • Each CompressedVideo message should contain enough NAL units to decode exactly one video frame
  • Each message containing a key frame (IDR) must also include a SPS NAL unit

• h265 (HEVC)

  • Use Annex B formatted data
  • Each CompressedVideo message should contain enough NAL units to decode exactly one video frame
  • Each message containing a key frame (IRAP) must also include relevant VPS/SPS/PPS NAL units

• vp9

  • Each CompressedVideo message should contain exactly one video frame

• av1

  • Use the "Low overhead bitstream format" (section 5.2)
  • Each CompressedVideo message should contain enough OBUs to decode exactly one video frame
  • Each message containing a key frame must also include a Sequence Header OBU

format

Supported values: h264, h265, vp9, av1.

Note: compressed video support is subject to hardware limitations and patent licensing, so not all encodings may be supported on all platforms. See more about H.265 support, VP9 support, and AV1 support.

Reference implementations

Foxglove schemas are framework-agnostic, and can be implemented using any supported message encoding:

encoding	schema
ROS 1	foxglove_msgs/CompressedVideo
ROS 2	foxglove_msgs/msg/CompressedVideo
JSON	foxglove.CompressedVideo
Protobuf	foxglove.CompressedVideo
FlatBuffers	foxglove.CompressedVideo
OMG IDL	foxglove::CompressedVideo

You must use the schema names specified above for Foxglove to recognize the schema.