Skip to main content

Integrate Luxonis OAK Cameras with Foxglove

You can stream data from Luxonis OAK cameras to Foxglove using DepthAI and the Foxglove SDK. Stream RGB images, colored point clouds, camera calibration, IMU, and transforms, with no ROS required. The camera calculates depth on-device (stereo or neural, chosen with --depth-source), and the SDK streams results over a WebSocket that Foxglove connects to directly.

note

If you are using ROS, launch your OAK ROS 2 node and follow the ROS 2 documentation to run Foxglove Bridge.

This approach works with all DepthAI-enabled devices. The examples require a stereo camera pair and an IMU — tested with the OAK 4 D. --depth-source neural needs an OAK 4 (RVC4) device; use --depth-source stereo (the default) on older OAK cameras. Use the Python and C++ SDK examples as the reference implementation.

Prerequisites​

  • A DepthAI-enabled device with a stereo camera pair and an IMU (for example a Luxonis OAK 4 D), connected over USB 3 or Ethernet
  • For Python: uv (installs foxglove-sdk and DepthAI v3 from pyproject.toml)
  • For C++: the DepthAI C++ library (depthai-core). If it's in a non-standard prefix, pass CMAKE_PREFIX_PATH or depthai_DIR when building
  • On Linux, install the Luxonis udev rules for USB devices

Published Topics​

The examples publish these topics:

TopicSchemaContents
/oak/pointsfoxglove.PointCloudColored point cloud in meters
/oak/rgb/imagefoxglove.RawImageColor video (bgr8)
/oak/rgb/calibrationfoxglove.CameraCalibrationColor-camera intrinsics and distortion
/oak/imuJSON (ROS sensor_msgs/Imu-like)Accelerometer (m/s²) and gyroscope (rad/s)
/tffoxglove.FrameTransformsTransform that orients the camera frame upright

Installation​

Python Installation​

  1. Clone the repository and open the Python example:

    git clone https://github.com/foxglove/foxglove-sdk.git
    cd foxglove-sdk/python/foxglove-sdk-examples/oak-camera-streaming

  2. Install dependencies with uv:

    uv sync

    This installs foxglove-sdk, DepthAI v3, and the other dependencies declared in pyproject.toml.

Usage​

Running the Python Application​

cd foxglove-sdk/python/foxglove-sdk-examples/oak-camera-streaming

# Stream to Foxglove via WebSocket (default port 8765)
uv run python main.py

# Use neural depth instead of stereo (OAK 4 / RVC4 only)
uv run python main.py --depth-source neural

# Also record to an MCAP file
uv run python main.py --record oak.mcap

Command Line Options​

OptionDescriptionDefault
--depth-sourceDepth source: stereo or neural (neural requires OAK 4 / RVC4)stereo
--portWebSocket server port8765
--record <file>Also record to an MCAP file at this pathDisabled
--point-unitUnit of the DepthAI point coordinates: auto, meters, or millimetersauto
--help, -hShow help message-

--point-unit auto detects whether DepthAI point coordinates are meter-scale or millimeter-scale before publishing Foxglove point clouds in meters. If the point cloud renders ~1000× too large or too small in the 3D panel, override the detection with --point-unit meters or --point-unit millimeters.

How It Works​

DepthAI v3 describes on-device processing as a graph of nodes. Color and depth feed an RGBD node that produces a colored point cloud; the IMU publishes on a separate path. The snippet below is abridged from main.py. The examples create typed Foxglove channels, start a WebSocket server, then poll RGBD and IMU queues and log Foxglove messages. The C++ example mirrors this structure; see the Python example and C++ example for the full convert-and-log loop.

Build the DepthAI Pipeline​

color = pipeline.create(dai.node.Camera).build(sensorFps=fps)
# stereo or neural depth from CAM_B / CAM_C ...
rgbd = pipeline.create(dai.node.RGBD).build(color, depth, size, fps)
rgbd.setDepthUnits(dai.LengthUnit.METER)

imu = pipeline.create(dai.node.IMU)
imu.enableIMUSensor(dai.IMUSensor.ACCELEROMETER_UNCALIBRATED, IMU_HZ)
imu.enableIMUSensor(dai.IMUSensor.GYROSCOPE_UNCALIBRATED, IMU_HZ)

Visualize OAK Camera Data in Foxglove​

Run the example, then connect Foxglove to the WebSocket server.

Make sure you are on the same network as your robot. In Foxglove, select Open connection from the dashboard or left-hand menu.

Select open connection

Select Foxglove WebSocket in the Open a new connection dialog, then enter the URL of the server:

Foxglove WebSocket dialog

Click "Open" to connect.

note

Local development: Use ws://localhost:8765 when running the server on the same machine as Foxglove.

Robot connection: Use ws://ROBOT_IP:8765 when connecting to a server running on your robot, where ROBOT_IP is your robot's IP address on the network.

Optionally import the example layout at foxglove/oak_layout.json (in the Python example directory) via Import from file… in the layout menu.

Visualize Camera Feed​

Add an Image panel to your layout by clicking the Add panel button in the top left corner and selecting the Image panel.

Add image panel

Configure the panel by selecting the appropriate image topic from the Topic dropdown in the panel settings.

To visualize the RGB camera feed, select the /oak/rgb/image topic. Set Calibration to /oak/rgb/calibration so the Image panel can undistort the feed using the device intrinsics.

Select Image panel

Visualize Point Cloud​

Add a 3D panel to your layout by clicking the Add panel button in the top left corner and selecting the 3D panel.

Add 3D panel

Enable visibility of /oak/points in the topic section. Select oak as the display frame. Optionally enable /oak/rgb/calibration in the topic list to see the camera frustum.

Set up 3D panel

Visualize IMU Data​

The /oak/imu payload is shaped like ROS sensor_msgs/Imu. Add a Plot panel and set the message path to /oak/imu.linear_acceleration.x or /oak/imu.angular_velocity.x (and .y / .z) to chart accelerometer or gyroscope readings over time.

Learn More​