Features

• [Beta] Stitching node
  • Stitches frames from two different sensors based on matching features in the overlap of the images
  • Supports two stitching modes:
    • Panorama stitching based on the image overlap
    • Planar projection onto a configurable plane
  • A panorama example and a planar projection example are available
• [RVC4] Align node
  • A generic way to align Transformable messages or ImgFrames with each other in any order
  • Can align an ImgFrame to any transformable message type
  • Supports transforming custom messages if they override the transformTo function and the node runs on the host. A Python example and a C++ example are available
  • ImgDetections alignment is shown in a C++ example and a Python example
• Message save and load functions
  • DepthAI messages can be written to a file and then read back with the new API: msg.save("name.dai") and msg.load("name.dai")
  • The saved file is a binary protobuf envelope
  • The following messages are supported: ImgFrame, NNData, ImgDetections, SpatialImgDetections, SegmentationMask, PointCloudData, RGBDData, IMUData, and ImgAnnotations
• [RVC4] ToF undistortion
  • setOutputUndistortion lets you undistort the ToF depth frame
• Extrinsics now carry a toDeviceId field
  • Used together with toCameraSocket for better transformation handling in multi-device setups
  • The field is automatically populated with the ID of the device that created the message, because that is the device toCameraSocket refers to

Bug fixes

• [RVC2] VideoEncoder checks parameter coherency
  • Unvalidated parameters caused significant performance degradation when rendering a video stream with incorrect parameters
• [RVC2] The LEON OS unwind section now uses the standard .eh_frame naming, so the linker cleans up stale unwind records
• getMaskData bindings for SpatialImgDetections and ImgDetections now return a generic py::object
• Device discovery functions (for example getAllConnectedDevices) now release the GIL, which prevents stuttering of a running pipeline
• Four-point transform in ImageManip now properly handles normalized coordinates
• Rotation is now properly applied to addCropRotatedRect when coordinates are normalized
• Stricter intrinsics matrix validation
• The device health check is now more robust when the device is in use
• DetectionParser only considers outputs with _yolo in the name when validating the number of strides

Misc

• [RVC4] Optimized the loading of NN models and reduced the RAM footprint by half
• Added a DepthAI-specific reviewer skill for easier development and a faster merge process
• Embedded visualizer updated to 3.9.6:
  • Improved performance and stability

Known issues

• Downgrading from OS 1.40.0 to any OS between 1.24.0 and 1.31.1 causes an STM flash error that leads to a device crash when running any pipeline. To fix the issue, downgrade to OS 1.35.0 first, before you downgrade to the above OS range

[RVC4] Luxonis OS compatibility

Integration tested with Luxonis OS 1.30.1, 1.35.0, and 1.40.0.

Features

• Beta features namespace
  • Staging area for experimental DepthAI nodes and features
  • Enables faster support for features whose public API may change without prior notice
  • All nodes run natively on RVC4 and on the host for RVC2
  • All parsers and unique messages from depthai_nodes are ported over and have native support
  • Parsers have exposed runtime configuration changes via config messages
  • Python and C++ examples are available
• [RVC2] PoE Bootloader update
  • Embedded Bootloader version updated to v0.0.29
  • Greatly improves RVC2 PoE device discoverability by fixing stale connection information that could result in X_LINK_DEVICE_NOT_FOUND errors
  • The bootloader is backward- and forward-compatible and independent of the DepthAI version
  • Updating the recoverable user bootloader is recommended for devices with frequent discovery failures. For full integration, we recommend flashing the factory bootloader.
  • The following three flashing paths are available:
    • For a GUI experience, use device_manager.py. The update function is accessible within the "danger zone" section
    • Using the provided flash_network_bootloader.py script. By default, the script flashes the user bootloader; use the -f flag to enable factory flashing
    • Using the CLI command depthai --flash <device_ip>. By default, the command flashes the user bootloader; use the -f flag to enable factory flashing
  • For more information, follow the tutorial in our docs
• [RVC4] Alpha parameter for image undistortion
  • StereoDepth node now properly handles alpha scaling of undistortion on RVC4
  • Camera output requests now accept an optional alphaScaling value for controlling the balance between cropping and retaining the full field of view
• Depth colorization utility
  • Added dai::utility::colorizeDepthFrame in C++ and dai.utility.colorizeDepthFrame in Python
  • Supports ImgFrame, OpenCV matrices, configurable depth ranges and color maps, and logarithmic or linear scaling
  • Can automatically derive visualization bounds from valid depth pixels and render invalid pixels as black
• [RVC4] VideoEncoder improvements
  • Improved latency for 4000 x 3000 H.264, H.265, and MJPEG encoding
  • Fixed an issue where setKeyframeFrequency had no effect on RVC4

Bug fixes

• [RVC4] 480 FPS mode
  • Fixed a regression from 3.7.1 where the ImageManip pool size was too small to properly support 480 FPS HFR mode
  • ImageManip can now grow its output pool when it is running low on frames
  • Added setMaxPoolSize to cap the maximum output-pool size
• [RVC2] Added a total power limit for DOT and FLOOD projectors to avoid power resets on the RVC2 ToF 63D sensor
• [RVC2] SpatialLocationCalculator now remaps regions of interest correctly when the depth input is not aligned
• Fixed YOLOv6 and YOLOv6-R1 decoding, including stride-based box reconstruction and confidence handling
• Rectification now uses the transformations carried by its input frames and reinitializes when those transformations change
• Fixed ImageManip four-point transform did not account for normalized coordinates
• Fixed colorization flicker and inconsistent frame rates in the ToF examples
• ReplayVideo now reports a runtime error when FFmpeg cannot open a video instead of relying on an assertion

Misc

• Added a getConnectedCameras(CameraSensorType) overload for filtering connected cameras by sensor type
• Reworked stereo-pair discovery to consider only connected and calibrated cameras with compatible sensors, valid extrinsics, and suitable orientation
  • Stereo pairs are now ordered by descending baseline length
• cam_test.py now selects supported resolutions from the connected camera features instead of assuming 1280 x 800
• Added background_class and strides to NN Archive head metadata
• Python bindings for NeuralNetwork, DetectionNetwork, and SpatialDetectionNetwork now use a common variant-based model argument
• DetectionParser uses metadata-defined YOLO strides, and SegmentationParser uses the dedicated background-class field
• Python message types now inherit common Buffer timestamp and sequence-number methods instead of declaring duplicate bindings
• Examples:
  • Depth examples now use the unified Depth node where possible
  • The multi-device frame-sync example can encode synchronized streams as H.265
  • A Python example and a C++ example for printing bootloader versions have been added
• Fixed MSVC builds that treated a DeviceNodeCRTP deprecation warning as an error
• Updated the vcpkg baseline and explicitly use OpenCV 4 for macOS Homebrew builds
• Embedded visualizer updated to 3.7.6:
  • Improved performance and stability

[RVC4] Luxonis OS compatibility

Integration tested with Luxonis OS 1.27.1, 1.30.1, and 1.33.0.

Features

• Depth node
  • Unified API access to all depth sources (StereoDepth, GPU Stereo, Neural Depth, NeuralAssistedStereo, ToF)
  • Automatically selects the best possible depth source based on connected device features and user requirements (FPS, resolution)
  • C++ examples can be found here, and Python examples can be found here
• Device health check
  • Quickly get device diagnostics for device status, connection type, bandwidth, power supply, calibration, camera and IR sensor functionality, and any reported issues
  • A C++ example can be found here, and a Python example can be found here
• ToF node update
  • Support for RVC4 Lite with ToF
  • Unified API for both RVC2 and RVC4
  • New confidence output
  • A C++ example can be found here and a Python example can be found here
• Support for YOLOv12
• [RVC4] Default GPU backend for ImageManip
  • Improves performance and lowers power usage compared with the CPU backend
  • Support for image undistortion
• [RVC4] Added support for OAK4 Lite and OAK4 Lite ToF
  • Recognition of Lite devices
  • STMicro VD55H1 ToF sensor drivers and processing
• [RVC4] Added minimum- and maximum-range filtering to NeuralDepth
• [RVC4] Added Unix system timestamps to all messages
• [RVC4] Merging flashed CBA calibration with device EEPROM calibration
  • For each socket, prefer device EEPROM calibration if available; otherwise, use the calibration flashed to the CBA EEPROM
  • Flash the selected CBA EEPROM with calibration data
• Multisensor Dynamic Calibration
  • Perform dynamic calibration between multiple sensors
• Embedded Visualizer updated to 3.5.2:
  • Performance improvements for higher FPS
  • Improved PointCloud visualization
  • Visualization support for Segmentation messages
  • Visualization support for instance segmentation in ImgDetections

Misc

• Function argument cleanup:
  • Function arguments that are passed by value and then cloned are now passed by reference
• Non-mutating functions are now const
• DetectionParser can now be set up with setNNArchiveHead or via the build function, with the head passed as a parameter
• SegmentationParser can now be set up with setNNArchive or setNNArchiveHead
• Control the camera's maximum ISO for auto exposure with setAutoExposureMaxISO
• Telemetry:
  • Support for Dynamic Calibration and Auto Calibration
  • Added rate limiting
• [RVC2] Set the FSync delay for the 63D ToF sensor

Bug fixes

• The specTranslation vector is now set in ImgTransformations for all messages
• Check whether the input file exists in the Replay node
• [RVC4] PointCloud node uses the calibrated translation vector instead of the spec translation vector
• [RVC2] Fixed phase rotation for the 63D ToF sensor
• [RVC2] Fixed incorrect argument names for Point2f and Size2f in the Python bindings
• [RVC2] FFC-4P now logs a warning if the number of sockets in the flashed calibration does not match the number of connected camera sockets

[RVC4] Luxonis OS compatibility

Integration tested with Luxonis OS 1.27.1, 1.30.1, and 1.33.0.

Features

• [RVC4] New GPUStereo node
  • C++ example here and Python example here
• [RVC4] Add the GPU backend to ImageManip
  • Improves performance and lowers power usage compared to the CPU backend
  • This will be the default backend on RVC4 starting with v3.8.0
• Embedded Visualizer updated to 2.5.3:
  • Improved pipeline debugging pane
• Add a Transformable interface to messages to allow easy remapping from one coordinate system to another
  • Exposed as a transformedMessage = message.transformTo(dai.ImgTransformations)
  • Supported messages: dai.ImgDetections, dai.SpatialImgDetections, dai.AprilTags, dai.Tracklets, dai.SegmentationMask
• Add telemetry to help us understand how the library is used so we can improve it
  • To opt out set DEPTHAI_TELEMETRY=0 in the environment
• Add support for 640x400 inputs to AutoCalibration, enabling more devices to be calibrated out of the box
• [RVC4] Add support for INT16 inputs in NeuralNetwork node
• [RVC4] Add sensor temperature metadata to ImgFrame when available
• [RVC2] Add support for a new Samsung 63D ToF sensor

Misc

• [RVC2] Add support for CBA calibration reading and flashing
• [RVC4] Unify color temperature behavior between RVC2 and RVC4
• [RVC4] VPP and Neural assisted stereo can now run at 50 FPS @ 1280x800 improved from 27 FPS from before
• [RVC4] Unify RVC4 IMU output to match RVC2 to never duplicate packets

Bug fixes

• Fix a rare issue on Windows where USB devices could fail to boot if discovery was running in another thread
• Improve stability of the .getAny API on the MessageQueue
• [RVC2] Fix a regression with magnetometer and rotation vector orientation on BNO086
• [RVC4] Fix IMU timestamp drifting over time

[RVC4] Luxonis OS compatibility

Integration tested on Luxonis OS 1.20.5, 1.27.1 and 1.30.1

Features

• AutoCalibration is now turned on by default and can be turned off with DEPTHAI_AUTOCALIBRATION=OFF

• Add support for IMU calibration and unify outputs with each sensor now having a RAW UNCALIBRATED CALIBRATED

  • Important: RAW used to be pre-rotated to the RDF coordinate system, but was now changed to always be captured directly from the IMU. To keep the previous behavior please switch to the UNCALIBRATED output.

• Add Extrinsics tracking to ImgTranformations along with a lot of nice utilities to remap/project points between images

  • Keep track on what is the rotation and translation in every message to a camera socket with the lowest index (eg. CAM_A)
  • C++ example here and Python here

• Add YOLO26 support to DetectionParser

• Add support for the new NeuralDepth models

  • Existing models with improved FPS

    • LARGE (768x480) from 10 FPS to 22
    • MEDIUM (576x360) from 26 FPS to 38
    • SMALL (480x300) from 42 FPS to 56
    • NANO (384x240) from 60 FPS to 85

  • New models added:

    • 1248x780 at 8.5 FPS
    • 1056x660 at 12.5 FPS
    • 960x600 at 14 FPS
    • 864x540 at 18 FPS

  • LuxonisOS 1.30.1 or newer is required

• Add a new PointCloud node with support to select a custom coordinate system

  • C++ examples here Python example here

• Improve Holistic record & replay support

  • Record & replay also devices calibration
  • Record & mock device capabilities (device.getConnectedCameraFeatures(), device.getConnectedIMU())
  • Auto sync the recordings for a more robust playback of the streams, when they need to be synced
  • Recording script here, for replay, use DEPTHAI_REPLAY={path to recording} without needing to change the script.

• Embedded Visualizer updated to 2.3.3:

  • Improved pipeline debugging pane
  • Small UI fixes
  • Improved device connection stability

• Improve automatic Camera sensor config selection and FPS selection

  • Fallback on cropped configs when a full resolution config cannot satisfy requirements of the requested outputs
  • Automatically put cameras that cannot be FSynced with other cameras into free-running mode
  • Correctly track intrinsics for the cropped sensor configs
  • Note, on RVC4, LuxonisOS 1.31 or newer is required

• Improve ObjectTracker with optional association in 3D and add velocity property to dai.Tracklet

• [RVC4] Add support for PTP in Camera syncing on RVC4

  • LuxonisOS 1.30.1 or newer is required
  • Supported on IMX586 and OG05 sensors
  • C++ example here Python example here

Misc

• Add LEFT and RIGHT alignment to StereoDepth and correctly respect RECTIFIED_LEFT and RECTIFIED_RIGHT on RVC4
• [RVC2] Add support for IMX380
• [RVC4] Use SNPE 2.41.0.251128 if available for NeuralNetwork engine
• Automatically switch DetectionParser to run on host in RVC2 in case the model contains a segmentation mask
• Add support for MessageDemux to run on host

[RVC4] Luxonis OS compatibility

• Integration tested on Luxonis OS 1.20.5, 1.27.1 and 1.30.1

Features

• Add AutoCalibration node which automatically performs dynamic calibration
  • It can be turned on implicitly by setting the environment variable as DEPTHAI_AUTOCALIBRATION=ON_START or CONTINUOUS
  • The other option is to use it explicitly; C++ example here, Python example here
• Merge v2.32.0 along with improvements on RVC2 in crashdump collection and device state retrieval

Misc

• Set tracked undistortion coefficients on legacy calibration to 0 when undistortion is turned on
• Add eventManager.waitForPendingUploads() to avoid losing Snaps and Events that are waiting in the queue at script exit
• Expand the stability test to include NeuralAssistedStereo, FeatureTracker and ObjectTracker

[RVC4] Luxonis OS compatibility

Integration tested on Luxonis OS 1.14.1, 1.20.5 and 1.27.1

Features

• Add support for semantic segmentation

  • C++ example and Python example

• Add support for ingesting instance segmentation in the SpatialLocationCalculator to improve 3D detections

  • C++ example and Python example

• Add support for spatializing keypoints into 3D

  • Python example

• [RVC4] Add support for USB protocol
  * Requires LuxonisOS 1.27.1 or newer
  * It can be forced with DEPTHAI_PROTOCOL=usb in the environment

• Add isp output to Camera node

  • C++ example and Python example

• Add Gate node

  • Enables easy opening and closing a message stream in runtime
  • C++ example and Python example

• [RVC4] Add support for HFR (High frame rate) on IMX586:

  • 1280x720 @ 480 FPS
  • 1920x1080 @ 240 FPS
  • C++ examples and Python examples

• Add support for XLink packetization

  • Optionally packetize messages sent through XLink and limit the bitrate per connection
  • Useful in advanced scenarios to allow streams with large messages to run in the background
  • Usage example

• Visualizer updated to 2.0.10

  • Support for live pipeline visualization when DEPTHAI_PIPELINE_DEBUGGING=ON is set
  • Visualize keypoints when showing ImgDetections
  • Automatically remap detections and annotations to the target stream

• Add a new CalibrationMetrics output to DynamicCalibration node

• Add support for a callback for Snaps & Events to get notified when an event gets sent to Hub example

• [RVC4] Add support for external FSync

  • C++ example and Python example

• [RVC2] Add support for setting custom sensor tuning per sensor

• [RVC2] Add support for setting sensor orientation in the Camera node

Bug fixes

• Fix ownership when dai::Pipeline is copy-assigned to another variable
• Fix replay for StereoDepth for the case where ImgTransformations aren't set

Misc

• Deprecate creating nodes without pipeline.create
• Make basalt a private dependency instead of public
• ImageAlign:
  • Warn if a distorted image is sent to ImageAlign's alignTo
  • Use ImgTransformations to get intrinsics and distortion coefficients instead of relying on heuristics and global calibration
• [RVC4] Add support for RVC4 devices to DeviceManager
• [RVC2] Add new fields to ImgDetections inside Script node bindings

[RVC4] Luxonis OS compatibility

Integration tested on Luxonis OS 1.14.1, 1.20.5 and 1.27.1

Features

• Add a device.getStatefunction, which returns the firmware state of the device in runtime
• Add setProcessor function to select the processor for Sync and Demux nodes

Misc

• Expand crashdump data in firmware with more context on what caused the crash

Bug fixes

• Fix a hard to hit cache coherency bug in MessageGroup message
• Remove fixed pattern noise for the OV9[2/7]82 sensor during bring scenes with low exposure times

Features

• Add Pipeline Debugging support
  • More details can be found in the README
  • The pipeline graph can be visualized with depthai_pipeline_graph
  • C++ examples here and Python examples here
• Add NeuralAssistedStereo node (classical stereo + neural depth fusion)
  • Uses VPP internally to fuse the classical stereo depth with neural depth
  • C++ examples here and Python examples here
• Add VPP node and configuration API
  • C++ examples here and Python examples here
• Expose XLinkBridges
  • For advanced use-cases, to configure the implicit XLink bridges created when crossing host-device boundary
  • C++ example here and Python example here
• Add Housing calibration support in CalibrationHandler
• [RVC4] Add SystemLogger node and expanded SystemInformation metrics
  • C++ example here and Python example here
• [RVC4] Update StereoDepth presets on RVC4 with improved quality/fillrate trade-offs

Misc

• Add binary services support to RemoteConnection and configurable callback thread count
• Add resize mode to build method in NeuralNetwork, DetectionNetwork and SpatialDetectionNetwork nodes
• Add support for more frame types in ImgFrame.setCvFrame()
• [RVC4] Add support for manual white-balance
• [RVC4] Turn off dot projectors on pipeline stop
• [RVC4] Add temporal filter to NeuralDepth node

Bug fixes and stability

• Fix SSD parser regression and add labelName support in keypoints/detections
• Fix file corruption when recording long videos with H264 encoding with Record node
• [RVC4] Turn off Nagle's algorithm to avoid latency spikes on Windows
• [RVC4] Fix a race condition in StereoDepth node changing the left and right input frames
• [RVC4] Fix camera mapping for custom RVC4 boards
• [RVC2] Fix an edge case for MessageGroup's cache coherency
• [RVC2] Remove false alarm warning in SpatialDetectionNetwork when input image was rotated

[RVC4] Luxonis OS compatibility

Integration tested on Luxonis OS 1.14.1, 1.18.3 and 1.23.1