Release v3.0.0-rc.2

DepthAI v3.0.0 release candidate is out 🎉

We’re proud to announce a new major revision of the DepthAI library is now in release candidate stage and ready to be used.

Simplified API

We've simplified and unified the API to lower the barrier to entry and make development more intuitive. Core tasks like setting up camera streams, running neural networks, and building spatial pipelines are now streamlined with fewer lines of code.

We have removed many deprecated functions and simplified node constructors, creating a cleaner, more consistent, and easier-to-use library for newcomers as well as more experienced users.

Refreshed nodes

We've revisited some of the most fundamental nodes to make them more powerful and flexible.

• Camera

  • The ColorCamera and MonoCamera nodes have been replaced by a single, generic Camera node that works across all platforms.
  • It supports a variable number of outputs, where each can have an independent:
    • Type (NV12, RGB interleaved or planar, ...)
    • Size
    • FPS
  • Python and C++ examples.

• ImageManip

  • The ImageManip node has been revised for more robust and predictable behavior (we now track the order of operations to make the output fully deterministic).
  • It now supports relative crops and can be used as a host-side node in addition to running on RVC2 and RVC4 devices.
  • Python and C++ examples.

New nodes

DepthAI v3 introduces new high-level nodes to simplify working with high level concepts (RGBD pointclouds, SLAM)

• RGBD: This node makes it easy to work with synchronized and aligned color and depth data. It features an "autocreate" mode that automatically constructs the required input pipeline, delivering a ready-to-use RGBD message or a colored point cloud.
  • Python and C++ examples.
• VSLAM: We've added SLAM and VIO as host nodes, enabling advanced spatial AI applications. These are currently available on Linux and macOS and are in early preview.
  • Python and C++ examples.

Support for host nodes

DepthAI v3 now supports custom host nodes, allowing you to run parts of your pipeline on the host with custom logic while keeping the rest on-device. Whether you need to run complex post-processing in Python or integrate with external libraries, it’s all part of the same unified pipeline. See the Python and C++ examples.

Visualizer

Debugging and development just got easier with our new integrated visualizer. You can now visualize camera streams, neural network outputs, and detection overlays in real-time. The visualizer is also able to display the pipeline graph, making it a great tool for understanding and debugging your application's data flow.
See the Python and C++ examples.

A lot of nice to have features:

• Auto reconnect: If a device has a flaky connection and crashes, the library will now automatically try to reconnect instead of exiting, making your applications more resilient.
• Unified Coordinate System: We’ve standardized all coordinate systems. Camera, IMU, and spatial data now use the RDF (Right-Down-Forward) convention, eliminating inconsistencies when fusing data from multiple sensors.
• Native Model Zoo Support: Take advantage of the new Luxonis Model Zoo, which integrates natively into v3. Loading and deploying models is now seamless, with no boilerplate required.
• Remapping points between different frames: We've much improved the developer experience where detections from one image in the pipeline (on the input of the NN for example) need to be remapped to another image in the pipeline, that's still in full resolution by internally tracking all of the image operations that are made to streams. See Python and C++ examples.

Support for RVC4 devices

DepthAI v3 brings support for both RVC2 and RVC4 devices, with a unified codebase that runs on both platforms. This allows you to prototype on one and deploy on another without code changes in most cases.

Migration from v2.x version of the library

We've prepared a porting guide to speed up the migration here.

Known issues

• On release flavors of the Luxonis OS on RVC4, the camera stack can get stuck in a bad state where cameras do not restart streaming, requiring a device reboot. It is recommended to use -debug versions of the OS until this is resolved.

Release v2.30.0

Features

• Add RVC4 discovery to point users to v3 version of the library for OAK4 devices
• Add support for a new VCM enabling autofocus on new IMX378 CCMs

Bug fixes

• Fix an edge case in ImageManip to make https://github.com/geaxgx/depthai_hand_tracker run in edge mode again
• Fix an edge case when sending MessageGroup from host to device and using more than 4 messages
• Fix a bug where standolone logging would cause a memory leak

Release v2.29.0

Features

• Add the ability to change the calibration on the device in runtime with the new dai::Device.setCalibration() method and to retrieve it with the dai::Device.getCalibration().
• New StereoDepth presets:
  • DEFAULT
  • FACE
  • HIGH_DETAIL
  • ROBOTICS
• Multiple camera improvements (more details in #972):
  • Expose more downsampling modes when picking a lower than native resolutions
  • Expose more binning modes when binning is picked on IMX582/586 (sum and avg)
  • HDR on IMX582/586
  • Option to bypass 3A for having manual expose/ISO take effect faster
  • Initial support for new Sony 4K Starvis sensors: IMX678 and IMX715
  • Option to set the main camera to drive auto-exposure and auto-white-balance in multi-camera configurations
• Improved StereoDepth filtering and an option to use a set a custom order of filters
  • Disparity is first scaled to 13 bit before going through filtering, which results in filters being more effective.

Misc

• Remove false reports on crashes that happened on device destruction
• Add getWidth() and getHeight() API to EncodedFrame

Release v2.28.0

Features

• Changed the automatic crashdump collection to always on unless DEPTHAI_DISABLE_CRASHDUMP_COLLECTION is set.
• Added DEPTHAI_ENABLE_ANALYTICS_COLLECTION environment varialbe - when set, analytic data (pipeline schema) is sent to Luxonis which will be used to further improve the library.
• Undistort both outputs of ToF by default.
• Improved 3A syncing on OAK-D-LR
• Added support for YoloV10

Bug fixes

• Fix Camera node to correctly allocate resources for undistortion
• Fix StereoDepth node when decimation filter and depth alignment are used
• Fix host timestamps of thermal frames to be synced

Misc

• Updated XLink to support clangd and shared libraries on Windows:
  • luxonis/XLink#81
  • luxonis/XLink#84
• Remove a custom assert to always produce a crash dump to improve the UX with the automatic crashdump collection
• Increased watchdog priority on device side to improve stability during high load

Release v2.27.0

Features

• Automatic crash dump collection when DEPTHAI_ENABLE_FEEDBACK_CRASHDUMP is set.

Bug fixes

• Fix PointCloud generation when depth comes out of the ImageAlign node
• Fix IMX214 (OAK Lite) potential crash on init with more complex pipelines
• Fix a rare bug on a crash if a camera stream is closed after 1 second while AF is running

Release v2.26.0

Features

• ImageAlign node
  • it can align depth to any other sensor on the device - works for ToF too.
  • it can align two sensors with a static depth (default at infinity), useful for thermal-rgb alignment
• Cast node
  • Cast NNData message to ImgFrame
  • Useful in case apps need to use outputs from NeuralNetwork node to be fed into nodes that only accept ImgFrame
• Full ToF support
  • Running live at 30 FPS
  • Measuring range of 20cm - 6m
  • < 1% error across the range
• Support for ToF in spatial nodes
• Add an option to limit bandwidth over XLink setXLinkRateLimit(int maxRateBytesPerSecond, int burstSize, int delayUs)

Stability improvements and bug fixes

• Improved PoE stability on reboots - eliminate the case where powercycle of the device was sometimes needed
  • NOTE: This requires flashing the factory bootloader - by running the flash script or using the device manager
• Improved runtime stability of heavy pipelines by increasing priority of the cameras in the NoC
• Improved ImageManip stability
• Improved XLink communication to be able to detect memory corruption and avoid it
• Fix a bug where stereo rectification was inaccurate when the calibration data didn't contain direct link between the two inputs
  • Relevant for custom setups on FFC devices

Misc

• Improve numerical stability of the rectification algorithm
  • Improves stereo quality on wide FOV sensors

Release v2.25.1

Features

• Added DeviceBootloader::getFlashedVersion to retrieve the bootloader version that is flashed on device.

Bug Fixes

• Fixed fsync on OAK-D-SR.
• Fixed boot issue on OAK-D-SR-POE and OAK-T.
• Fixed compilation in same cases, because of problems with jsoncpp. (#980)

Misc

Release v2.25.0

Features

• PointCloud node along with the PointCloudData message
  • On device pointcloud generation
  • Native conversion to PCL message type - similar to getCvFrame()
• Support for the OAK Thermal
• New setManualFocusRaw() API with more granularity for setting manual focus position
• New getStereoPairs() and getAvailableStereoPairs() API to allow more device agnostic programming
• Add the intensity output to the ToF node
• Add the crop sensor configs in getConnectedCameraFeatures() now contain how each of the resolutions is cropped from the native sensor resolution

Bug fixes

• Bug fix for v2.24.0 regression with flashing the bootloader link
• Bug fix in message groups for large resolution frames when going through XLinkOut node
• Bug fix to apply AE limit during AF lens movement (before the fix, the AE limit wasn't applied during active AF movements)
• Bug fix for grayscale VideoEncoder node with 2 inputs of different sizes

Misc

• Fix timings for IMX214 4K resolution, which were slightly off

Release v2.24.0

Features

• New nodes and messages:
  • Sync node - syncs multiple inputs based on the timestamp, outputs a message group message
  • Demux node - demultiplexes message group in multiple messages
  • Message group message - a new message that can contain a map of arbitrary depthai messages, it's the output of the sync node and input to the demux node
  • Encoded frame message - a new message specialized for encoded frames
  • New output for the VideoEncoder node (out) for the encoded frame message
• Automatic crash dump retrieval for firmware crashes
• Added setIrFloodLightIntensity and setIrLaserDotProjectorIntensity methods for setting the intensity normalized between 0 and 1
• Added getConnectionInterfaces method to retrieve the list of available interfaces on a device
• Added an option to cap maximum time for exposure when using auto exposure with setAutoExposureLimit
• Initial integration for IMX283 and IMX462
• Improve time-syncing between the host and device to achieve sub 300 us offset
• Improved max FPS and image quality under low light for OV9282 and OV9782 #926 ,new ranges per resolution of:
  • THE_800_P: 1.687 .. 129.6 fps
  • THE_720_P: 1.687 .. 143.1 fps
  • THE_400_P: 1.687 .. 255.7 fps

Bug fixes

• Avoid overflow for XLink profiling #933
• Improve XLink stability when using multiple devices luxonis/XLink#73
• Fix a rare bug where the device would hand in the constructor #922
• Fix a bug where XLinkIn didn't work correctly for very small and very large buffers
• Fix a bug for running multiple stereo nodes with a shared input

Misc

• On multi-input NeuralNetworks set the output NNData to the newest input timestamp (previously undefined)
• Add NOC DDR usage reporting on DEPTHAI_LEVEL=info

Release v2.23.0

Features

• ImageManip - configurable interpolation type
• Script - added missing bindings for reading device releated data (on device calibration, etc...)
• Added Device::getDeviceName and Device::getProductName that target a user name and a "SKU" name
• Timesync - improved synchronization with tweaks to XLink
• Calibration - added housingExtrinsics

Bug fixes

• Fix CAM_D enumeration on OAK-FFC-4P R7
• BMI270 IMU - fix latency when slow reading
• Device related logging
• OAK-D-SR-POE - new revision bugfixes
• Fix 4 cameras crash on PoE due to memory allocation
• Fixed incorrect warp in issue #882

Misc

• #867
• #873
• XLink - updated number of devices to 64
• ToF - median filter and Z map support (still not the final decoding pipeline)
• hasAutofocus value fixed and introduced hasAutofocusIC
• Added timestamps and sequence numbers to all Message types
• Added DEPTHAI_ENABLE_LIBUSB for potentially disabling USB protocol (to not require libusb library)
• OV9782 and AR0234 improvements:
  • OV9782 FPS improvements
  • AR0234 max exposure time bump to 33ms and improved FSync INPUT mode