Depthkit app

❗️

Depthkit projects are not backwards compatible.

This means that you cannot open your current project in a previous version of Depthkit.

Version 0.8.1

Released March 1, 2024

Supports Femto Bolt firmware version 1.0.9.

Updates

• Added WebXR geometry export preset for streaming Depthkit Core and Studio captures recordings over the internet to be rendered in web browsers on mobile devices and XR headsets. This preset is designed for use with our newly-refreshed Depthkit.js module and 8th Wall templates - See our tutorial to get started right away and WebGL page for more information. The format is based on open Draco standard and native HTML5 Video playback.
• Added new mesh simplification feature within the Depthkit Export panel for both Depthkit Studio and Depthkit Core/Cinema geometry exports. This allows users to preserve fidelity while reducing the exported asset’s file size, resulting in Depthkit meshes which are more easily stored, moved, and transferred/streamed over the internet.
• Added advanced controls and diagnostic visualization to filter out floor geometry while persevering feet details on Depthkit Studio clips.
• Added PLY pointcloud export format for visual effects workflows.
• Added a Combined Per Pixel Video export preset for Apple Vision Pro.
• Added a VFX Preset for Textured Geometry Sequences that preserves high quality output while dramatically reducing file size for use in Visual Effects Tools
• Added frame-stepping transport controls within the Depthkit Editor.

Bug Fixes

• Fixed an issue where Depthkit would produce invalid OBJ’s if Windows was set to a localization which used commas as decimal punctuation.
• Fixed inconsistent corrupted frame issue for certain Depthkit Studio exports.
• Fixed an issue where Depthkit would intermittently export blank frames of geometry.
• Fixed intermittent crash related to polling for sensors which are newly-connected to the computer.
• Improved connection stability with Orbbec sensors.
• Fixed viewport resetting itself when changing the bounding box on Studio clips.
• Fixed potential crash when using OSC remote control interface.
• Fix for sensor status icons not showing proper state in some circumstances.
• Fixed issue where color coding tint was not applied to perspectives during Depthkit Studio calibration
• Fixed an issue with Depthkit Core geometry exports where extra unwanted geometry was being introduced in areas of high depth disparity.

Known Issues

• Depthkit Studio Livestreams with 10 sensors sometimes stop streaming. We are investigating this issue.
• The Livestream Resolution Constraints set in preferences do not affect Core livestreams until Depthkit is restarted.
• Depthkit sometimes crashes when logging in with a different license than previously used. We are investigating this issue.
• In the Depthkit Editor, some clips may not automatically appear in the 3D viewport, Depth preview window, or Color preview window, until triggering a refresh with an event like starting playback. We are investigating this issue.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• When exporting Depthkit Core clips into Combined Per Pixel format, if the far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity Video Player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Depthkit object’s video player. This will remove the artifact.
• When capturing Depthkit Core clips in the Record context, Depthkit may get stuck in the finalizing state when the take is complete if the GPU-accelerated video encoding has been enabled Preferences immediately prior to clicking the Depthkit Core record button. To work around this, click another portion of the interface before clicking record when toggling this setting.
• When editing Depthkit Core clips, if you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.

Version 0.8.0

Released January 16, 2024

Updates

• Full support for the Orbbec Femto Bolt sensor, successor to the Microsoft Azure Kinect, and Femto Bolt firmware 1.0.9.
• Experimental support for Orbbec Femto Mega (Depthkit Core only)
• Added connection type info in the Sensor Configuration details pane to quickly troubleshoot USB connection issues.
• Support for Intel RealSense 400-series sensors removed due to a conflict with the Orbbec SDK.
• Added a warning popup dialog when starting sensors in an unsynchronized state.

Bug Fixes

• Fixed an issue where the meshes in the Depthkit Studio Calibration context were not properly tinted according to sensor color code.
• Fixed an issue where during finalization, the number of reported dropped frames would increase beyond the number of frames which were dropped during the recording.
• Fixed the orientation of Azure Kinect sensor 3D model within Depthkit Studio Calibration and Streaming 3D Viewports.

Known Issues

• Depthkit Studio Livestreams with 10 sensors sometimes stop streaming. We are investigating this issue.
• On computers set to some localizations which use a comma (”,”) to represent decimal place, Depthkit Core & Studio OBJ exports are written with incorrect formatting. This can be resolved by changing the localization settings in Windows to a localization which uses periods (”.”) to represent decimals, or by replacing all of the commas in the resulting OBJ files with periods. This will be resolved in an upcoming release of Depthkit.
• The Livestream Resolution Constraints set in preferences do not affect Core livestreams until Depthkit is restarted.
• Depthkit sometimes crashes when logging in with a different license than previously used. We are investigating this issue.
• In the Depthkit Editor, some clips may not automatically appear in the 3D viewport, Depth preview window, or Color preview window, until triggering a refresh with an event like starting playback. We are investigating this issue.
• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• If recording 3840x2160 footage in Depthkit Core with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• When editing Depthkit Core clips, if the far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity Video Player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Depthkit object’s video player. This will remove the artifact.
• When capturing Depthkit Core clips in the Record context, Depthkit may get stuck in the finalizing state when the take is complete if the GPU-accelerated video encoding has been enabled Preferences immediately prior to clicking the Depthkit Core record button. To work around this, click another portion of the interface before clicking record when toggling this setting.
• When editing Depthkit Core clips, if you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.

Version 0.7.1

Released November 14, 2023

Updates

• New 3D viewport visualizations are available in Core capture & editing contexts, as well as Studio calibration, capture, and editing contexts, including mesh and pointcloud visualization, each with textured, shaded, normals, or sensor contribution texturing.
• Depthkit Studio capture media is now organized into sensor-specific files and folders naming to make it faster and easier to locate clips when reviewing, manipulating in post production, or troubleshooting sensor-related issues.
• Depthkit now generates a report in each Depthkit Studio take folder which logs all sensor events, including dropped frames, organized per sensor, to more quickly and effectively trace sensor-related issues.
• Console logs now specify the sensor serial number to more quickly and effectively trace sensor-related issues.

Bug Fixes

• Fixed an issue while recording where an unstable connection to a sensor would make Depthkit unresponsive and fail to finalize properly. Depthkit now stops and finalizes the capture when the connection to the sensor times out.
• Fixed an issue where Depthkit would crash if a sensor becomes disconnected mid-recording. Depthkit now stops and finalizes the capture when the connection to the sensor is severed.

Known Issues

• On computers set to some localizations which use a comma (”,”) to represent decimal place, Depthkit Core & Studio OBJ exports are written with incorrect formatting. This can be resolved by changing the localization settings in Windows to a localization which uses periods (”.”) to represent decimals, or by replacing all of the commas in the resulting OBJ files with periods. We are investigating this issue.
• The Livestream Resolution Constraints set in preferences do not affect Core livestreams until Depthkit is restarted.
• Depthkit sometimes crashes when logging in with a different license than previously used. We are investigating this issue.
• In the Depthkit Editor, some clips may not automatically appear in the 3D viewport, Depth preview window, or Color preview window, until triggering a refresh with an event like starting playback. We are investigating this issue.
• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• If recording 3840x2160 footage in Depthkit Core with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• When editing Depthkit Core clips, if the far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity Video Player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Depthkit object’s video player. This will remove the artifact.
• When capturing Depthkit Core clips in the Record context, Depthkit may get stuck in the finalizing state when the take is complete if the GPU-accelerated video encoding has been enabled Preferences immediately prior to clicking the Depthkit Core record button. To work around this, click another portion of the interface before clicking record when toggling this setting.
• When editing Depthkit Core clips, if you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.

Version 0.7.0

Released July 10, 2023

New Features

• Depthkit Studio no longer requires green screens! There is no longer a need for green screen and manually created keys in the Depthkit Studio workflow. Depthkit Core and Cinema captures can still make use of greenscreens using the Depthkit Refinement workflow.
• Instant Mesh Fusion See Depthkit Studio captures fused and textured immediately in the Editor preview. Previously, fused results were only seen when exported to Unity.
• Higher Quality Meshes The new Mesh fusion system is an entirely new surface reconstruction algorithm from what is available in Unity. This new system produces higher quality surfaces, with fewer errors and artifacts. It also has fewer adjustment parameters meaning less guess work when tweaking surface settings.
• In-Editor Bounding Box A new bounding box system in the Editor allows for more quickly and precisely segmenting the subject. It can be used for removing unwanted floor and background geometry.
• Automatic Crop and Pack Each perspective’s frame crop and depth ranges are now automatically calculated from the bounding box. This means it's no longer necessary to set the depth ranges or frame crop on every perspective, further speeding up post processing time, increasing quality and reducing export errors.
• Refined Texture Blending Significant improvements to texture blending which include new tools for removing texture spill artifacts between disparate surfaces, for example on captures where subjects are holding their hands in front of their chests. Texture blend settings now carry through to Unity by default, via encoding the Depthkit settings into the JSON metadata.
• High-quality exports Depthkit exported combined-per-pixel format now uses the fused geometry to synthesize new depth maps for each perspective, leading to depth images with fewer holes and crisper edges without the need for Refinement masking. These “Pre-fused” depth maps also reconstruct with higher quality and fewer adjustments needed in Unity.
• Meta Quest Export Preset For Combined Per Pixel Video exports, users can select a Meta Quest preset to configures newly added advanced encoding controls optimized for Meta Quest playback. Additional controls include H.265 and Color Range.
• High Resolution Video Exports with HEVC (H.265) Depthkit can now export Combined Per Pixel Video sequences in HEVC (H.265) format, which increases the maximum export resolution up to 8192x8192, which is 4x the size of the previous AVC (H.264) limit of 4096x4096.
• Support for Limited color range video exports Depthkit can now export both Full (PC) and Limited (TV) color range H.264 videos, and Limited (TV) color range H.265 videos, which may be necessary to properly render or some devices.
• Instant Textured Geometry Sequences Depthkit can now export Studio captures as Textured Geometry sequences with a single OBJ and a single texture per frame. This format is compatible with Arcturus HoloEdit compression, as well as various visual effects software. This feature replaces the previous cumbersome workflow of using Unity Mesh Export package.
• Recordings Library Improvements The Recordings Library now displays Depthkit Studio captures as single entries, where they were previously recorded as a separate entry per perspective. Recordings in the library can now be deleted and renamed, allowing for better media management.

Bug Fixes

• Fixed an issue where the ‘Cancel Export’ button would fail to cancel the recording.
• Fixed an issue where the record buffer visualization would remain unchanged as a Depthkit recording finalized and the buffer would write to disk.
• Fixed an issue where Depthkit Core and Cinema Combined Per Pixel exports would contain a misplaced bounding box.

Known Issues

• The Livestream Resolution Constraints set in preferences do not affect Core livestreams until Depthkit is restarted.
• Depthkit sometimes crashes when logging in with a different license than previously used. We are investigating this issue.
• In the Depthkit Editor, some clips may not automatically appear in the 3D viewport, Depth preview window, or Color preview window, until triggering a refresh with an event like starting playback. We are investigating this issue.
• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• If recording 3840x2160 footage in Depthkit Core with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity Video Player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Depthkit object’s video player. This will remove the artifact.
• When enabling GPU-accelerated video encoder under Preferences, and then immediately clicking the Depthkit Core record button, the user may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264 MP4 so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4K display.

Version 0.6.2

Released March 13, 2023

Bug Fixes

• Fixed a bug in Depthkit Studio that would lead to potentially losing depth frames during successive recordings where frames were stored in the backlog.
• Added recording diagnostic display for Depthkit Studio during finalization of a capture while frames remain in the backlog being written to disk.

Known Issues

• When the record buffer fills all the way while capturing in Depthkit Core’s Record context, the record buffer visualization may remain unchanged as the recording finalizes and the buffer writes to disk, and will only refresh once the recording is finished finalizing. This appears to only be a UI issue, and does not affect the media being sucessfully saved in the background.
• In the Depthkit Editor, some clips may not automatically appear in the 3D viewport, Depth preview window, or Color preview window, until triggering a refresh with an event like starting playback. We are investigating this issue.
• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• If recording 3840x2160 footage in Depthkit Core with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity Video Player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Depthkit object’s video player. This will remove the artifact.
• When enabling GPU-accelerated video encoder under Preferences, and then immediately clicking the Depthkit Core record button, the user may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264 MP4 so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4K display.

Version 0.6.1

Released January 3, 2023

Features

• Fusion Lite Rendering: In addition to pointcloud rendering, Multicam Calibration and Streaming now supports a solid mesh rendering mode. This rendering method performantly combines meshes and textures together for any arbitrary camera positions, making it ideal for assessing in-progress calibration quality live during calibration. Depthkit uses a similar approach to the Studio Lite renderer found in the Depthkit Studio Expansion Packages for Unity.
• Infrared & Edge-Detection Diagnostic Views: When Fusion Lite rendering is enabled, the user can now switch between RGB, IR, and edge-detection texturing modes to identify misalignment based on the current calibration quality, as well as isolate issues from misaligned sensors.
• Sensor Position, Sensor Link, and Sampled Marker Visualization: Multicam Calibration and Streaming now optionally shows the positions of each sensor in the 3D Viewport, as well as the active and inactive links which compose the calibration. These diagnostics provide valuable feedback when capturing calibration samples.
• Smartphone Remote Control interface (OSC): Depthkit can now be controlled remotely using a smartphone during calibration and capture. This allows for significantly faster calibration from within the volume, especially for scenarios requiring solo operators. Additionally this update allows other applications to control Depthkit capturing for use within interactive installations or tradeshow environments. The update is powered by a new OSC (UDP) command API for triggering remote events during calibration and capture. Users can easily also customize the port number for the OSC connection.
• Viewport Updates: Recording, Livestreaming, and Sync status indicators have been added to the Multicam Calibration & Streaming 3D viewport for immediate feedback. A floor plane and skybox have been added to better orient the user. Calibration sample markers can optionally be hidden.
• Sensor Configuration Updates: Depthkit now reports sensor information in the Sensor Configuration pane, like firmware version, sync port connections, and sync role, which can speed up sensor troubleshooting. Depthkit now also warns against old and mismatched firmware among connected sensors.
• Status Feedback: Added flashing activity buttons during Depthkit Studio Calibration and Streaming to better indicate the current application state. Disabled other buttons while capturing and sampling to reduce chance for user error.
• Improved Sensor Sync Indicators: Depthkit now shows more informative feedback about the sync status of the sensors for faster troubleshooting.
• Clarified interface to indicate that capturing a second Floor Sample will entirely replace any previous floor samples.
• Depthkit Studio sensor information panels now default to collapsed to reduce visual clutter.
• Added direct text-entry for Maximum Livestream Resolution constraints in Depthkit preferences, allowing the user to specify precise values.
• Added the ability to edit the livestream resolutions while the livestream pipeline is running, and includes a warning to update the metadata at the livestream endpoint after making changes to the Livestream resolution.
• Updated a warning message about a missing DLL in the console to clarify that it only applies if the user intends to use a Microsoft Kinect v2.

Bug Fixes

• Fixed an issue where Depthkit Studio would crash when creating a Depthkit Studio calibration with a large number of calibration samples.
• Fixed an issue with Depthkit Core that caused a few frames to be dropped at the beginning of a recording.
• Fixed an issue during calibration where sensors placed only with a floor sample would lose their positioning upon capturing the first calibration sample.
• Addressed an issue where users could begin a Depthkit Studio capture before sensor streaming was established, which would lead to a broken state.
• Opening a new Depthkit Studio project after one has previously been loaded no longer leads to a state where sensor streaming cannot be established.
• Addressed underlying issues with GPU texture synchronization on some GPUs which may have led to occasional data corruption.

Known Issues

• When the record buffer fills all the way while capturing in Depthkit Core’s Record context, the record buffer visualization may remain unchanged as the recording finalizes and the buffer writes to disk, and will only refresh once the recording is finished finalizing. This appears to only be a UI issue, and does not affect the media being sucessfully saved in the background.
• In the Depthkit Editor, some clips may not automatically appear in the 3D viewport, Depth preview window, or Color preview window, until triggering a refresh with an event like starting playback. We are investigating this issue.
• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• If recording 3840x2160 footage in Depthkit Core with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity Video Player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Depthkit object’s video player. This will remove the artifact.
• When enabling GPU-accelerated video encoder under Preferences, and then immediately clicking the Depthkit Core record button, the user may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264 MP4 so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4K display.

Version 0.6.0

Released July 25, 2022

Updates

• High Resolution Live-Streaming: Depthkit Livestreams resolution is now calculated based on the color resolution of the sensors, and can be restricted to user-defined resolution constraints in Preferences.
• External Sync: Multi-sensor capture now supports sync provided by an external sync generator, eliminating drift from other simultaneous recordings like audio or Cinema cameras. See Sensor sync for more details.
• Custom Export Resolution & Bitrate: The Depthkit Edit context now supports user-defined resolution contraints for (Multiperspective) Combined-Per-Pixel video and image sequence export formats, as well as user-defined bitrate for (Multiperspective) Combined-Per-Pixel video export formats.
• Automatic Multi-Row Formatting: Multiperpective Combined per Pixel (CPP) export formats now automatically are formatted into multiple rows to optimize for export resolution constraints. This replaces the multi-row stacking Python script workflow.
• Improved Export Resolution Calculation: Depthkit now factors the resolution of each component sensor into the scaling of Multiperpective Combined per Pixel (CPP) export formats, improving resolution and reducing aliasing artifacts.
• Automatic Backup Project JSON: Depthkit automatically creates a backup of the project JSON, enabling users to restore their projects after a failure like power loss.
• Improved performance during multi-sensor capture.

Bug Fixes

• Fixes an issue where, while setting the manual exposure of the Azure Kinect, the shutter speed didn't change when adjusting between values 1/125 and 1/250.
• Fixes an issue where a project would fail to load if portions of the project JSON were corrupted.
• Fixes an issue where exporting multi-sensor captures with Refinement enabled in the Multiperpective Combined per Pixel (CPP) Video format would fail.
• Fixes an issue where multiple sensors would interfere with each other during multi-sensor capture.

Known Issues

• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• If exporting a combined-per-pixel image sequence with a select shorter than a few seconds, your first frame may be blank. To work around this issue, increase the duration of your select slightly and export a new sequence.
• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264 MP4 so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.12

Released March 9, 2022

Updates

• Studio Calibration interface improvements, including adjusted calibration metric calculation and renamed & simplified metrics and tools. - See updated Calibration documentation.
• Depthkit user interface and navigation updated - All panels are now left aligned. Hovering over navigation icons will display a tool tip with the panel name.
• A Depthkit project can be launched from the command line by specifying the project path as an argument. If you are signed into your license, the launcher window will be skipped and Depthkit will run with the project loaded.

Bug Fixes

• Fixes an issue with the “Report a bug” menu where the link was broken.
• Fixes an issue where filtered points were occasionally contributing to the Calibration Error Metric computation.
• Fixes an issue with the Calibration panel viewport appeared blurry in some cases.
• Fixes an issue with the blending of overlapping calibration samples clipping each other.

Known Issues

• Some OBJ Background Plates may only export the PNG texture and may fail to export the OBJ file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• When setting the manual exposure of the Azure Kinect, the shutter speed slider does not change when adjusting between values 1/125 and 1/250.
• If exporting a combined-per-pixel image sequence with a select shorter than a few seconds, your first frame may be blank. To work around this issue, increase the duration of your select slightly and export a new sequence.
• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264 MP4 so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.10

Released July 6, 2021

Updates

• Improved Multicam Calibration tools & metrics.
  • Displays the current values of Sample Filtering, Angle of Incidence Threshold, and Sample Deviation Threshold filters.
  • Added 'Calibration Complete/Incomplete' indicator to determine if any of the sensors are orphaned in the current calibration.
  • Added 'Filtering Breakdown', which specifies how many samples each filter is excluding, as well as a tooltip to indicate which filter(s) are excluding each filtered sample.
  • Sampled Volume more accurate for different-sized volumes.
• Improved Cinema Calibration metrics are now applied to new and existing Cinema calibrations.
• Added 'Low Disk Space' warning to Multicam Recording context. Recording will automatically stop if there is less than 1GB of free space available on the drive.
• Reduces the time taken for the user interface to refresh when making changes in the Multicam Calibration context.

Bug fixes

• Fixes an issue where exporting an unrefined clip containing a video from a Cinema camera mounted 180 degrees off-axis to the sensor would cause Depthkit to crash.
• Fixes an issue where Depthkit would crash when adjusting filters in Multicam the calibration context.
• Fixes an issue where Depthkit would freeze when stopping a Multicam recording.
• Fixes an issue where some of the Cinema calibration chart cross points would be detected at multiple depths.
• Fixes an issue where the Depth Preview window would be partially hidden by the sidebar.

Known issues

• Some OBJ Background Plates may only export the png texture and may fail to export the obj file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• When setting the manual exposure of the Azure Kinect, the shutter speed slider does not change when adjusting between values 1/125 and 1/250.
• If exporting a combined-per-pixel image sequence with a select shorter than a few seconds, your first frame may be blank. To work around this issue, increase the duration of your select slightly and export a new sequence.
• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.9

Released April 8, 2021

Updates

• Refinements to the multi-sensor calibration interface with the goal of reducing the chance of clearing a calibration unintentionally.

Known issues

• Some OBJ Background Plates may only export the png texture and may fail to export the obj file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• When setting the manual exposure of the Azure Kinect, the shutter speed slider does not change when adjusting between values 1/125 and 1/250.
• If exporting a combined-per-pixel image sequence with a select shorter than a few seconds, your first frame may be blank. To work around this issue, increase the duration of your select slightly and export a new sequence.
• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.8

Released March 26, 2021

Bug fixes

• Resolved a bug in the Depthkit Studio export controller that caused crashes on export for multi-perspective clips with linked Cinema files.
• Updated the visual studio redistributable version shipped with the Depthkit installer, resolving a vcruntime error when installing Depthkit.

Known issues

• Some OBJ Background Plates may only export the png texture and may fail to export the obj file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.
• When setting the manual exposure of the Azure Kinect, the shutter speed slider does not change when adjusting between values 1/125 and 1/250.
• If exporting a combined-per-pixel image sequence with a select shorter than a few seconds, your first frame may be blank. To work around this issue, increase the duration of your select slightly and export a new sequence.
• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.7-17

Released January 19, 2021

Updates

• Advancement to the multi-sensor calibration error metrics with the addition of the Sampled Volume. This represents the volume of successful samples captured in the 3D space. This value is represented as cubic meters.
• Introduction of the Angle of Incidence Threshold, a multi-sensor calibration setting that allows you to increase the accuracy of the calibration by filtering out samples captured at a sheer angle. This slider default is 50 degrees.

• Refinements to multi-sensor calibration chart detection, specifically the addition of the normal generation for each sample. This will contribute to your calibration success in combination with the newly introduced Angle of Incidence Threshold.
• Introduction of the Sample Deviation Threshold slider. This slider will assist in filtering out samples that are too noisy or blurry. This can be particularly helpful in removing a sample that was created when the calibration chart was not completely stable.

Bug fixes

• Fixed a bug that caused a crash while recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33 while GPU acceleration was enabled in Depthkit preferences.
• Fixed a bug that caused synchronization issues on export when linking a Cinema clip within a Depthkit Studio project.
• Fixed a bug that flickering artifacts on export when linking a Cinema clip within a Depthkit Studio project.
• Fixed a depth reprojection bug that may have impacted some Camera Pairings in the Depthkit Cinema workflow.
• Resolved crashing when exporting or recording with hardware accelerated video encoding enabled. This will be particularly helpful for those using computers with AMD GPUs, where the issue was most prominent.
• Fixed a bug that caused manually specified sensor gain values to persist even after setting the sensor configuration to manual exposure in the sensor video settings.
• When capturing samples in the Multicam tab, you may have found that the Sample Filtering slider was cropped out of the Extrinsic Calibration interface, making it difficult to interact with the slider. This is now resolved.
• Resolved an issue that occasionally caused the Sensor Configuration panel to be hidden if disconnecting/reconnecting a depth sensor.
• Resolved an issue that caused occasional crashing when clicking the "Stop Sampling" button during multi-sensor calibration.

Version 0.5.7-8

Released October 16, 2020

Updates

• Introduction of automatic refinement. With this update, activate the power of Depthkit's Refinement Algorithm without the requirement of a Refinement Mask.
• Update to the sensor calibration pipeline to detect calibration markers via the infrared data instead of color data, for increased calibration accuracy.

📘

Calibration markers reprint

This update may require you to reprint your calibration markers if you find that they are not being successfully detected in Depthkit.

Known issues

OBJ Background Plate

• Some OBJ Background Plates may only export the png texture and may fail to export the obj file. This may happen in cases where your in and out points are very close to each other on the timeline. To work around this, please readjust or expand your timeline selects and re-export.
• Some OBJ Background Plates will export with an incomplete mesh when the Refinement Algorithm is enabled. In these cases, your mesh may be missing geometry. To work around this, please readjust adjust the crop parameters slightly and re-export.

Calibration

• Importing a Depthkit Cinema Camera Pairing into a Depthkit Studio project with an existing multi-sensor calibration may interfere with the project's multi-sensor extrinsic data. To work around this, return to the Multicam tab and click Continue Calibrate. This will snap the existing sample data back into place.
• If your synchronization cables are not properly configured or one or more sensors have become disconnected from the synchronization cables while in the Multicam workspace, the sensor Sync Mode will default to Standalone. You may also see a warning in the console stating that your sync cables are not properly connected. If this happens, ensure that all synchronization cables are securely connected as described here.

Capture

• When setting the manual exposure of the Azure Kinect, the shutter speed slider does not change when adjusting between values 1/125 and 1/250.
• When transitioning from calibration to recording in the Multicam tab, it may take a few moments for the interface and 3D viewport to launch. Wait a few minutes for the interface to load fully before recording your first take. Clicking record immediately may increase the chance of dropped frames at the start of the clip.

Exporting

• When exporting with more than 5 sensors , you may run into resolution limitations when exporting Multi-perspective Combined Per Pixel video. To work around this issue, export as a Multi-perspective Combined Per Pixel image sequence and encode using an external video encoder such as Adobe Media Encoder or FFMPEG.
• In the export panel, your desired export format selection may not be saved between clips.
• When exporting OBJ Background Plates, you may find that some exported OBJ Background Plates are only exporting small areas of the subject. This issue may be more common when the clip has Refinement enabled. To work around this issue, adjust your crop parameters and re-export.
• When exporting an OBJ sequence, you may find that an incorrect color texture was exported. To work around this issue, relaunch Depthkit and locate your intended take for export. Export the desired take without previewing or loading other takes.
• Some visual artifacts have been seen in Multi-perspective Combined Per Pixel exports when exporting on a computer using an AMD GPU. These artifacts may appear as a banding artifact between the sensor perspectives, as seen below. This issue is still being investigated. If you run into this, try moving your project to a different computer to export using a different GPU.

Interface

• When launching switching between Depthkit projects, the 3D viewport may not have cleared from the previously opened project. This will clear as soon as an action is made in the interface or a new clip is loaded into the 3D viewport.

The Azure Kinect

• Skin Subsurface Scattering. We’ve discovered that one of the biggest causes of error in captures is that Azure Kinect does not measure human skin as accurately as other surfaces. Under the hood, Azure Kinect uses a technology called Time of Flight, which involves pulsating infrared lights not visible to the naked eye. Much like Sonar or Echolocation for sound, these light pulses create patterns that illustrate the contours of surfaces to estimate depth. Skin is semi-translucent (ever shine a flashlight through your hand?), and therefore the pulses are seen slightly set back from the actual surface. We are working with Microsoft on a way to recognize skin and compensate for this physical phenomenon in software. This will result in better alignment on skin, most importantly on faces and hands.
  Read more about the issue on Azure Kinect Github.
• Global Depth Bias. We’ve also discovered another related issue that we raised with Microsoft. The Azure Kinect sensors will always measure surfaces anywhere from 2mm to 10mm further away from the true depth location, no matter the material. Microsoft is not sure why this is happening and several of their engineers are on the case. We imagine this can be fixed via a sensor recalibration step that measures the offset and compensates for the bias in software. We are collaborating with Microsoft to build this utility which will enable you to recalibrate your sensors and increase the quality of your captures.
  Read more about the global sensor bias on Azure Kinect Github.
• Azure Kinect Hardware Misalignment
  Similar to the above issue, sensors have also been observed to have varying degrees of error between depth and color cameras built into the firmware calibration from the factory. This will show up as ghosting artifacts in the Depthkit Studio captures. It can also be solved with the sensor recalibration step discussed above. While we work on getting that recalibration utility ready, we’ve created this guide to diagnose the misalignment of your sensors. Microsoft has agreed to replace sensors on a case by case basis, so if you find that your sensors are suffering from this issue please reach out to us!
  Read more about the factory calibration errors on Azure Kinect Github.

Version 0.5.6

Released 7/30/20

Updates

• Increased capture performance! In this release we have reduced GPU and CPU overhead related to drawing the 3D viewport. What does this mean for you? If you were among those who minimized their Depthkit window to reduce dropped frames, it means that this work around is no longer required. In addition, those capturing on lower powered computers will experience improved performance.
• Updated lens distortion system. This enables Depthkit to use every bit of depth data available, ensuring no information is clipped in your volumetric footage.
• Introduction of a new and improved crop interface for interactive crop selection. This is available in the Edit window when you have enabled Refinement for your clip.

*[Depthkit Cinema] Improved user feedback and error messages during Camera Pairing.
*[Depthkit Cinema] Introduction of the texture alignment that can solve for minor misalignment between the color and depth data for a Cinema Capture. This will be helpful if solving for the slight misalignment between the sensor depth and color found when using some Azure Kinects that experience this known issue.

Bug fixes

• For those facing a crash when trying to import a legacy project, this issue is resolved!

Known issues

• If exporting a combined-per-pixel image sequence with a select shorter than a few seconds, your first frame may be blank. To work around this issue, increase the duration of your select slightly and export a new sequence.
• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.4

Released 3/5/20

Updates

• Introduction of the Pairing Evaluation feedback panel for Depthkit Cinema in Early Access.
• Improved functionality in the Camera Pairing interface for Depthkit Cinema in Early Access.

Bug Fixes

• Solved a crash related bug that could occur when creating a new project from the file menu.
• Solved an error in the Depthkit interface that caused the disappearance of Depthkit tool-tips in the Record window.

Known Issues

• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.
• When exporting Cinema Capture combined-per-pixel videos above 4k, Depthkit may crash on export. To work around this, export the clip as a combined-per-pixel image sequence.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Recording performance for the Azure Kinect has improved as of v0.4.1, however, you may still experience dropped frames at higher resolutions. See Azure Kinect computer and power recommendations to ensure best performance from the sensor.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.3

Released 1/28/20

Updates

• Introduction of our new Depthkit logo!

Bug Fixes

• Solved a Refinement Algorithm bug in which you may see depth artifacts if your depth range was set to the maximum range.
• Solved an error in the Depthkit interface that caused the disappearance of Depthkit tooltips in the Record window.

Known Issues

• If recording 3840x2160 footage with the Azure Kinect on a computer that uses AMD Radeon driver 19.30.01.33, Depthkit may crash if GPU acceleration is enabled in Depthkit preferences. To work around this issue, go into your Depthkit preferences and disable the GPU accelerated video encoder.

• When exporting Cinema Capture combined-per-pixel videos above 4k, Depthkit may crash on export. To work around this, export the clip as a combined-per-pixel image sequence.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Recording performance for the Azure Kinect has improved as of v0.4.1, however, you may still experience dropped frames at higher resolutions. See Depthkit Core hardware requirements to ensure best performance from the sensor.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.5.1

Released 12/9/19

Version 0.5.1 fixes a bug that you may have experienced when linking refinement masks to takes using external camera footage with Cinema Capture.

Version 0.5.0

Released 11/1/19

Updates

• Support for the Azure Kinect in Depthkit Evaluation, ideal for testing your sensor before subscribing to Depthkit Pro.
• Support for short, five second exports from Depthkit Evaluation, helpful for testing your workflow before subscribing to Depthkit Pro or Depthkit Record.

Bug Fixes

• Solved a crash related issue that would cause Depthkit to crash if the application had been sitting in an idle state in the Edit panel.

Known Issues

• When exporting Cinema Capture combined-per-pixel videos above 4k, Depthkit may crash on export. To work around this, export the clip as a combined-per-pixel image sequence.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:
  1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
  2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.
• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Recording performance for the Azure Kinect has improved as of v0.4.1, however, you may still experience dropped frames at higher resolutions. See Azure Kinect computer and power recommendations to ensure best performance from the sensor.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.4.3

Released 10/1/19

Updates

• Support for the latest firmware for the Azure Kinect. For best results with the Azure Kinect, you can update your sensor firmware by following these instruction.

Early Access Cinema Capture

• User interface improvements to the Cinema Capture interface.
• Introduction of a new and more effective calibration chart for pairing samples.
• Introduction of error metrics for qualitative feedback on the pairing of your camera and sensor.
• Introduction of live detection of the calibration chart in the Pairing window.
• Improved pairing indicators, to visualize pairing quality in the 3D viewport.
• Improved accuracy in calibration chart detection.
• Increased performance during camera pairing for faster calibration chart detection and generation of your Camera Pairing.

Bug Fixes

• Fixed a bug in the Refinement algorithm that introduced pixelated artifacts when the Sharpness value was above 80%.

Known Issues

• When exporting Cinema Capture combined-per-pixel videos above 4k, Depthkit may crash on export. To work around this, export the clip as a combined-per-pixel image sequence.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:

1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.

• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Recording performance for the Azure Kinect has improved as of v0.4.1, however, you may still experience dropped frames at higher resolutions. See Azure Kinect computer and power recommendations to ensure best performance from the sensor.
• If Depthkit sits in an idle state in the Edit window, you may increase the risk of Depthkit crashing when you select a new take from the Library. Don’t worry, thanks to the application’s auto save feature, the crash should not negatively impact your work. To avoid this issue, exit Depthkit if you know the application may be sitting idle for a length of time.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.4.2

Released 8/22/19

Updates

• Introduction of crash reporting. This is highly recommended for users to opt into so our team can solve bugs and crash related issues as quickly as possible.
• User interface improvements to the sensor configuration panel for the Azure Kinect.

Bug Fixes

• Fixed a bug in the Refine panel that caused an inaccurate Sharpness value to occur when relaunching Depthkit.
• Fixed a cancellation bug that caused some refined clips to hang if they were canceled mid export.
• Fixed a timeline bug that locked the in point on the timeline if it was located at the end of the clip.

Known Issues

• When using the Refinement algorithm, you may see blocky artifacts in your depth data if you increase the Fill Amount to 5 or 6 in combination setting the Sharpness to above 50%.
• If your far plane is set too close to you subject to the point where your hue encoded depth range is dark red, you may see artifacts in Unity when using the Unity video player. To avoid this:

1. Before exporting your clips, expand your depth range so the far plane is not as close to your subject.
2. In Unity, use AVPro as your Unity plugin video player. This will remove the artifact.

• Occasionally when launching Depthkit and using the Azure Kinect, the console may report that it "failed to open device" and the sensor may not be recognized. We are working on the solution, however, in the meantime, you can work around this by restarting your computer.
• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Recording performance for the Azure Kinect has improved as of v0.4.1, however, you may still experience dropped frames at higher resolutions. See Azure Kinect computer and power recommendations to ensure best performance from the sensor.
• If Depthkit sits in an idle state in the Edit window, you may increase the risk of Depthkit crashing when you select a new take from the Library. Don’t worry, thanks to the application’s auto save feature, the crash should not negatively impact your work. To avoid this issue, exit Depthkit if you know the application may be sitting idle for a length of time.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.4.1

Released on 7/23/19

Updates

• Increased capture performance for the Azure Kinect.
• User interface updates to the Azure Kinect sensor configuration panel.

Bug Fixes

• Bug fixes related to the functionality of the low performance warning.
• Bug fix that caused Depthkit to freeze if setting the application to full screen when you did not have a depth sensor plugged in to your computer.

Known Issues

• If using an Intel Skull Canyon NUC, you may run into blank recordings if your Depthkit settings are set to GPU accelerated video encoder. For best results on this computer, disable the checkbox for GPU accelerated video encoding under Preferences.

• When you enable GPU accelerated video encoder under Preferences, and then immediately click the record button, you may have a clip that gets stuck in the finalizing state when the take is complete. To work around this, click another portion of the interface before clicking record when toggling this setting.
• If you have removed a refinement mask from one of your takes in the Edit window, you may find that the Export button is inactive. To work around this, click the Disable Refinement Mask button to re-activate the Export button.
• Recording performance for the Azure Kinect has improved in this latest update, however, you may still experience dropped frames at higher resolutions. See Azure Kinect computer and power recommendations to ensure best performance from the sensor.
• If Depthkit sits in an idle state in the Edit window, you may increase the risk of Depthkit crashing when you select a new take from the Library. Don’t worry, thanks to the application’s auto save feature, the crash should not negatively impact your work. To avoid this issue, exit Depthkit if you know the application may be sitting idle for a length of time.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• You may experience issues with the Depthkit interface if you are using a 4k display.

Version 0.3.12

Released on 7/2/19

Updates

• Support for the Azure Kinect, available only in Depthkit Pro.
• Updates to the Depthkit Pro Refinement Algorithm ], providing Pro users with even more power to refine their volumetric video. This update introduces a global improvement to the algorithm as well as the implementation of the Sharpness refinement parameter for reduced depth artifacts. See more on how to use this in the Depthkit Refinement documentation.
• Improved Depthkit file management for easier navigation to your projects from within the Record and Edit windows. See updates in the Library panel.

Bug Fixes

• Bug fix related to broken file paths after relocating a Depthkit project. If you experienced the error “error loading asset: file not found” after moving and relaunching your project, this fix will prevent these broken links for your future projects. If you are running into this issue with an existing project, reach out to [email protected] for assistance.
• Fixed a project management bug, in which there was an application launch delay when creating a new project.
• Fixed a clip management bug, in which there was a delay in loading clips when rapidly selecting takes in the Edit window Library.

Known Issues

• If capturing with the Azure Kinect at high resolutions, you may experience performance issues. See Azure Kinect computer and power recommendations to ensure best performance from the sensor.
• If Depthkit sits in an idle state in the Edit window, you may increase the risk of Depthkit crashing when you select a new take from the Library. Don’t worry, thanks to the application’s auto save feature, the crash should not negatively impact your work. To avoid this issue, exit Depthkit if you know the application may be sitting idle for a length of time.
• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• When recording in low light situations with the Kinect, the sensor may capture at 15 fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.

Depthkit for Visual Effects

Released 6/14/19

For best results when exporting OBJ sequences, make sure you have updated to the latest version of Depthkit.

Updates

• Introduction of OBJ sequences as an export format. Note that this export format is only available for Depthkit Core.
• Introduction of mesh decimation to optimize your sequence exports.

Known Issues

• The polygon normals of the OBJ sequence may need to be averaged when taken into your preferred DCC tool. Continued development is in progress to improve the per vertex normals on export from Depthkit.

Version 0.3.11

Released on 6/13/19

Updates

• Solved an issue in the functionality of the depth range.
• Solved an issue with the refinement mask file path.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• When recording in low light situations with the Kinect, the sensor may capture at 15 fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.

Version 0.3.10

Released on 5/31/19

Updates

• Introduction of combined-per-pixel png sequences as an export format in Depthkit Pro (only). This promotes higher quality exports and is helpful for those taking their Depthkit exports into another video editing tool while avoiding extra video compression.
• Depthkit now tracks available hard drive space. This is helpful when recording on a computer with limited space and will prevent any operating system issues related to low disk space.
• Introduced support for the RealSense D435i.
• Improved user interface for the depth range slider, which is helpful for those working in the minimum depth range.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• When recording in low light situations with the Kinect, the sensor may capture at 15 fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.

Version 0.3.9

Released on 5/15/19

Projects created in version 0.3.9 cannot be opened in previous versions of Depthkit.

Updates

• Introduction of a cancel button for easy export cancellations in the Edit window.
• Automatic assignment of the preferred graphics processor for users with Nvidia discreet graphics cards. This will greatly increase capture performance for those working with this kind of GPU.
• When switching between different kinds of depth sensors, the depth range now automatically updates to the range of the new sensor.
• Improved functionality of take selection and playback in the Edit window.
• Usability refinement of the timeline in the Edit window.
• Improved error warnings for users running into issues in the Edit window.
• The default camera view now accurately reflects the sensor's field of view, allowing for an accurate view of your capture space.

Bug Fixes

• Solved the following compiling issue: "error creating shader from source", which occurred on certain low performance computers.
• Solved export and playback issues on computer with AMD graphics cards where the first frame of the video export would contain a blank frame and scrubbing the timeline occasionally fell out of synchronization between the color and depth inputs.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• When recording in low light situations with the Kinect, the sensor may capture at 15 fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.
• Projects may not load if trying to open a Depthkit project located on an exFat external hard drive. If needing to open a project from an external drive, use NTFS.

Version 0.3.7

Released on 4/18/19

Projects created in version 0.3.7 cannot be opened in previous versions of Depthkit.

Updates

• Continued improvements to timeline functionality.

Bug Fixes

• Solved installer bug that caused an error claiming that Visual Studio .dll files were missing.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• If switching back and forth between senors, the depth range may not reset to the new sensor. This could cause clipping in your recording. To avoid this, simply re-adjust your depth range when you start to record with a new sensor.
• When recording in low light situations with the Kinect, the sensor may capture at 15fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.
• Projects may not load if trying to open a Depthkit project located on an exFat external hard drive. If needing to open a project from an external drive, use NTFS.
• If you are working on a computer with a dedicated GPU, you may need to configure it so Depthkit will run with the most powerful option. See the instructions to select your preferred GPU with NVIDIA and AMD graphics cards.

Version 0.3.6

Released on 3/18/19

Projects created in version 0.3.6 cannot be opened in previous versions of Depthkit.

Updates

• Improved timeline functionality.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• If switching back and forth between senors, the depth range may not reset to the new sensor. This could cause clipping in your recording. To avoid this, simply re-adjust your depth range when you start to record with a new sensor.
• When recording in low light situations with the Kinect, the sensor may capture at 15fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.
• Projects may not load if trying to open a Depthkit project located on an exFat external hard drive. If needing to open a project from an external drive, use NTFS.

Version 0.3.5

Released on 2/25/19

❗️

Updating Existing Depthkit Projects

Version 0.3.5 implements a change in project versioning. Projects created in v0.3.5 cannot be opened in previous versions of Depthkit. Similarly, once you open a project created with a previous version, the project will be converted and will also no longer work in versions prior to v0.3.5.

Updates

• Fixed bug when the enhancement parameters and crop values may not save in the user interface sliders when re-launching Depthkit.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• If switching back and forth between senors, the depth range may not reset to the new sensor. This could cause clipping in your recording. To avoid this, simply re-adjust your depth range when you start to record with a new sensor.
• When recording in low light situations with the Kinect, the sensor may capture at 15fps instead of 30 fps. You will be notified of this in the Depthkit console. If you the console reports "low light activated", your capture has dropped to 15 fps. Once the console reports "low light deactivated", it has reverted back to 30 fps.
• Projects may not load if trying to open a Depthkit project located on an exFat external hard drive. If needing to open a project from an external drive, use NTFS.

Version 0.3.4

Released on 2/5/19

Updates

• Refined sensor configuration for selecting sensor types and customizing capture resolution.
• Fixed bug when projects did not refresh when opening a project via File -> Recent Projects.
• Fixed bug with account badge so your account type can be viewed when offline.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Enhancement parameters and crop values may not save in the user interface sliders when re-launching Depthkit.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.

Version 0.3.3

Released on 1/18/19

Updates

• Increased 3D viewport performance for reduced dropped frames during recording.
• Reduction of minimum depth range. This is helpful for those looking to set very narrow depth ranges.
• Implementation of a refinement preview in Depthkit Record.
• Introduction of the Recent Project menu selection for ease of navigation back to the Depthkit launch menu.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.

Version 0.3.2

Released on 1/7/19

Updates

• Increased performance, with fewer dropped frames during capture.
• With the introduction of the Preferences menu drop down, users can now enable or disable GPU acceleration for switching between the GPU and CPU.
• Toggle timeline playback by playing and pausing your clip with the spacebar.
• Start or stop a recording with a click of the spacebar.

Known Issues

• The window size of Depthkit may impact performance during recording. If experiencing dropped frames, reduce the size of the window for increased performance.
• Low hard drive space may contribute to a lower performance and dropped frames. Keep and eye on your drive space during recording to make sure you computer has extra room.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.

Version 0.3.1

Released on 11/4/18

Updates

_Introduction of Live Preview Enhancement. Gain valuable insight into your depth refinements with a live preview into all depth edits.
_New Depth Refinement Parameters. Enhance your depth data with color and depth contribution values, allowing you to refine your capture based on the source color and depth data itself.
*Increased performance: enjoy fewer dropped frames and increased performance during capture.

Version 0.3.0

Updates

*Introduction of the **Sensor Enhancement Workflow

• Significantly increase sensor video capture quality by refining your depth data.
• Apply an optional custom matte to captures filmed on a green screen to export with clean edges.
• Crop the export frame to take advantage of every available depth pixel.
• Set custom refinement parameters to fill in depth holes and increase your export quality.

Known Issues

• When setting your near plane, the depth range represented in bright red may get clipped when viewed in Unity. When setting your near plane, avoid bright red and stick with dark orange as the representation of the nearest depth range.
• Editing your source color video may create issues if you do not maintain the source codec. If color correcting the video, make sure to export it as H264(mp4) so the app can read the file properly.
• Depthkit v0.3.0 is experiencing a drop in performance on the Dell G3 15 Gaming Laptop, one of our recommended capture machines. If experiencing issues, try you can download Depthkit v0.2.0.

Version 0.2.1

Updates

• Introduction of the Depth Range. Set and preview the depth range before capture.
• Adjust and preview the depth range in the Edit & Export window.

Known Issues

• Depthkit v0.2.1 is experiencing increased dropped frames on lower performance computers. If experiencing this issue, you can revert to v0.2.0.
• When setting your near plane, the depth range represented in bright red may get clipped when viewed in Unity. When setting your near plane, avoid bright red and stick with dark orange as the representation of the nearest depth range.

Version 0.2.0

Updates

• Introduction of the Edit & Export Window
• Experience 3D playback.
• Visualize background subtraction options before export.
• Set in and out points for your desired clip.
  *Live Preview of the Near & Far Planes. Visualize the depth values while you record so you know exactly when in being captured by the depth stream.
  *Increased Depth Range. Experience an extended shooting range with clear indication of when the sensor clips in the near or far plane.
  *Increased Capture Performance. Experience fewer dropped frames during capture.

Version 0.1.2

Updates

• Increased capture stability and performance.
• Take Diagnostics will alert you to any dropped frames that some may experience when working with an older or low powered computer.
• A capture backlog allows you to create flawless recordings even on these low powered machines by having your frames queue up in system memory and are write to disk after recording completes.
• Live 3D preview of your capture, as well as individual displays of your color and depth streams provide complete clarity into what you are capturing and how it will look.
  *Optional Background Subtraction for the Kinect. Once capturing a take with the Kinect, you can now hide or show the environment background via your export settings with the help of body index.
  *Optimized depth data compression. This means efficient and fast processing of your volumetric captures. Now you can go from capture to export to Unity in seconds!
  *In App Video Encoding. Now with hardware accelerated video encoding, there is no need to encode an image sequence with an external tool.
  *Optimized for Unity. The combined-per-pixel video export format allows for direct import into Unity after your takes have been exported.