Welcome to the Depthkit Documentation Portal!

Depthkit is the first volumetric filmmaking tool designed to empower creators of all experience levels to participate in the cutting edge of immersive storytelling.


Computer Requirements

Depth Sensors

Getting Started

Depth Sensors

Sensor Compatibility

Depthkit is designed to access a variety of hardware to capture volumetric depth information. However in this current beta, we only officially support Microsoft Kinect V2 for Windows. We have shipped experimental support for Intel RealSense, but please use at your own risk and not in a critical setting. We have observed stability issues as well as lower than acceptable data quality.

Supported Sensors include:

  • Kinect V2 for Windows (with Kinect adapter for Windows)
  • Intel RealSense D435 (Experimental)
  • Intel RealSense D415 (Experimental)

Due to the experimental support of RealSense depth cameras, we strongly suggest that you capture with the Kinect until further development.

The Kinect Sensor

Required

Installation of the Kinect For Windows SDK is required to run Depthkit with a Kinect.

The Kinect for Windows V2 / Kinect for Xbox One + Windows Adaptor

These two Kinects are identical in how they function. Since the Kinect has been discontinued, the Kinect for Xbox One is the sensor that is still readily available to purchase. You will just need the addition of the Kinect Adapter for Windows, which usually needs to be purchased separately.

Where to Buy

Don't be fooled by the inflated price of the Kinect from some retailers, it is still easy to find at stores like Amazon and Target. Please note that you will need to purchase the Kinect Adapter for Windows in order to use the sensor with a PC. Check out these locations for a few options on where to purchase.

Kinect for Xbox One
Kinect Adapter for Windows

Amazon has quite a few options for around $70 - $100.

Target also has a reliable stock, although not in store usually.

Range

The Kinect has a range of around .5m and 4.5m (1'8"-14'6".) We also find that based on the brightness of the IR illuminators on the Kinect V2 that when objects are close to the near range, they're often too bright for it to sense.

Best Practice

The quality of the depth data falls off as you step away from the sensor, so the general recommendation is that you fill your depth frame with your subject whenever possible to get the optimal resolution.

Resolution

The Kinect V2 has a depth resolution of 512 x 424 pixels with a field of view (FoV) of 70.6° x 60° resulting in an average of around 7 x 7 depth pixels per degree. Due to the use time-of-flight as the core mechanism for depth capture, each pixel in the depth image contains a real measured depth value (z-coordinate). Read a more thorough breakdown.

Field of View

The depth camera on the Kinect V2 has a FoV of 70.6° x 60° and the color camera has a resolution of 1920 x 1080px with a FoV of 84.1° x 53.8°, resulting in an average of about 22 x 20 color pixels per degree. There is a great interactive representation of the Kinect V2 FoV here.

Body Index

The Depthkit Capture App allows for body index segmentation, or the detection of one or multiple bodies in the depth data. When generating your assets, this acts as a body mask, allowing you to simply hide the background for clean exports of your subjects. This eliminates the need to shoot on a green screen.

It also acts as a non-destructive export setting. For example, you can export a take with the background hidden and then immediately export the same clip with the background included in the clip by simply disabling the Hide Background toggle.

When capturing with Kinect’s body index, you can confirm a body is detected when it appears illuminated in the 3D preview. You will also notice this in the depth preview.

Note that the Kinect cannot detect body index - body index - The detection of one or multiple bodies in the depth data. When generating your assets, this acts as a body mask, allowing you to simply hide the background for clean exports of your subjects. This eliminates the need to shoot on a green screen. if the Kinect is rotated to shoot vertically or if the subject is only partially in the frame.

Verify Kinect Performance

We recommend downloading the Kinect Verifier Tool and running it while your Kinect is plugged in to your computer. For stable performance with Depthkit, all sections will be checked in green if successful, with the exception of the USB Controller. You can disregard the yellow warning here.

What if a warning is reported in the Verify Kinect Depth and Color Streams section?

Make sure you have your Kinect connected via a USB 3.1 (type C) Gen 2 port. This will increase your data transfer rate and improve the performance of your depth and color streams. Learn more about USB ports.

Important

You cannot run the Kinect Configuration Verifier while you are running Depthkit, as you will find that the depth/color streams may not show up in Depthkit when both apps are active. With this in mind, once your Kinect performance is confirmed, close the verifier before you open Depthkit.

Intel RealSense Depth Camera

Depthkit provides experimental support for the RealSense D435 and D415. When we refer to experimental support, this means that we do not recommend relying on these sensors for production.

What RealSense depth camera is right for me?

The primary difference between these two models is the sensor's effective distance. They also have different resolutions, fields of view and the shutter types. Learn more about sensor differences. The D415 also has higher pixel density, making it more accurate than the D435. On the other hand, the D435 has a smaller, min z. This is the distance from the depth sensor to the object being captured. In other words, you can get closer to your subject with the D435.

RealSense D415
RealSense D435

Long Range Sensor

Short Range Sensor

70-degree field of view

90-degree field of view

Rolling Shutter

Global Shutter

Minimum z of 20 cm

Minimum z of 17 cm

Known Issues with RealSense

  • Sensors exhibit a high rate of dropped frames and inconsistent backlog activity.
  • Sensor data quality is very poor.
  • Sensors will occasionally not be recognized by Depthkit. To address this, unplug-and-plug the sensor in again, or restart the computer.
  • Sensors may stop streaming data unexpectedly.
  • If unplugging the sensor from the computer while using Depthkit, you may experience a frozen 3D viewport. You may also see this when switching between RealSense sensors in Record Settings. When this happens do not attempt to record. Instead, restart the app and verify that the 3D preview is active before recording.
  • The 3D preview may freeze during recording. If this occurs, stop the recording and restart the app. If unresolved, you may need to restart your computer.
  • If you unplug your sensor during recording, your 3D preview will freeze and you will need to restart the app. Most likely, your output files will be intact, even if the take does not show up in the library.
  • Some have experienced an error when updating the D435 firmware where the sensor becomes recognized as a D430. Depthkit will not capture with a RealSense that is not a D415 or D435. See what this looks like below.

Up Coming Sensor Integrations

The Depthkit team is speeding ahead on new integrations with high quality sensors. To learn more, reach out to support@depthkit.tv.


What's Next

Capture Location

Depth Sensors


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.