Time-of-Flight depth sensors like the Azure Kinect use the infrared spectrum to map depth. You'll need to factor in certain considerations when you plan your production.
The technology that makes the Kinect work relies on the near-infrared spectrum for illumination. If you use lights containing a lot of infrared content it can disrupt the Kinect, degrading your depth capture.
Recommended Light Types
Unsuitable Light Types
Fluorescent tube lights
LED Light Panels
How to choose your lights
A rule of thumb is that any lights that are hot to the touch (including our sun) generally introduce some near-infrared pollution, making them not ideal for scanning. When in doubt, we always recommend that you test your lights before shooting!
When considering how to light your scene for Depthkit we recommend taking into account the of the experience you are shooting for. Similar to an 'in-camera' look designing your lighting for that context. For example, if you intend to place your subject in VR so that their left side is illuminated by window, then it's best to match that virtual light source with a practical light source, like a large softbox, to the left of the subject to mimic that light.
Avoid the use of transparent or reflective materials
This means glasses, mirrors, windows, shiny leather, waxed jeans, etc. These materials will degrade your depth data.
Most depth sensors rely on a small infrared emitter to visually detect depth of surfaces. This technology can fail if the material properties don't reflect the light back into the camera's lens. For example, it will not be able to detect transparent surfaces like glasses, windows or water. It will also represent reflective surfaces in strange ways – for example if I film something in a mirror, the object will appear to be inside or behind the mirror.
The depth sensor will also occasionally struggle with certain fabrics or materials. Black leather and some kinds of waxed or otherwise treated jeans will just not appear in the data stream. On critical shoots, always test your materials in front of the sensor before shooting.
When choosing patterns or colors in clothing, adhere to all the same constraints that govern a typical film shoot – small repeating patterns can cause moire effects, and if garments are too bright or too dark they will indeed appear bright or dark respectively when filmed with Depthkit.
Depthkit does not record audio. You will need to capture audio with an external device and lay it back into the clip after exporting from Depthkit. For ease of syncing, remember to clap or have a slate at the beginning of your take so there is no doubt about the sync point between your Depthkit export and your audio file.
Updated 6 months ago