Watch this tutorial for an in-depth demonstration of multi-sensor capture in Depthkit Studio. Or read our step by step guide below.
Calibration and capture often use different sensor settings. Be sure to change the sensors to the appropriate settings in the Sensor Configuration pane.
- Color Resolution: The performance of your PC and the number of sensors you are using will dictate which resolution you can successfully capture on each sensor without dropping frames.
- Depth Mode: 640x576 Narrow Raw is recommended for most captures, but 512x512 Wide can be used for specific capture scenarios.
- Exposure & White Balance: Uncheck the 'Auto' settings for each of these, then choose a setting which best suits your lighting. Make sure the settings match between sensors to make the textures from each sensor seamlessly blend together. To preview these settings, enter the 'Record' context to see the color feed from each sensor.
- Under Controls, click Start Streaming. This will reveal the Record context of the Multicam workspace.
- Ensure that your cameras are synchronized before capturing by checking that there is one camera as Multicam Controller and the others are Multicam Subordinate. Ensure that they do not say Standalone.
Capturing without proper synchronization will lead to drift or temporal offset, which may render your footage unusable!
You can test to see if there is a sync offset between sensors by having someone enter your capture volume and moving quickly (e.g. swinging their arm around). If the pointcloud from one sensor seems to lag behind, check the sync cables are configured properly.
- Set your camera settings for each sensor:
Manually Setting Uniform Camera Settings
In most cases, you'll get the best results by setting the sensors to have matching exposure and color settings.
- Auto Exposure: similar to camera auto exposure, this parameter will automatically set the exposure time and gain based on the lighting conditions of your surrounding environment. When enabled, the exposure time and gain values will be set for automatically. For best results, disable this for multi-sensor captures.
- Shutter Speed: can be adjusted to control the exposure of the sensor’s color video.
- Gain: in combination with exposure time, gain will increase exposure of the sensor color video. Note that similar to camera gain, it may also increase noise, or grain, in the color image.
- Auto White Balance: similar to auto exposure, this parameter will set the recommended color temperature based on the lighting conditions of your surrounding environment. When auto white balance is enabled, the color temperature slider will be set for you. For best results, disable this for multi-sensor captures.
- Color Temperature: represented in Kelvin, adjusts the characteristic of the light temperature in your sensor color video. Please note that currently, the Azure Kinect color temperature slider values are in reverse of standard color temperature camera settings.
- Brightness: adjusts the overall brightness of the sensor color video.
- Contrast: adjusts the overall contrast of the sensor color video.
- Saturation: adjusts the overall saturation of the sensor color video.
- Sharpness: accentuates fine color detail represented in the sensor color video. This may be most noticeable in high contrast areas of the image.
- Powerline Frequency: a parameter often changed to prevent flickering or banding seen in video that is not compatible with the AC frequency of the surrounding capture space. If you find that you have flickering in your sensor color video, you can change the sensor frequency to match the common AC frequency of your environment. Most countries have an AC frequency of 50Hz, while 60Hz is most common in North America.
- Backlight Compensation: can be enabled if you are shooting in a low or inconsistently lit environment. It can also be helpful when shooting on a green screen to compensate for dim areas on the green screen between the subject and your green background.
- Under Recording Prefix, you can add a custom prefix to the next recording to help stay organized.
- Clicking Record immediately begins writing the captured frames to disk. This will reveal a Diagnostics panel.
The Diagnostics section will inform you of each sensor's performance for the active recording. It is important to keep a close eye on this performance to ensure you are recording with minimal or no dropped frames.
Frame Backlog generated a queue in your system memory to optimize your performance and reduce dropped frames. The Frame Backlog meter represents the maximum amount of frames that can be stored in the backlog. If the meter fills up, the recording is automatically stopped. Depthkit then finalizes the recording by writing all frames to disk before the next capture can begin.
Dropped Frames are non-recoverable data. If you have excessive dropped frames during capture, you may need to reduce the color resolution of each sensor to a resolution which is supported by your hardware.
Dropped frames as red errors.
When the sensors are streaming in the Multicam Streaming and Calibration contexts, the console will report dropped frames as red errors. Though this may appear alarming at first, it is expected behavior, and can largely be ignored. The dropped frames in the Diagnostic panel during recording are a better indicator of your system's performance.
- Click End Recording to stop the multi-sensor capture. If you have a remaining frame backlog, your clip may take a few moments to finalize.
- Click Stop Steaming when you are complete with your takes and ready for processing.
When opening an existing project with many recorded takes, you may get an error that closes Depthkit.
To work around this, make a copy of your project folder (complete with all of the contents), and remove some of the take folders. Open this pared-down project folder instead of the original. The clip names of the removed takes will still appear in the Edit context's Library panel, but show red alert icons indicating that the clips are offline.
Updated 3 months ago