Lens Correction Profile

Prior to pairing your depth sensor and camera, you will need to correct for the lens distortion of your camera. You can do this by creating a custom lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination.. If you've ever used Lens Corrections in Lightroom or Photoshop, this process can be pretty familiar. Essentially, it is an operation to remove any level of lens distortion. However, instead of applying it as a post-process, we will create a lens correction profile at the start.

The following steps are a best practice for a successful lens profile for Depthkit. See here for a complete guide on creating a lens profile.

This lens profile is specific to your camera and lens combination. It cannot be shared between similar lens and camera rigs. The good news is, once you generate this profile for a given camera and lens combination, you can use it repeatedly for all Depthkit footage captured with that rig if you lock the lens at the focal length that you generated the lens correction file with. If you change the focal length or video resolution, you will need to generate a new lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination. .

Choosing Your Production Zoom Amount

Before understanding the distortion of the lens, we need to decide on the level of zoom that you are going to use for your final shoot. Our intention is to match the video camera's field of view to the depth sensor field of view to ensure that they overlap as closely as possible. Open Depthkit to the Record Tab and compare what you see in the Depth Preview at the bottom left corner of the viewport to the field of view of your camera.

📘

For Azure Kinect Choose Final Field of View You Intend To Use

If you're using the Azure Kinect, settle on the field of view that you intend to use for your production and select it in the Stream Configuration panel before continuing. We recommend a depth mode of 640x576 Narrow raw for most capture environments.

Looking at Depth Preview in the bottom left corner of the viewport, tilt your rig until the top of the frame aligns to a recognizable feature in the scene. Now looking at your cameras viewfinder or LCD zoom out until you can see that same feature in the camera preview.

🚧

Test Your Camera Aspect Ratio With A Test Recording

Occasionally, the aspect ratio that you see in your camera viewport will not match the video that your camera will produce. For example, your viewport may be 4:3 and your recorded video could be 16:9 so when you record little black bars appear at the top and bottom of the frame. Test a recording to ensure that your LCD preview matches your recorded video.

Capturing a Lens Correction Profile

There are several ways to generate a lens correction profile.

We recommend using the Adobe Lens Profile Creator, a free utility that can provide a quick and easy way to create a lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination.. See the following steps for creating a lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination. for Depthkit using Adobe's Lens Profile Creator.

📘

If you are unable to access the Adobe Lens Profile Creator link, reach out to [email protected] for assistance.

Camera Set Up

For best results

  • Keep your camera at manual exposure for consistency during the capture.
  • Set you ISO to 400 or less to avoid excess noise that may impact the calibration.
  • Shoot at an aperture of f/11.

Capturing The Calibration Chart

A good lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination. begins by capturing a calibration chart. This is a specific chessboard pattern available in the Adobe Lens Profile Creator. There is a folder of PDF charts that come with the application.

If printing out the calibration chartcalibration chart - A chessboard pattern used to calibrate multiple devices, either for a camera/lens combination or a camera/depth sensor combination. follow Adobe's best practices.
If capturing the calibration chartcalibration chart - A chessboard pattern used to calibrate multiple devices, either for a camera/lens combination or a camera/depth sensor combination. from a laptop or display, continue with these steps.

  1. Download the Adobe Lens Profile Creator.

  2. Select your calibration chartcalibration chart - A chessboard pattern used to calibrate multiple devices, either for a camera/lens combination or a camera/depth sensor combination. . In the Adobe Lens Profile Creator folder, find the calibration charts folder. Typically, these charts are printed. However, you can also display the chart on a laptop or monitor for ease of capturing. Please use the workflow that best works for you. Select a calibration chart that will fill your screen as much as possible. For a 16:9 screen we find that the Landscape ARCH_C - 18in x 24in (Square print dimension 108 Pts, Version 9 x 15) works quite well.

The precise one doesn't matter very much as long as it fills your screen.The precise one doesn't matter very much as long as it fills your screen.

The precise one doesn't matter very much as long as it fills your screen.

  1. Open the chart PDF and make it full screen.

  2. Avoid glare on the screen. Increase your screen brightness and turn off any surrounding lights, particularly lights that are behind the camera or create glare on the screen. The light coming from your display will be plenty to generate a good lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination..

  3. Once you calibration screen is ready, set up your camera on a tripod, roughly at the same level as the chart. The chart should be perpendicular to your camera.

The screen is a few feet in front of the camera, oriented to face the camera.The screen is a few feet in front of the camera, oriented to face the camera.

The screen is a few feet in front of the camera, oriented to face the camera.

  1. Set your camera to manual exposure and stop it down so that the whites of the chart are not too 'clipped' or over-exposed. Ensure that you are using an aperture or f-stop of f5.6 or above. Focus your lens on the calibration chart.

  2. Now you are ready to capture the calibration screen. The goal is to capture a single video of the chart placed across the frame in a grid like pattern. Think of your video frame as a grid, starting at the center and then positioning the calibration chart as close to the edges of the frame as possible. Make sure no part of the calibration chart is outside of the frame. With this in mind, you can record a video and adjust the camera so the calibration chartcalibration chart - A chessboard pattern used to calibrate multiple devices, either for a camera/lens combination or a camera/depth sensor combination. fits into each grid section. We will need to pull still images from this video for generating the profile. The reason why we capture a video first is to reduce the error that may be caused from differences in frame size between a still image and a video when using a camera that introduces crop.

Depending on the size of your calibration chart, your frame maybe be broken into nine sections or more. The below example captured in a 4x4 grid pattern to fill the frame with the calibration chart.

Place the checkerboard in all quadrants of the screen.Place the checkerboard in all quadrants of the screen.

Place the checkerboard in all quadrants of the screen.

  1. Now you have captured a single calibration chart video. Next, grab single frames representing sections of the grid. You can grab these frame quickly using Quicktime X, Windows Media Player, or Adobe Media Encode.

Extracting Frames From Video

📘

Exporting Frames From Your Video

We recommend exporting your images as TIFF or JPEG files for best results.

To extract frames in Adobe Premiere:

  1. Import the video into Premiere.
  2. In the timeline, position the play head over the frame that you would like to extract and hit Ctrl+Shift+E (or Shift+E on a Mac) or click the camera icon in the project panel. This will bring up an export dialog to export the desired frame.

To extract frames in Quicktime X (OS X):

  1. Select the desired frame. Use the arrow keys to select the exact frame that you want.
  2. Hit Command+C
  3. Hit Command+N in Preview and save the new image. Repeat for the remaining sections of the grid.

To extract frames in Windows Media Player (Windows):

  1. Select the desired frame and hit Control+I.
  2. When you see a Save Captured Image dialog box, save the image and repeat for the rest of the grid sections.

Generating the Lens Correction Profile

Processing Calibration Chart Images

  1. Open the Adobe Len Profile Creator tool and create a new project.

  2. Under File, click "Add Images to Project".

Add Images to ProjectAdd Images to Project

Add Images to Project

  1. Type in your camera and lens model name under Display Names at the top of the panel on the right. It is important to be specific here, including the full model names. This profile will be reusable for this specific camera and lens combination.

Calibration Settings

  1. In the Calibration Menu in the panel on the right under Checkerboard Info, specify your version, row and column values. This refers to the rows and columns listed at the bottom on your Adobe calibration chart image. Specify the Screen Dimension in pixels, also listed on your calibration chart.
Input the values from the bottom of your chartInput the values from the bottom of your chart

Input the values from the bottom of your chart

  1. Specify your Print Dimension using the ruler tool, found at the bottom right corner of the preview screen. For this value, measure a single checker square that is the smallest one in your image set using the ruler tool in the bottom right corner of the Adobe Lens tool. Measure the width (or height) of this square making sure your line is straight across and not angled diagonally. Input the value as your Screen Dimension in the corresponding dialog.

Disable Chromatic Aberration Model and Vignetting Model to reduce processing time.

  1. Switch from the Calibration tab to the Advanced tab in the menu on the right. Set the Rectilinear Lens Model to "Five-Parameter Radial and Tangential Distortion". However, if you are using a very wide lens and are prompted to "change your lens model or disable outlier images" you may have to leave this setting on automatic.
Set lens model to Five-Parameter Radial and Tangential DistortionSet lens model to Five-Parameter Radial and Tangential Distortion

Set lens model to Five-Parameter Radial and Tangential Distortion

You are now ready to click Generate Profile in the top right corner of the interface. This may take a few moments. You will be prompted to save your profile. The result is a camera/lens specific lens correction profilelens correction profile - A profile that solves for the lens distortion of a given camera and lens combination., or .lcp file, for use in Depthkit in the Camera Pairing workflow in Depthkit.

A calibration being processedA calibration being processed

A calibration being processed

Lens Correction Profile Troubleshooting

What if the generation of the Len Profile is unsuccessful?

  1. Make sure the calibration chartcalibration chart - A chessboard pattern used to calibrate multiple devices, either for a camera/lens combination or a camera/depth sensor combination. is in focus in your images.
  2. You may have images in which the calibration chartcalibration chart - A chessboard pattern used to calibrate multiple devices, either for a camera/lens combination or a camera/depth sensor combination. is not detected. It is fine for a few images to be disabled due to a failure to detect, but if the majority of your images are unsuccessful, you may need to re-shoot the image set.
  3. With some wide angle lenses, you may have a successful generation of a Lens Profile by switching the Rectilinear Lens Model to Auto. This isn't the best practice, but if you are running into errors (as seen below) and you don't have time to take new images, this may be a solution.

In this section