Studio Mesh Sequence export

The Depthkit Studio textured geometry sequence export is a two-step process involving an export out of the Depthkit app, as well as mesh generation from within Unity. Once both of these are complete, you will have a data set ready to be processed in your tool of choice.

What you'll need

This workflow requires a Depthkit Studio capture exported as an MultiPerspective CPP Image Sequence.

🚧

Experimental

This is an experimental Depthkit Studio Expansion package

Unity instructions

In Unity, create a new project with the 3D preset (Built-in Render Pipeline).

In Package Manager, import the following Depthkit packages in this order:

  1. depthkit.core
  2. depthkit.streamingimagesequenceplayer
  3. depthkit.studio
  4. depthkit.studio.meshsequence

Set up the Studio clip with the Depthkit Image Sequence Player

  1. Create an empty game object.
  2. Add a Depthkit clip to the game object, either using the Add Component button or the Component menu, then choosing Depthkit → Depthkit Clip.
  3. Depthkit Clip
    • Change the Depthkit Clip Player setting to Image Sequence Player.
    • Select the metadata file for the asset to export.
    • Under Advanced Settings, check the disable poster frame check box.
  4. Add a Depthkit → Studio → Built-in RP → Studio Look component.
  5. Playable Director:
    • Create a Timeline Object.
    • With the timeline object selected, in the Timeline window, add a track to the timeline "+" → Unity.StreamingImageSequence → Streaming Image Sequence Track.
    • In the project window, navigate to the folder that contains the image sequence, this folder should be placed under the StreamingAssets folders for it to be processed by the streaming image player. Drag the folder to the the Streaming Image Track created in the previous step.
    • Click on the track in the time line and verify in the inspector that the fps is 30 and frame range reflects the number of images in your folder.
    • Select the game object, link the timeline created to the playable field in the Playable Director component.
    • A new binding is created and needs to be linked, unfold the bindings foldout and link the game object to the binding for the Streaming Image Sequence Track.

Now your clip is setup to play in Unity.

Configure reconstruction settings

  1. Follow the general Depthkit Studio best practices for setting your reconstruction settings. In addition to this, special consideration is needed for a few parameters.
  2. Volume Density: higher density will give the mesh more detail, but also generate larger files. Recommended range: 150-200 voxels/meter.
  3. Enable Surface Smoothing: especially important at higher voxel densities, surface smoothing can remove unwanted "floating" geometry and high frequency noise. Try to balance out noise reduction while avoiding too much loss of detail. Recommended range: 0.4 - 0.6.
  4. Texture blending: to preview the texture blending that you will get with your exported asset, undisclose the Experimental Texture Settings panel in the Depthkit Studio Mesh Source.
    • Reduce View Dependent Color Blend Weight to its lowest value (0.001).
    • Increase Surface Normal Color Blend Weight to its highest value.
    • Leave everything else at the default values.
  5. Depth Bias Compensation: set your depth bias such that the texture aligns as well as possible on the model (after following the above texture blending steps). Depth bias will alter the surface geometry which in turn affects how the texture is projected onto the surface. Getting this right at this stage is crucial to ensure good quality texturing.

Configure Geometry Export settings for Textured Geometry Sequence

  1. Add a Depthkit → Studio → Mesh Sequence → Geometry Sequence Exporter to the game object. This will add all the necessary components to the game object.
  2. Ensure Export Textures is checked.
  3. Choose your desired texture format.
  4. Choose your desired export units, file prefix, and indexing mode based on your workflow.
  5. Click the Output Path button to choose a directory to save the mesh + texture sequence into.

Exporting the geometry sequence

  1. Enter Play mode.
  2. Click the Start Export button on the Depthkit Geometry Sequence Exporter component.
  3. Wait for the export to finish.
  4. Once done, you will see a message in the console stating Capturing Finished, and the timeline playhead will return to the beginning. At this point you can exit Play mode.