Studio Mesh Sequence export

The Depthkit Studio textured geometry sequence export is a two-step process involving an export out of the Depthkit app, as well as mesh generation from within Unity. Once both of these are complete, you will have a data set ready to be processed in third-party tools, like Arcturus' HoloSuite.

What you'll need

This workflow requires a Depthkit Studio capture exported as an MultiPerspective CPP Image Sequence.

Unity instructions

In Unity, create a new project with the 3D preset (Built-in Render Pipeline).

In Package Manager, import the following Depthkit packages in this order:

1. depthkit.core
2. depthkit.streamingimagesequenceplayer
3. depthkit.studio
4. depthkit.studio.meshsequence

Import your Depthkit image sequence, and place it within a 'StreamingAssets' folder within your 'Assets' folder so it's properly processed by the streaming image sequence player.

🚧

Compatible with Built-in Render Pipeline Only

The Studio Mesh Sequence Exporter is currently only compatible with the Built-in Render Pipeline. It will not work with Scriptable Render Pipelines such as URP and HDRP.

🚧

StreamingAssets

If you don't place your image sequence in a 'StreamingAssets' folder, you will need to change the import settings for each image to get them to render at full quality.

Set up the Studio clip with the Depthkit Image Sequence Player

1. Create an empty game object.
2. Add a Depthkit clip to the game object, either using the Add Component button or the Component menu, then choosing Depthkit → Depthkit Clip.

3. Depthkit Clip
   • Change the Depthkit Clip Player setting to Image Sequence Player.
   • Select the metadata file for the asset to export.
   • Under Advanced Settings, check the disable poster frame check box.
4. Add a Depthkit → Studio → Built-in RP → Studio Look component.
5. Playable Director:
   • With the Depthkit object selected in the Hierarchy, create a Timeline Object.
   • With the timeline object selected, in the Timeline window, add a track to the timeline "+" → Unity.StreamingImageSequence → Streaming Image Sequence Track.
   • In the project window, navigate to the folder that contains the image sequence. Drag the folder to the the Streaming Image Track created in the previous step.
   • In the Timeline Settings Cog menu, select Framerate > 30.

• Expand the Keyframe Editor of the track, and right-click the start keyframe, and select Both Tangents > Linear.
• Repeat this for the end keyframe.

• Click on the track in the time line, and in the Inspector, verify that the FPS is 30 and frame range reflects the number of images in your folder.
• Select the game object, link the timeline created to the playable field in the Playable Director component.
• In the background, a new binding has been created and needs to be linked. With the Depthkit object selected in the Hierarchy, in the Inspector, unfold the bindings foldout and drag the Depthkit game object from the Hierarchy to the binding for the Streaming Image Sequence Track.

Now your clip is setup to play in Unity.

Configure reconstruction settings

1. Follow the general Depthkit Studio best practices for setting your reconstruction settings. In addition to this, special consideration is needed for a few parameters.
2. Volume Density: higher density will give the mesh more detail, but also generate larger files. Recommended range: 150-200 voxels/meter.
3. Enable Surface Smoothing: especially important at higher voxel densities, surface smoothing can remove unwanted "floating" geometry and high frequency noise. Try to balance out noise reduction while avoiding too much loss of detail. Recommended range: 0.4 - 0.6.
4. Texture blending: to preview the texture blending that you will get with your exported asset, undisclose the Experimental Texture Settings panel in the Depthkit Studio Mesh Source.
   • Reduce View Dependent Color Blend Weight to its lowest value (0.001).
   • Increase Surface Normal Color Blend Weight to its highest value.
   • Leave everything else at the default values.
5. Depth Bias Compensation: set your depth bias such that the texture aligns as well as possible on the model (after following the above texture blending steps). Depth bias will alter the surface geometry which in turn affects how the texture is projected onto the surface. Getting this right at this stage is crucial to ensure good quality texturing.

Configure Geometry Export settings for Textured Geometry Sequence

1. Add a Depthkit → Studio → Mesh Sequence → Geometry Sequence Exporter to the game object. This will add all the necessary components to the game object.
2. Ensure Export Textures is checked.
3. Choose your desired texture format.
4. Choose your desired export file prefix, and indexing mode based on your workflow.
5. Choose the appropriate units for your workflow. For Arcturus' HoloSuite, select 'meters'.
6. Click the Output Path button to choose a directory to save the mesh + texture sequence into.

Exporting the geometry sequence

1. Enter Play mode.
2. Click the Start Export button on the Depthkit Geometry Sequence Exporter component.
3. Wait for the export to finish.
4. Once done, you will see a message in the console stating Capturing Finished, and the timeline playhead will return to the beginning. At this point you can exit Play mode.