🕑Timecode Synchronization

Timecode

Timecode is a standardized system for labeling each frame of video or audio with a unique timestamp. In Virtual Productions, timecode ensures that all elements, cameras, audio, visual effects, and external devices, are perfectly aligned in time. This synchronization is essential for:

• Matching live-action footage with virtual elements.

• Ensuring precise playback and editing.

• Maintaining frame-accurate control across multiple systems.

• Simplifying post-production by providing a common temporal reference.

Timecode is the backbone of reliable, frame-accurate workflows in complex, multi-device environments.

Unreal Engine Timecode Provider

A Timecode Provider in Unreal Engine is a module that supplies timecode information to the engine, allowing it to synchronize its internal clock with external sources. This is crucial for integrating Unreal with other hardware and software in a virtual production pipeline. Timecode Provider in Unreal Engine can be any of the following:

• AjaTimecodeProvider

• AppleARKitTimecodeProvider

• BlackmagicTimecodeProvider

• LiveLinkTimecodeProvider

• NDITimecodeProvider

• SystemTimeTimecodeProvider

CameraTwin Timecode Synchronization

Although a CameraTwin Source could act as a Timecode Provider itself and we could create a specific CameraTwin Timecode Provider our team opted to offer the option to synchronize the internal Timecode of CameraTwin Source with the Selected Timecode Provider of Unreal Engine. That way CameraTwin Source would be in position to synchronize with external software or hardware Timecode Providers via the Unreal Engine and allow for flexibility on Timecode Providers

To achive the timecode synchronization CameraTwin Sources start a brief synchronization process when they are connected with the Unreal Engine. This process shouldn't take more that a couple of seconds and most times it's almost instant. To allow users to chekc the status of the synchronization we offer that information as a simple label in the Control Panel.

Synchronization accuracy is mainly affected by the roundtrip delay between the messages that are transmitted between the CameraTwin source and the Unreal Engine. Roundtrips of <8ms are considered exceptional, <16ms are considered acceptable and above that the synchronization will report a failure. These thresholds are relative to the 30fps target of the CameraTwin system.

Users Can manually trigger a synchronization process from the Control Panel in case they change or re-initialize the Timecode Provider or if synchronization process failed at first due to high wifi traffic.

Timed Data Monitor

The Timed Data Monitor is a specialized view in Unreal Engine for monitoring the flow and synchronization of time-stamped data streams. In this view, users can:

• Inspect incoming data from various sources (e.g., motion capture, camera tracking, audio).

• Verify that timecode information is being received and applied correctly.

• Monitor for dropped frames, latency, or synchronization issues.

• Compare timecode from multiple sources to diagnose and resolve discrepancies.

• Calibrate the time offset of the rendering part of Timecode

This tool is essential for troubleshooting and validating the integrity of synchronized workflows such as live streaming.

To access Timed Data Monitor first enable the plugin

Open the Timed Data Monitor Tab and observe the Data frames that you receive from the CameraTwin Source (Disable monitoring of the Stage Setup signal). You can see below that the inherent delay of the data being streamed to the Unreal Engine are causing the streams to be out of sync from the presentation Timecode. To achieve in sync presentation you can click the Calibration button or manually edit the Global Frame Offset value

After the calibration of the Global Frame Offset , in this case the value achieved is 5 frames, the streams are coming in time for the presentation timecode. That means that on average all signals are coming in time within the 5 frames time of 30 fps (5 x 33.33ms = 166.667ms delay).

Timecode Synchronization in Recordings

In CameraTwin, timecode synchronization and embedding timecode information in recordings are critical for several reasons:

• Frame-Accurate Editing: Ensures that recorded camera moves, takes, and virtual elements can be matched precisely in post-production.

• Multi-Device Sync: Allows footage from multiple devices (e.g., several smartphones, cameras, and Unreal Engine renders) to be aligned perfectly, even if recorded separately.

• VFX and Compositing: Facilitates seamless integration of live and virtual elements, as every frame can be matched by its timecode.

• Troubleshooting: Makes it easier to identify and resolve timing issues by providing a clear temporal reference in all assets.

Without proper timecode synchronization, even small timing mismatches can lead to significant problems in editing, compositing, and final delivery.