ComfyUI Node: 🎞️ Latent Temporal Upsampler

🎞️ Latent Temporal Upsampler:

The LatentTemporalUpsampler is a sophisticated tool designed for motion-compensated temporal upsampling of latent video representations. It operates on video latents structured in a 5D format [B, C, F, H, W], where B is the batch size, C is the number of channels, F is the number of frames, and H and W are the height and width of each frame. This node is particularly beneficial for enhancing the temporal resolution of video latents, allowing for smoother playback at higher frame rates. By adjusting the frame rate from a source to a target, it ensures that the motion within the video remains consistent and natural, even when the playback speed changes. The node achieves this by employing a motion-compensated interpolation technique, which reduces per-frame displacement and maintains the apparent speed of the video. This makes it an invaluable tool for AI artists looking to create high-quality video content with seamless motion transitions.

🎞️ Latent Temporal Upsampler Input Parameters:

auto_retime

The auto_retime parameter is a boolean setting that, when enabled, automatically adjusts the hermite velocity scale based on the ratio of the source frame rate to the target frame rate. This is particularly useful for maintaining consistent motion speed when changing frame rates, such as converting a video from 24 frames per second (fps) to 30 fps. By default, this parameter is set to True, which means the node will automatically handle the velocity scaling to ensure smooth motion transitions.

source_fps

The source_fps parameter represents the original frame rate of the video latent. It is crucial for calculating the necessary adjustments to achieve the desired target frame rate. This parameter does not have a specified default value, as it depends on the input video latent's properties.

target_fps

The target_fps parameter specifies the desired frame rate for the output video latent. It determines how many frames per second the upsampled video should have, allowing for smoother playback and enhanced temporal resolution. Like source_fps, this parameter does not have a predefined default value and should be set according to the user's requirements.

motion_scale

The motion_scale parameter is a float value that influences the scaling of motion within the video latent. It is used when auto_retime is disabled, allowing users to manually control the motion scaling. This parameter provides flexibility in adjusting the perceived speed of motion in the video, ensuring that it aligns with the artistic intent.

spatial_sharpen

The spatial_sharpen parameter is a float value that applies motion-adaptive spatial sharpening to the video latent. It enhances the clarity and detail of each frame, making the video appear sharper and more defined. The default value is 0.0, meaning no sharpening is applied unless specified by the user.

🎞️ Latent Temporal Upsampler Output Parameters:

samples

The samples output parameter represents the upsampled video latent, which has been processed to achieve the desired temporal resolution. This output retains the original structure of the input latent but with an increased number of frames, providing smoother motion and enhanced playback quality. The upsampled latent is ready for further processing or rendering, depending on the user's workflow.

🎞️ Latent Temporal Upsampler Usage Tips:

• To maintain consistent motion speed when changing frame rates, enable the auto_retime parameter. This will automatically adjust the velocity scale based on the source and target frame rates.
• Use the spatial_sharpen parameter to enhance the clarity of your video latents. Start with a small value and gradually increase it to achieve the desired level of sharpness without introducing artifacts.
• When working with videos that have significant motion, consider adjusting the motion_scale parameter to fine-tune the perceived speed of motion and ensure it aligns with your artistic vision.

🎞️ Latent Temporal Upsampler Common Errors and Solutions:

"Expected 5D, got {samples.shape} - passthrough"

• Explanation: This error occurs when the input video latent does not have the expected 5D shape [B, C, F, H, W]. The node requires this specific format to function correctly.
• Solution: Ensure that your input video latent is structured in the correct 5D format before passing it to the node. Check the dimensions and reshape the latent if necessary.

"Target frame rate must be greater than source frame rate"

• Explanation: This error indicates that the target frame rate specified is not greater than the source frame rate, which is necessary for upsampling.
• Solution: Adjust the target_fps parameter to be greater than the source_fps to enable temporal upsampling.