ComfyUI Node: 👻 Latent Temporal Inpainter

👻 Latent Temporal Inpainter:

The LatentTemporalInpainter is a specialized node designed to enhance video frames by injecting noise in a ghost-aware manner, allowing for targeted corrective resampling. This node is particularly useful in scenarios where uniform denoising fails due to the model's inherent frame rate cadence, which can lead to unintended speed changes during playback. By focusing on per-frame ghost detection and noise application, the LatentTemporalInpainter ensures that real anchor frames are preserved while allowing for the correction of ghost frames. This approach is beneficial for maintaining the integrity of the original content while applying necessary adjustments to improve the overall quality of the video output.

👻 Latent Temporal Inpainter Input Parameters:

anchor_sigma

anchor_sigma controls the level of noise applied to anchor frames, which are frames that should remain stable and unchanged. The default value is 0.05, with a minimum of 0.0 and no specified maximum. Adjusting this parameter affects how much the anchor frames are preserved during the inpainting process.

ghost_sigma

ghost_sigma determines the intensity of noise applied to ghost frames, which are frames that require corrective resampling. The default value is 0.35, with a minimum of 0.0 and no specified maximum. This parameter is crucial for effectively targeting and correcting ghost frames without affecting the anchor frames.

score_gamma

score_gamma is a parameter that influences the weighting of noise application based on the detected ghost frames. It has a default value of 2.0, with a range from 0.5 to 5.0. This parameter helps in fine-tuning the balance between noise application and frame preservation.

anchor_blend

anchor_blend specifies the blending factor between the original and noised frames for anchor frames. The default value is 0.4, with a range from 0.0 to 1.0. This parameter allows for a smooth transition between the original and modified frames, ensuring that the anchor frames remain visually consistent.

seed

seed is an integer parameter used to initialize the random number generator for noise application. It has a default value of 0, with a range from 0 to 2^31. Setting a specific seed ensures reproducibility of the inpainting results.

debug_scores

debug_scores is a boolean parameter that, when enabled, provides additional debugging information about the noise application process. The default value is False. This parameter is useful for developers or advanced users who need to understand the internal workings of the node.

👻 Latent Temporal Inpainter Output Parameters:

latent

The latent output is a modified version of the input latent, where ghost frames have been corrected through targeted noise application. This output retains the original structure of the input while enhancing the quality of the video frames by addressing ghosting issues.

👻 Latent Temporal Inpainter Usage Tips:

• Use a lower anchor_sigma to ensure that important frames remain stable and unchanged during the inpainting process.
• Adjust ghost_sigma to effectively target and correct ghost frames without affecting the overall video quality.
• Experiment with score_gamma to find the optimal balance between noise application and frame preservation for your specific video content.

👻 Latent Temporal Inpainter Common Errors and Solutions:

"Expected 5D, got <shape>"

• Explanation: This error occurs when the input tensor does not have the expected five dimensions, which are necessary for processing video frames.
• Solution: Ensure that the input tensor is correctly formatted with five dimensions, typically representing batch size, channels, frames, height, and width.

"Noise mask shape mismatch"

• Explanation: This error indicates a mismatch between the expected and actual shapes of the noise mask, which is used to apply noise selectively.
• Solution: Verify that the noise mask is correctly generated and matches the dimensions of the input tensor to ensure proper noise application.