ComfyUI Node: Temporal Flow Average

Temporal Flow Average:

TemporalFlowAverage is a sophisticated node designed to perform motion-compensated temporal averaging using optical flow techniques such as MEMFOF or RAFT. This node is particularly beneficial for applications that require smoothing or denoising of video frames while preserving motion details. By leveraging optical flow, TemporalFlowAverage can accurately track and compensate for motion between frames, allowing for a more coherent and visually appealing result. This process is crucial in scenarios where maintaining the integrity of moving objects is essential, such as in video editing, animation, or any AI-driven art creation that involves dynamic scenes. The node's ability to handle motion compensation ensures that the temporal averaging process does not introduce artifacts or blurring, thus enhancing the overall quality of the output.

Temporal Flow Average Input Parameters:

batch_size

The batch_size parameter determines the number of frames processed simultaneously during the temporal averaging operation. A higher batch size can significantly speed up the processing but requires more VRAM. The default value is 1, with a minimum of 1 and a maximum of 64. For instance, on an RTX 5090 with 32GB VRAM at 720p resolution, a batch size of 8 to 16 is recommended for optimal performance.

precision

The precision parameter specifies the numerical precision used during inference. It offers two options: bf16 and fp32. The default is bf16, which is approximately 1.5 to 2 times faster on RTX 30/40/50 series GPUs with negligible quality differences. However, fp32 should be used if strict reproducibility of results is required.

flow_scale

The flow_scale parameter controls the resolution at which optical flow is computed. A value of 1.0 indicates full resolution, providing the best quality, while lower values (down to 0.3) allow for faster computation by reducing the resolution. The default is 1.0, and adjusting this parameter can lead to a speed increase of up to 3 times, although warping is still performed at full resolution to maintain quality.

Temporal Flow Average Output Parameters:

weighted_sums

The weighted_sums output represents the accumulated weighted sum of frames after motion compensation. This output is crucial for understanding how each frame contributes to the final averaged result, taking into account the motion between frames.

weight_sums

The weight_sums output provides the sum of weights used during the averaging process. This output is important for normalizing the weighted_sums to produce the final averaged frame, ensuring that the contribution of each frame is appropriately balanced.

Temporal Flow Average Usage Tips:

• To optimize performance, adjust the batch_size according to your GPU's VRAM capacity. A larger batch size can speed up processing but requires more memory.
• Use bf16 precision for faster processing on compatible GPUs, unless you need exact reproducibility, in which case fp32 is recommended.
• Experiment with the flow_scale parameter to find a balance between speed and quality. Lowering the scale can significantly increase processing speed with minimal quality loss.

Temporal Flow Average Common Errors and Solutions:

"CUDA out of memory"

• Explanation: This error occurs when the GPU does not have enough memory to process the current batch size.
• Solution: Reduce the batch_size or close other applications using GPU resources to free up memory.

"Invalid flow_scale value"

• Explanation: The flow_scale parameter is set outside the allowed range.
• Solution: Ensure that flow_scale is set between 0.3 and 1.0.

"Precision not supported"

• Explanation: The selected precision is not supported by the current hardware.
• Solution: Switch to a supported precision option, such as bf16 or fp32, depending on your GPU's capabilities.