ComfyUI Node: Video Frame Pair Extractor

Video Frame Pair Extractor:

The VideoFramePairExtractor is a specialized node designed to efficiently extract pairs of frames from a batch of video frames for interpolation purposes. This node is particularly beneficial for tasks that require frame pairing without the overhead of data duplication, thereby optimizing memory usage. It replaces the ImagePairIterator by offering improved memory management through zero-copy operations, which means it accesses the frames directly without creating additional copies. This capability is crucial for handling large video datasets where memory efficiency is a priority. The node supports different modes of pairing, allowing you to extract frames sequentially, in all possible combinations, or by pairing the first frame with each subsequent frame. This flexibility makes it a versatile tool for various video processing workflows, especially in scenarios where frame interpolation is needed to create smooth transitions or animations.

Video Frame Pair Extractor Input Parameters:

images

This parameter represents the batch of video frames from which pairs will be extracted. It is expected to be a 4D tensor with dimensions corresponding to batch size, height, width, and channels (B, H, W, C). The node will raise an error if the input does not conform to this dimensionality, ensuring that the data is in the correct format for processing.

mode

The mode parameter determines the method used to extract frame pairs. It offers three options: sequential_pairs, all_combinations, and first_to_each. The default mode is sequential_pairs, which extracts consecutive frame pairs like (0,1), (1,2), etc. The all_combinations mode generates all possible pairs, such as (0,1), (0,2), (1,2), etc., while first_to_each pairs the first frame with every other frame. This parameter allows you to tailor the frame extraction process to your specific needs.

pair_index

This integer parameter specifies the index of the frame pair to extract. It defaults to 0 and must be a non-negative value. The pair_index is used to select a specific pair based on the chosen mode, allowing you to iterate through pairs in a controlled manner.

stride

The stride parameter is an optional integer that defines the step size between pairs when using the sequential_pairs mode. It defaults to 1, with a minimum value of 1 and a maximum of 10. Adjusting the stride can help you control the spacing between frames in the extracted pairs, which can be useful for varying the temporal resolution of the output.

Video Frame Pair Extractor Output Parameters:

start_frame

This output represents the first frame in the extracted pair. It is a view of the original frame data, ensuring that no additional memory is used for copying.

end_frame

The end_frame is the second frame in the extracted pair, similar to start_frame, it is also a view of the original data, maintaining memory efficiency.

total_pairs

This integer output indicates the total number of frame pairs available based on the input parameters. It helps you understand the extent of the pairing process and manage iterations accordingly.

has_more

A boolean value that signifies whether there are more pairs available for extraction beyond the current pair_index. This output is useful for controlling loops or iterations when processing multiple pairs.

info

The info output provides a string description of the current pair, including the indices of the frames involved. This information can be helpful for debugging or logging purposes, giving you insight into the pairing process.

Video Frame Pair Extractor Usage Tips:

• Use the sequential_pairs mode with a stride of 1 for smooth frame transitions, ideal for creating animations or interpolations.
• Opt for the all_combinations mode when you need to explore all possible frame interactions, which can be useful for comprehensive analysis or testing.

Video Frame Pair Extractor Common Errors and Solutions:

Expected 4D tensor (B,H,W,C), got <images.dim()>D

• Explanation: This error occurs when the input images tensor does not have the expected four dimensions.
• Solution: Ensure that your input tensor is formatted correctly with dimensions corresponding to batch size, height, width, and channels.

Index out of range

• Explanation: This error indicates that the specified pair_index or calculated indices exceed the available frames in the batch.
• Solution: Verify that your pair_index is within the valid range based on the total number of pairs and adjust the stride or mode if necessary.