ComfyUI Node: Extract Storyboards

Extract Storyboards:

The ExtractStoryboards_xuhuan1024 node is designed to facilitate the extraction of keyframes from a sequence of images, which is particularly useful in the creation of storyboards. This node leverages the structural similarity index (SSIM) to identify frames that are significantly different from their predecessors, thus marking them as keyframes. By doing so, it helps in efficiently summarizing a video sequence into its most representative frames, which can be crucial for tasks such as video editing, animation planning, or visual storytelling. The node is capable of handling large image sequences and provides options to merge closely spaced keyframes and split sequences into manageable batches, ensuring flexibility and control over the storyboard extraction process.

Extract Storyboards Input Parameters:

image

This parameter accepts a sequence of images, which serves as the input for the keyframe extraction process. The images should be provided in a format that the node can process, typically as a batch with dimensions [B, H, W, C], where B is the number of images, H is the height, W is the width, and C is the number of color channels.

threshold

The threshold parameter is a floating-point value that determines the sensitivity of the SSIM-based keyframe detection. It influences how different two consecutive frames need to be for the latter to be considered a keyframe. A lower threshold will result in more keyframes being detected, while a higher threshold will reduce the number of keyframes. The default value is 0.1, with a range from -1.0 to 1.0, allowing for fine-tuning based on the desired level of detail in the storyboard.

mergeInterFrames

This integer parameter specifies the minimum number of frames that should exist between two keyframes for them to be considered distinct. If the number of frames between two detected keyframes is less than this value, they will be merged, retaining only the latter keyframe. This helps in reducing redundancy in the storyboard. The default value is 10, with a minimum of 0 and a maximum of 999.

maxFrames

The maxFrames parameter sets the maximum number of frames allowed in a single batch of the extracted storyboard. If the interval between keyframes exceeds this number, the sequence will be split into smaller batches, each containing no more than maxFrames frames. This is particularly useful for managing large sequences and ensuring that the output is easy to handle. The default value is 999999, with a range from 1 to 999999999.

Extract Storyboards Output Parameters:

KEYFRAMES

This output provides the extracted keyframes as images. These are the frames identified as most representative of the input sequence, based on the SSIM analysis and the specified parameters. They form the core of the storyboard, capturing the essential visual elements of the sequence.

indexes_string

The indexes_string output is a string representation of the indices of the keyframes within the original sequence. This allows for easy reference and documentation of which frames were selected as keyframes, facilitating further processing or review.

indexes_int

The indexes_int output provides the indices of the keyframes as integers. This is useful for programmatic access and manipulation, enabling integration with other processes or systems that require numerical index data.

Extract Storyboards Usage Tips:

• Adjust the threshold parameter to control the sensitivity of keyframe detection. A lower threshold will capture more subtle changes, which can be useful for detailed storyboards.
• Use the mergeInterFrames parameter to avoid redundant keyframes in sequences with frequent minor changes, ensuring a concise storyboard.
• Set the maxFrames parameter according to your processing capabilities and the desired granularity of the storyboard batches, especially when dealing with long sequences.

Extract Storyboards Common Errors and Solutions:

IndexError: list index out of range

• Explanation: This error may occur if the int_index provided to the node is outside the range of available indices in the ints list.
• Solution: Ensure that the int_index is within the bounds of the ints list by checking its length before execution.

ValueError: could not convert string to float

• Explanation: This error might arise if the threshold parameter is provided with a non-numeric value.
• Solution: Verify that the threshold is a valid floating-point number within the specified range.

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

• Explanation: This error can occur if the SSIM calculation returns None, possibly due to invalid image data.
• Solution: Check the input images to ensure they are correctly formatted and contain valid data for SSIM computation.