ComfyUI Node: SAM3 Propagate

SAM3 Propagate:

The SAM3Propagate node is designed to facilitate the propagation of prompts across video frames, leveraging advanced video inference techniques. This node is particularly beneficial for tasks that require consistent application of prompts, such as object tracking or video editing, where maintaining the integrity of prompts across frames is crucial. By utilizing caching mechanisms, SAM3Propagate optimizes performance by storing and reusing results from previous computations, thus reducing redundant processing. It supports both full and partial propagation, allowing for flexibility in how prompts are applied throughout the video. The node ensures that the model is efficiently managed in terms of memory usage, with options to offload the model to the CPU when necessary, thereby freeing up GPU resources. This makes SAM3Propagate a powerful tool for artists and developers working with video content, providing a seamless and efficient way to propagate prompts across frames.

SAM3 Propagate Input Parameters:

sam3_model

The sam3_model parameter represents the model used for video inference. It is crucial for executing the propagation process, as it contains the necessary algorithms and data structures to interpret and apply prompts across video frames. This parameter should be a pre-trained model compatible with the SAM3 framework, ensuring accurate and efficient propagation.

video_state

The video_state parameter holds the current state of the video, including metadata such as the session UUID, number of frames, and any prompts that have been added. It is essential for tracking the progress of the propagation and ensuring that prompts are applied consistently across the specified frames. This parameter is immutable and is used to generate cache keys for efficient processing.

start_frame

The start_frame parameter specifies the index of the first frame where propagation should begin. It allows users to define the starting point of the propagation process, providing control over which part of the video is affected. The minimum value is 0, and it defaults to 0 if not specified.

end_frame

The end_frame parameter indicates the index of the last frame for propagation. It determines the endpoint of the propagation process, allowing users to limit the scope of the operation. If set to -1, the propagation will continue to the last frame of the video. This parameter provides flexibility in defining the range of frames to be processed.

direction

The direction parameter defines the direction of propagation, which can be "forward," "backward," or "both." This parameter allows users to control the flow of propagation, ensuring that prompts are applied in the desired sequence. It is crucial for tasks that require specific directional processing, such as reverse tracking or bidirectional consistency.

offload_model

The offload_model parameter is a boolean flag that determines whether the model should be offloaded to the CPU after propagation. This is useful for freeing up GPU resources, especially in environments with limited VRAM. When set to True, the model is moved to the CPU, and GPU memory is cleared, optimizing resource usage for subsequent operations.

SAM3 Propagate Output Parameters:

masks_dict

The masks_dict output parameter contains the masks generated for each frame during the propagation process. These masks represent the areas of interest as defined by the prompts, allowing for precise application of effects or tracking across the video. The masks are crucial for visualizing the results of the propagation and ensuring that the intended areas are accurately processed.

scores_dict

The scores_dict output parameter provides confidence scores for each frame, indicating the reliability of the propagated prompts. These scores help users assess the quality of the propagation and make informed decisions about further processing or adjustments. High confidence scores suggest accurate propagation, while lower scores may indicate areas that require refinement.

SAM3 Propagate Usage Tips:

• Ensure that the video_state includes all necessary prompts before starting propagation to avoid errors and ensure accurate results.
• Utilize the offload_model parameter to manage GPU resources effectively, especially when working with large videos or limited hardware capabilities.
• Experiment with different direction settings to achieve the desired propagation effect, whether it's forward, backward, or bidirectional.

SAM3 Propagate Common Errors and Solutions:

[SAM3 Video] No prompts added. Add point, box, or text prompts before propagating.

• Explanation: This error occurs when the video_state does not contain any prompts, which are necessary for the propagation process.
• Solution: Ensure that you add at least one prompt (point, box, or text) to the video_state before initiating the propagation.

[SAM3 Propagate] CACHE MISS - running propagation for session=<session_id>

• Explanation: This message indicates that there is no cached result for the current session, and the node will perform the propagation from scratch.
• Solution: This is not an error but a notification. If you frequently encounter cache misses, consider optimizing your workflow to reuse cached results when possible.