MiniMax H3 SAM3.1 Multi-Person Track / 多人分色追踪 (Advanced):
The MiniMaxH3SAM31MultiPersonTrackT8Advanced node is designed to facilitate advanced multi-person tracking within video sequences using the SAM3.1 model. This node is particularly useful for AI artists who need to track multiple individuals in a scene, as it provides color-coded shot-local track IDs for up to 2-3 people per detected shot. The node operates by running the ComfyUI native SAM3.1 multiplex tracking on each detected shot, ensuring efficient and accurate tracking. One of the key features of this node is its ability to selectively unload the SAM model and its clones before performing H3 repair, optimizing resource usage. It's important to note that the color coding used in the tracking does not correspond to character identity, which allows for flexible application across various projects.
MiniMax H3 SAM3.1 Multi-Person Track / 多人分色追踪 (Advanced) Input Parameters:
frames
This parameter represents the sequence of frames to be analyzed for tracking. It is crucial as it defines the visual data input for the node. The frames should be provided in a format compatible with the node's processing capabilities, typically as a list or array of image data.
model
The model parameter specifies the SAM3.1 model to be used for tracking. This model is responsible for detecting and tracking individuals within the provided frames. It is essential to ensure that the model is correctly loaded and compatible with the node's requirements.
conditioning
Conditioning refers to any additional data or parameters that influence the model's behavior during tracking. This can include pre-processing steps or specific configurations that tailor the model's performance to the task at hand.
fps
Frames per second (fps) is a parameter that indicates the frame rate of the input video. It is important for synchronizing the tracking process with the video playback speed, ensuring accurate temporal analysis.
maximum_people
This parameter sets the maximum number of people to track per shot. It is an integer value, typically capped at 2-3, to maintain performance and accuracy. Adjusting this parameter allows you to control the complexity of the tracking task.
detection_threshold
The detection threshold determines the sensitivity of the model in identifying individuals within the frames. A lower threshold may result in more detections, including false positives, while a higher threshold may miss some individuals. The default value is 0.53.
detect_interval
Detect interval specifies the frequency at which the model performs detection within the video sequence. A lower interval results in more frequent detections, which can improve tracking accuracy but may increase computational load. The default value is 3.
scene_cut_threshold
This parameter defines the threshold for detecting scene cuts within the video. It helps the node to reset tracking when a significant change in the scene is detected, ensuring that tracking remains accurate across different shots. The default value is 0.28.
analysis_max_side
Analysis max side sets the maximum dimension for resizing frames during analysis. This helps in managing computational resources by limiting the size of the input data. The default value is 512.
preview_stride
Preview stride determines the interval at which preview frames are generated during tracking. This can be useful for visualizing the tracking process without processing every frame. The default value is 8.
release_policy
Release policy specifies how the SAM3.1 model is managed after tracking is complete. The option "offload_sam31_after_track" indicates that the model should be unloaded to free up resources once tracking is finished.
MiniMax H3 SAM3.1 Multi-Person Track / 多人分色追踪 (Advanced) Output Parameters:
packed_masks
Packed masks are the output of the tracking process, containing the tracked individuals' data in a compact format. This output is crucial for further analysis or visualization, as it provides the necessary information to identify and differentiate between tracked individuals.
scores
Scores represent the confidence levels associated with each tracked individual. These values help in assessing the reliability of the tracking results, allowing you to make informed decisions about the quality of the output.
MiniMax H3 SAM3.1 Multi-Person Track / 多人分色追踪 (Advanced) Usage Tips:
- Ensure that the input frames are of high quality and properly pre-processed to improve tracking accuracy.
- Adjust the detection threshold and detect interval parameters to balance between detection sensitivity and computational efficiency.
- Use the preview stride parameter to visualize the tracking process without overwhelming your system with data.
MiniMax H3 SAM3.1 Multi-Person Track / 多人分色追踪 (Advanced) Common Errors and Solutions:
ValueError: SAM3.1 detected no people in shot {shot_id}
- Explanation: This error occurs when the SAM3.1 model fails to detect any individuals in the specified shot.
- Solution: Verify that the input frames are clear and contain visible individuals. Adjust the detection threshold to increase sensitivity if necessary.
ValueError: SAM3.1 returned only empty tracks in shot {shot_id}
- Explanation: This error indicates that the model detected individuals but failed to track them across frames.
- Solution: Ensure that the detect interval is set appropriately to maintain consistent tracking. Consider increasing the maximum_people parameter if the scene is crowded.