H3 Sparse Attention:
H3SparseAttention is a node designed to implement fixed-density sparse attention for the MiniMax H3 model, optimizing the processing of video data by managing attention budgets. This node is particularly beneficial for scenarios where computational efficiency is crucial, as it allows for faster processing by reducing the attention budget allocated to video data. However, this reduction can impact the adherence to prompts, alter motion or detail, and change the generated results. The node ensures that text, reference conditioning, audio, non-video queries, and mixed boundary tiles remain dense, maintaining their quality. The backend automatically selects the most efficient processing path, preferring native Kitchen INT8, followed by Sparse Sage, BF16 Triton, FP8 FlexAttention, and finally, the dense attention path. This hierarchy ensures that the node operates efficiently while balancing quality and performance.
H3 Sparse Attention Input Parameters:
model
This parameter represents the model to which the sparse attention mechanism will be applied. It is crucial for defining the context in which the node operates, ensuring that the attention adjustments are tailored to the specific model architecture and requirements.
video_budget
The video_budget parameter controls the proportion of attention resources allocated to video data. A lower budget results in faster processing but may reduce the quality of the generated output by affecting prompt adherence and detail. The default value is 0.3, with a range from 0 to 1, where 1 represents full attention allocation.
denser_early_late_steps
This boolean parameter determines whether to apply denser attention during the early and late steps of processing. Enabling this option can help maintain quality during critical phases of generation, although it defaults to False, indicating that denser attention is not applied unless explicitly specified.
layer_video_budgets
This optional parameter allows for specifying individual attention budgets for different layers within the model. It provides fine-grained control over how attention resources are distributed across the model's layers, enabling more precise optimization of performance and quality.
H3 Sparse Attention Output Parameters:
model
The output model is the modified version of the input model, now optimized with the sparse attention mechanism applied. This output is crucial as it reflects the adjustments made to enhance processing efficiency while maintaining as much quality as possible given the specified attention budgets.
H3 Sparse Attention Usage Tips:
- To optimize performance, start with a moderate video_budget and adjust based on the quality of the output. Lower budgets increase speed but may affect detail and adherence to prompts.
- Use the denser_early_late_steps option if you notice a significant drop in quality during the initial or final stages of generation, as this can help preserve important details.
H3 Sparse Attention Common Errors and Solutions:
Sparse Sage kernel failed
- Explanation: This error occurs when the Sparse Sage kernel encounters an issue during execution, possibly due to incompatible input data or configuration settings.
- Solution: Verify that the input data and configuration settings are compatible with the Sparse Sage kernel. Ensure that the model and attention parameters are correctly specified and within acceptable ranges.
Invalid video_budget value
- Explanation: This error arises when the video_budget parameter is set outside its valid range of 0 to 1.
- Solution: Adjust the video_budget parameter to a value within the 0 to 1 range, ensuring it reflects the desired balance between speed and quality.