ComfyUI Node: Apply First Block Cache

Apply First Block Cache:

The ApplyFBCacheOnModel node is designed to enhance the efficiency of model execution by implementing a caching mechanism known as First Block Cache (FBCache). This node is particularly beneficial in scenarios where repeated computations can be avoided, thereby reducing processing time and resource usage. By caching intermediate results during model execution, it allows for faster retrieval of these results in subsequent operations, provided certain conditions are met. This caching strategy is especially useful in iterative processes like diffusion models, where similar computations are performed repeatedly. The node intelligently manages cache hits and ensures that the cache is only used when it is beneficial, thus maintaining the accuracy and integrity of the model's output.

Apply First Block Cache Input Parameters:

model

This parameter represents the model on which the FBCache will be applied. It is crucial as it determines the context in which the caching mechanism operates. The model should be compatible with the FBCache system to ensure proper functionality.

object_to_patch

This string parameter specifies the object within the model that the FBCache will target. By default, it is set to "diffusion_model". This allows the node to know exactly where to apply the caching mechanism, ensuring that the correct part of the model benefits from the cache.

residual_diff_threshold

A float parameter that controls the tolerance for caching, with a default value of 0.0. It ranges from 0.0 to 1.0, where lower values enforce stricter caching conditions. Setting this to 0 disables the FBCache effect, meaning no caching will occur. This parameter is crucial for balancing between performance gains and the precision of the model's output.

start

This float parameter defines the start time as a percentage of the sampling process where the FBCache effect can begin to apply. It ranges from 0.0 to 1.0, with a default of 0.0, indicating the beginning of the sampling. Adjusting this parameter allows you to control when the caching mechanism should start being effective during the model's execution.

end

Similar to the start parameter, this float parameter specifies the end time as a percentage of the sampling process where the FBCache effect can apply. It also ranges from 0.0 to 1.0, with a default of 1.0, indicating the end of the sampling. This parameter helps in defining the duration over which caching is active.

max_consecutive_cache_hits

An integer parameter that limits the number of consecutive cache hits allowed. It has a default value of -1, which permits unlimited consecutive cache hits. Setting it to 0 disables the FBCache effect, while a positive value ensures that there is at least one full model call after each cached result. This parameter is essential for controlling the balance between caching efficiency and model accuracy.

Apply First Block Cache Output Parameters:

model

The output is the modified model with the FBCache applied. This model is optimized for performance through the caching mechanism, allowing for faster execution times in subsequent operations. The output model retains its original functionality while benefiting from the efficiency improvements provided by the FBCache.

Apply First Block Cache Usage Tips:

• To maximize the performance benefits of the FBCache, adjust the residual_diff_threshold to a value that balances speed and accuracy based on your specific use case.
• Use the start and end parameters to fine-tune the period during which caching is active, ensuring that it aligns with the most computationally intensive parts of your model's execution.
• If you notice a degradation in model accuracy, consider reducing the max_consecutive_cache_hits to ensure more frequent full model evaluations.

Apply First Block Cache Common Errors and Solutions:

Unsupported model <model_name>

• Explanation: This error occurs when the model provided is not compatible with the FBCache system.
• Solution: Ensure that the model is either a UNetModel or Flux, as these are the supported models for FBCache application.

STG is not supported with FBCache yet

• Explanation: This error indicates that the current model configuration uses a feature (STG) that is not yet compatible with the FBCache.
• Solution: Disable the STG feature or use a model configuration that does not rely on STG when applying the FBCache.

RuntimeError: Invalid cache state

• Explanation: This error suggests that the cache state is not valid, possibly due to incorrect parameter settings or model state.
• Solution: Verify that all input parameters are set correctly and that the model is in a valid state before applying the FBCache. Reset the cache state if necessary.