🚀 Flux KV Cache (GGUF):
FluxKVCacheGGUF adds a reference-image attention cache to a Flux-family model loaded through GGUF. During a generation, it saves information computed from reference-image tokens so later denoising steps can reuse it instead of processing those tokens again. By default, the saved information lives in system memory, allowing the cache to persist when the model is offloaded from GPU memory and loaded again. This is intended to reduce repeated work in GGUF workflows that use reference images; it does not provide a benefit when the workflow supplies no reference-image tokens. The node creates a fresh cache for each execution, helping prevent cached information from an earlier prompt from carrying into a new one.
🚀 Flux KV Cache (GGUF) Input Parameters:
model
The required MODEL connection supplies the Flux-family GGUF model to patch. Connect the model from your GGUF UNet loader here, then use the node’s output in place of the loader’s direct model connection to the sampler. The node clones the supplied model before applying its attention and input patches. There is no numeric minimum, maximum, default, or selectable option for this connection; you must provide a compatible MODEL.
cache_on_cpu
This optional BOOLEAN chooses where the reference-image cache is stored. The default is True, which stores cached information in system memory so it can remain available across model offload and reload cycles, at the cost of transferring it back to the processing device when reused. Set it to False to store the cache on CUDA GPU memory instead, which avoids those CPU-to-GPU transfers but uses more GPU memory and may be less suitable when your workflow experiences GPU memory pressure or frequent offloading. The available options are True and False; a numeric minimum or maximum does not apply.
🚀 Flux KV Cache (GGUF) Output Parameters:
model
The sole output is a MODEL: a clone of the connected model with the GGUF reference-image caching patches applied. Connect this output to the sampler’s model input so generation uses the patched model. It is not an image or a saved cache file. When reference-image tokens are present, the patch stores their attention information on its first pass and reuses it on later passes; when none are present, the attention patch makes no cache change.
🚀 Flux KV Cache (GGUF) Usage Tips:
- Connect your workflow as GGUF UNet loader → FluxKVCacheGGUF → sampler. Use the node’s
modeloutput for sampling; leaving the sampler connected to the original loader output bypasses this patch. - Leave
cache_on_cpuat its default,True, when GPU memory is limited or the model is frequently offloaded and reloaded. TryFalseonly when you have enough CUDA GPU memory and want to avoid transfers from system memory. - Use the node in a Flux-family GGUF workflow that actually supplies reference-image tokens. Without them, the attention patch returns without building a reference cache, so you should not expect a caching speedup.
- Treat the cache as belonging to one generation. The node is designed to execute again for each prompt and create a new cache object; it is not a persistent cache shared across prompts.
🚀 Flux KV Cache (GGUF) Common Errors and Solutions:
Required input is missing: model
- Explanation: The node cannot patch a model when its required
modelsocket has no connection. The exact validation wording may vary by ComfyUI version. - Solution: Connect a compatible
MODELoutput from your GGUF UNet loader to the node’smodelinput, then connect this node’smodeloutput to your sampler.
RuntimeError: Found no NVIDIA driver on your system
- Explanation: With
cache_on_cpuset toFalse, the node selectscudafor cached tensors. A CUDA initialization error can occur if the environment cannot use a supported NVIDIA GPU; the exact error text depends on your PyTorch installation. - Solution: Set
cache_on_cputoTrue. Check that the rest of your GGUF workflow can run in your environment as well; this setting only changes where this node stores its cache.
torch.OutOfMemoryError: CUDA out of memory
- Explanation: The model, generation, or GPU-resident cache may exceed available GPU memory. This is particularly relevant when
cache_on_cpuisFalse, though other parts of the workflow can also cause the error. - Solution: Set
cache_on_cputoTrueto keep the saved cache in system memory. If memory errors continue, reduce the memory demands of the broader workflow.
KeyError: 'block_type' or KeyError: 'block_index'
- Explanation: The attention patch expects the model’s attention options to identify the current block. If those fields are absent, it cannot assign a cache entry to that block.
- Solution: Confirm that the connected model and its loader are compatible with this Flux-family patch. Remove other model or attention patches temporarily to identify a compatibility conflict.
KeyError: 'transformer_options' or KeyError: 'img'
- Explanation: The model-input patch expects these fields in the data passed into the model. If a model implementation or another patch supplies a different structure, the patch cannot process it.
- Solution: Check that you are using a compatible Flux-family GGUF model workflow. Test without other model-input patches and check whether your loader and this node are intended to work together.
RuntimeError: Sizes of tensors must match except in dimension 2
- Explanation: When reusing cached attention information, the patch joins saved tensors with the current tensors along their token dimension. A mismatch in another dimension can prevent that operation. This is a possible PyTorch error, not a message defined by the node.
- Solution: Check that your reference-image configuration remains consistent during a generation and test without other attention patches that may change tensor shapes. Note that the cache checks reference token counts, not the image content itself, when deciding whether to rebuild it within an execution.
Generation runs, but there is no noticeable speedup
- Explanation: This is not necessarily an error. The cache is built only when the workflow supplies reference-image tokens, and its benefit depends on how much reference-image work can be reused. Storing the cache on CPU also requires transfers when it is reused.
- Solution: Verify that your workflow supplies reference-image tokens and that the sampler uses this node’s
modeloutput. Compare otherwise identical generations; considercache_on_cpu=Falseonly if CUDA GPU memory is sufficient.
