ComfyUI-Local-GGUF-LLM Introduction
ComfyUI-Local-GGUF-LLM adds a local language model to your ComfyUI workflows. You can use it to draft image prompts, expand short ideas into detailed descriptions, create variations, or describe reference images and video frames when you have a compatible vision model.
The extension runs GGUF models through a shared Local LLM Server inside ComfyUI. GGUF is a model file format commonly used for local inference. “Local” means the model runs on your machine rather than being sent to an external text-generation service. Running locally does not mean every request is private by default: the optional external API and content logging have their own controls, described below.
The package provides exactly three workflow nodes:
| Node | What you can use it for |
|---|---|
| Local LLM Generate | Send instructions, text, and optional reference media to the model; receive generated text and, when available, separate reasoning text. |
| Local LLM Settings | Reuse the current server settings or a saved configuration in a workflow. |
| Local LLM Prompt Enhancer | Turn a starting idea into enhanced prompts, save variations, and cycle through them during workflow runs. |
It also includes a server panel for choosing the model, managing memory, tuning performance, and optionally enabling an OpenAI-compatible API.
Package boundary: The old H3 Shot Generator is not included. It is a separate
ComfyUI-H3-Shot-Generatorpackage. This extension still includes MiniMax H3 prompt-writing templates for Prompt Enhancer.
How ComfyUI-Local-GGUF-LLM Works
Think of the Local LLM Server as one shared writing assistant. The workflow nodes send it requests, rather than each loading a separate copy of the model. This helps avoid duplicate model loads when you use Generate and Prompt Enhancer in the same ComfyUI session.
A typical workflow looks like this:
- Choose a GGUF model in the Local LLM Server panel.
- Set its behavior and memory use. You can start with
Auto (Detected)for the model preset and Auto Yield to ComfyUI for VRAM management. - Send a request from a node. For example, Prompt Enhancer can turn “a lighthouse in a storm” into a more detailed image prompt.
- Use the returned text downstream. Connect the generated prompt to the text input of another node in your image or video workflow.
With On Demand startup, the server loads the model when the first request arrives. It can keep the model available for later requests, or release it to make room for other ComfyUI work.
A vision-capable model can also receive images or sampled video frames as references. The model file alone may not be enough: many vision models need a matching vision projector, often called an mmproj, and support from the installed llama.cpp binding. A text-only model cannot interpret an image just because an image is connected to the node.
ComfyUI-Local-GGUF-LLM Features
Local LLM Server: your shared control panel
Open Local LLM Server from its ComfyUI sidebar launcher or floating status control. Its seven tabs group the main controls:
| Tab | What it controls |
|---|---|
| Server | Service status, request activity, startup behavior, and Start/Suspend/Stop controls. |
| Presets | Saved Complete Settings Presets for model and generation configurations. |
| Model | Model choice, vision projector, generation style, sampling, and vision limits. |
| Memory | Context size, CPU/GPU use, VRAM policy, and performance-related memory settings. |
| Tuner | Benchmarks settings and suggests a configuration for your workload. |
| API | Optional OpenAI-compatible access for other applications. |
| Logs | Runtime diagnostics and the optional content-logging setting. |
Start, Suspend, and Stop / Unload do different things. Suspend releases the loaded model while keeping its configuration, so a later request can reload it. Stop / Unload also interrupts Local LLM work and unloads the model when it is safe to do so; it does not trigger ComfyUI’s global workflow interrupt.
Startup choices are Off, On Demand, and Auto Start. If you only occasionally use an LLM while making art, On Demand avoids loading it at every ComfyUI startup.
Generate text in a workflow
Local LLM Generate accepts an editable prompt and system prompt, with reusable presets for each. A system prompt gives the model an overall instruction, such as “Write concise image-generation prompts”; the prompt is the specific request for that run.
The node also accepts an optional image or image batch, ordered video_frames, and a request-local seed. Its outputs are:
response: the final generated text.thinking: reasoning text, if the model and its template expose it.info: formatted JSON diagnostics about the request.
If you do not connect Local LLM Settings, Generate uses the current server configuration. If you connect it, Generate uses the settings supplied by that node. Generate’s seed remains its own request setting either way.
For media input, the active vision settings limit image size and the number of video frames used. Video frames are sampled evenly from the supplied frame batch. These limits can help control the amount of media the model must process, but a compatible multimodal model, projector, and handler are still required.
Enhance, save, and cycle through prompts
Local LLM Prompt Enhancer helps you build a collection of prompt variations from an original idea. Its enhanced_prompt output supplies the selected or newly generated enhancement; its prompt output preserves the original prompt unchanged.
You can:
- Use Enhance Prompt manually to generate 1–64 enhancements in a batch.
- Edit generated prompts and move through them with previous/next controls or a direct index.
- Delete an entry, clear the collection, or use Undo/Redo for up to 20 array states in each direction.
- Save the full collection and selected position as a Prompt Set.
- Apply an enhancement template or write your own instructions.
- Supply optional image or native ComfyUI
VIDEOreferences, subject to model support. - Connect Local LLM Settings to control the request.
When Enhance with Workflow is on, each normal workflow execution asks the LLM for a fresh enhancement. When it is off, the node returns one of its stored enhanced prompts without calling the LLM. This distinction is useful when you want to reuse prepared prompts instead of waiting for text generation on every image run.
With workflow enhancement off, Prompt Cycle controls which stored prompt is selected:
| Mode | Selection behavior |
|---|---|
fixed | Keep the selected entry. |
increment | Move forward through the collection, wrapping around at the end. |
decrement | Move backward, wrapping around at the beginning. |
shuffle | Work through a shuffled, no-repeat selection until the deck is exhausted. |
random | Choose freely; repeats are possible. |
For example, store four scene descriptions in a Prompt Set and use increment while queueing multiple image generations. Each queued item can receive the next description. Active cycling modes intentionally re-execute the Prompt Enhancer so the selected text can change; fixed can be cached when its inputs do not change.
The bundled protected enhancement templates are Krea 2 - Image and MiniMax H3 - T2VA, I2VA, FL2VA, L2VA, and Ref2VA. The MiniMax entries help write prompts; they do not add H3 shot-generation nodes.
Reuse settings without duplicating the server
Local LLM Settings carries a configuration into Generate or Prompt Enhancer. Choose:
- Current Server to follow the live server configuration.
- Custom to use values saved in the Settings node.
- A saved Complete Settings Preset to reuse a named configuration.
The node exposes frequently adjusted options such as sampling, reasoning, and vision limits. Model choice and detailed memory settings come from the server or a Complete Settings Preset. If you edit a visible field belonging to a selected preset, the node switches to Custom. Seeds are not stored in this node; each request node owns its seed.
Complete Settings Presets can include the model, projector, sampling, vision limits, context, memory policy, and other runtime options. They do not include API keys, startup mode, content logging, or interface preferences.
Balance LLM use with image and video workflows
The Memory tab is especially useful if your LLM and diffusion or video models share a GPU.
- Auto Yield to ComfyUI allows the LLM to release its loaded model/context when ComfyUI needs GPU memory, then reload for a later LLM request. This is the recommended starting point for a shared machine.
- Keep Resident avoids voluntary unloading. It can reduce reloads when you have enough memory for the rest of your workflow.
- The panel estimates model, context, working, speculative-decoding, and—when relevant—vision memory, along with available GPU memory and projected headroom.
A larger context size lets the model work with more text at once but generally needs more memory. CPU-only mode keeps this extension’s model, context, and vision projector off the GPU, preserving VRAM for art generation at the cost of typically slower LLM output. The Use CPU-only Auto button supplies a starting configuration for CPU use.
Other controls include GPU layer offload, KV-cache formats and placement, Flash Attention, batching, and speculative decoding. Their availability and effect depend on your model and llama.cpp build. If you are unsure, begin with the detected/default settings and change one option at a time.
The Performance Tuner tests the saved server configuration and can recommend changes. Choose ComfyUI Cycle if repeated loading and yielding are part of your workflow, or Inference Only if you mainly care about prompt-processing and generation speed. Its Quick and Standard profiles trade test time for breadth of testing. The tuner does not silently lower your configured context size.
Optional API and privacy controls
The optional OpenAI-compatible API lets a compatible external client send requests to the same active model. It is disabled by default and is not a complete implementation of every OpenAI feature. Its base URL is displayed in the server’s API tab and follows this form:
http://<comfy-host>:<port>/local-llm/v1
The API offers model-list, chat-completion, and text-completion endpoints. It does not switch GGUF models based on a client’s model field: select the actual model in Local LLM Server. Streaming can be enabled or disabled in the API tab. If you expose ComfyUI beyond a trusted local environment, use an API key.
Runtime logs are available for diagnostics. Prompt and response content are not logged by default; content logging must be explicitly enabled in the Logs tab.
ComfyUI-Local-GGUF-LLM Models
This extension does not bundle a fixed set of GGUF model weights. You choose your own files, and the model browser discovers GGUFs recursively under:
ComfyUI/models/LLM/
You can organize them in subfolders—for example, one folder per model family. Files named like mmproj, vision-proj, or projector appear as vision-projector choices rather than ordinary text-model choices.
The extension includes tuned or detected model-family presets, not downloadable models, for:
- Qwen 3.8, Qwen 3.5, and Qwen 3
- GPT-OSS 20B
- Mistral Small 3.2 and Ministral 3
- Gemma 3
- Llama 3.1/3.2 Instruct
- DeepSeek R1 Distill
- Phi-4 Reasoning
- Nemotron 3 Nano
Use Auto (Detected) to let the extension select the closest known behavior from model metadata and its filename. An unfamiliar community model can fall back to generic behavior rather than being rejected.
The extension also recognizes vision-capability information for a wider range of multimodal families, including Qwen VL variants, Gemma vision variants, GLM vision models, LFM vision models, MiniCPM, LLaVA, Llama 3 Vision, Moondream, and OCR-oriented models. Recognition alone does not guarantee that your installed binding supports a particular model or that you have the right projector.
For choosing a model, start with the task:
- Prompt writing without references: choose a text-capable GGUF and use Generate or Prompt Enhancer.
- Prompts informed by images or frames: choose a compatible multimodal GGUF and its matching projector.
- A workflow short on VRAM: use a model and runtime configuration that fit alongside your art models, or choose Auto Yield to ComfyUI or CPU-only operation.
Different models may write in different styles, understand references differently, and use different amounts of memory. Sampling and model presets further affect how varied or restrained the resulting text feels.
What’s New with ComfyUI-Local-GGUF-LLM
Package version: 0.18.103-alpha
Bundled Prompt Enhancer version: 0.6.54-alpha
The 0.18.103-alpha update rebuilds Prompt Cycle around ComfyUI’s native control_after_generate queue lifecycle—the mechanism also used by KSampler Seed. Each queued item carries its own hidden cycle counter, which is mapped to an entry in the stored prompt array.
For artists queueing multiple runs, the practical benefit is that prompt selection is tied to each queued job rather than relying on a shared server-side cycling cursor. Fixed, increment, decrement, and random modes use ComfyUI’s native control behavior; shuffle adds a no-repeat selection behavior. The visible prompt and index follow the queued selection for display.
The previous /prompt admission-cycle handler is no longer used for normal Prompt Cycle. Older queue-related fields remain for workflow compatibility, but they do not control normal cycling.
Troubleshooting ComfyUI-Local-GGUF-LLM
No GGUF models appear
- Check that your model files are inside
ComfyUI/models/LLM/or one of its subfolders. - Confirm that the text model file ends in
.gguf. A file named likemmprojis listed as a projector instead of a text model. - Restart ComfyUI and reopen the model list.
- If your files are in an older lowercase
ComfyUI/models/llm/folder, move them into the uppercaseLLMfolder.
An image or video reference is rejected or has no effect
- Confirm that the selected GGUF is a multimodal model, not text-only.
- Select its compatible Vision / mmproj projector, or use
Autoif a safe match is available. - Check that the installed
llama-cpp-pythonbuild supports the model’s required vision handler or multimodal path. - Check the active vision limits if large images or many frames are involved.
Connecting media cannot give vision abilities to a text-only model.
The LLM is using VRAM needed for image generation
- Open Local LLM Server → Memory.
- Select Auto Yield to ComfyUI.
- If you need to release the model immediately, use Suspend.
- The next LLM request can reload the configured model.
If GPU memory remains a constraint, consider CPU-only operation, with the understanding that LLM generation is usually slower.
Prompt Enhancer runs again even though the text looks unchanged
Check Prompt Cycle. increment, decrement, shuffle, and random intentionally re-execute the node so each queued item can select a prompt. Use fixed when you want the selected stored prompt to remain cacheable. Also check Enhance with Workflow: when enabled, it requests a new enhancement on every normal workflow execution.
An external client shows the wrong context size
The usable context is the Context Size configured in Local LLM Server or supplied through Local LLM Settings—not a value assumed by the client. Check the server panel or the API’s /local-llm/v1/models response.
Rerolling through the API produces identical text
- Check whether the client sends an explicit seed. An omitted,
null, or-1API seed requests a fresh random seed; a nonnegative seed is explicit. - Check generation controls. A temperature of
0or top-k of1makes output effectively independent of the seed. - If needed, compare the request and sampling details in the server logs.
Different seeds can still produce the same wording when a model strongly favors particular next words.
Frequently asked questions
Does each workflow node load its own model?
No. Generate and Prompt Enhancer use the same persistent Local LLM service.
Will connecting Local LLM Settings change my Generate seed?
No. Generate’s seed is request-local; Settings does not contain a seed.
Can Prompt Enhancer use saved results without calling the LLM?
Yes. Turn Enhance with Workflow off, then select a stored enhancement or use Prompt Cycle.
Can I use an image with any GGUF?
No. Image and frame inputs require a compatible multimodal model, projector, and supported handler.
Are my prompts written to diagnostic logs?
Not by default. Prompt and response content logging must be enabled explicitly.
Learn More about ComfyUI-Local-GGUF-LLM
The most useful starting point is the extension’s Local LLM Server panel inside ComfyUI. Its Model and Memory tabs show the active configuration; the Server tab shows whether requests are running or queued; and the Logs tab provides diagnostics when something does not behave as expected.
To explore prompt-writing styles, try the Prompt Enhancer node’s bundled templates, save useful variations in a Prompt Set, and compare fixed, increment, and shuffle across queued workflow runs. For model-specific capabilities, consult the documentation supplied with the GGUF and its matching vision projector: multimodal support depends on those files and your installed llama.cpp binding, not on the node connection alone.
