ComfyUI-LLM-Widget Introduction
ComfyUI-LLM-Widget adds one node, LLM Widget, to ComfyUI. It lets you ask a language model a question, optionally include connected images or a video, and keep the answer on the node. An LLM, or large language model, is the tool that writes the response.
You can use it to draft or rewrite an image-generation prompt, translate a prompt, describe a reference image, compare visual ideas, or ask a question about a picture. For example, you might connect a reference image and ask, “Describe the lighting and composition as a prompt I can adapt.”
The ✦ button works in the ComfyUI editor without running your workflow. Pressing it does not add a job to the queue or run other nodes. The author intends the node to serve as a workspace tool, usually left bypassed or muted in the workflow. You can also choose to make it generate an answer during workflow execution when you need its text as part of an automated process.
| Node | Settings |
|---|---|
| LLM Widget node | LLM Widget settings |
How ComfyUI-LLM-Widget Works
Think of the node as a notepad connected to a writing assistant. You write a question, optionally give the assistant standing instructions in a system prompt, and press ✦. The answer appears in the editable answer box and is saved with your workflow.
There are two ways to use it:
- In the editor: Press ✦ whenever you want an answer. This is the default way to use the widget. It does not execute the graph.
- During workflow execution: Turn on
generate_on_execute. The node then asks its question as the workflow runs, sends the result through itstextoutput, and writes the result back to the answer box. This mode is useful for headless or API-driven workflows.
That distinction matters when you connect another node to question or system_prompt. Pressing ✦ uses the text saved on the LLM Widget, because the upstream node has not run. To use the upstream node’s output, turn on generate_on_execute and run the workflow.
The editor can send connected media only when it already exists as a file in ComfyUI’s input/, output/, or temp/ folders. A Load Image node can therefore provide a reference image. An image that exists only as the future output of VAE Decode cannot be sent by pressing ✦; it has not been created yet. During workflow execution, generate_on_execute can instead use the actual image or video data produced by the graph. The widget follows connections through Reroute nodes, and its media indicators show what will be sent. A crossed-out red indicator means the media will be skipped; the result also reports a warning.
For OpenAI-compatible backends and local GGUF models, the widget represents a video using selected still frames. It keeps the first and last frames and favors frames showing significant changes, such as a scene cut. Timestamps help the model understand when each frame occurred. Gemini receives the video file directly.
ComfyUI-LLM-Widget Features
Questions, Answers, and Reusable Instructions
question: Write what you want the model to do. Specific requests tend to be easier to use in an art workflow, such as “Turn this reference into a concise prompt describing its color palette, lighting, and camera angle.”answer: View or edit the response. The text is saved with the workflow, so you can keep a useful prompt draft alongside your nodes.system_prompt: Set ongoing instructions, such as a preferred prompt style. You can save several system-prompt tabs on each node and switch between them. Only the open tab is sent. The tabs and their contents are saved with the workflow.textoutput: Passes the answer to another node. Withgenerate_on_executeoff, running the workflow passes along the answer already saved in the widget; it does not ask a new question.
The toolbar provides ✦ to ask, ■ to stop an active request, ⧉ to copy the latest answer, and ⚙ to open settings. Where applicable, ⏏ releases a loaded model. The ⌫ button appears when conversation mode is enabled and clears its log after confirmation.
Image and Video References
The node accepts optional image through image10 inputs and one optional video input. Images retain their slot numbers when sent: image3 is called image 3 even if image2 is empty. You can refer to those numbers in a question, for example: “Use image 1 for composition and image 3 for the clothing colors.”
In settings, Send the connected image / video can be switched off when you want a text-only answer without removing your connections. Video frames sent controls how many still frames are selected for applicable backends, from 2 to 12; the default is 4. A higher value gives the model more moments to inspect but sends more image content. This setting does not affect Gemini’s direct video upload.
Conversation Mode
Turn on Continue the conversation to build on previous answers. The answer box becomes an editable log of your questions and the model’s replies. For example, you can first request a portrait prompt, then ask, “Make the lighting softer.” The earlier exchange is sent with the follow-up.
The log itself is the conversation history: editing or removing an entry changes what the model can refer to next time. Previous images and videos are not resent with the history; only their associated text remains. Exchanges kept in the history limits how many recent question-and-answer pairs are sent, which can help when a conversation becomes too long. Blank line between the log entries changes the log’s appearance without changing how it is read.
In this mode, ⧉ copies only the most recent model answer, and the text output also contains only that answer, not the full conversation log.
Response and Model Controls
Settings opened with ⚙ are shared by all LLM Widget nodes in the same ComfyUI installation. Per-node system-prompt tabs and saved answers remain part of their individual workflows.
- Temperature controls response variation. A lower setting is useful when you want more consistent wording; a higher setting can encourage more varied drafts.
- Max answer tokens limits response length. Increase it for a detailed reference-image description or reduce it for a short prompt.
- Model thinking defaults to Off, which requests a final answer without displayed reasoning where the backend supports that behavior. Choose Keep when you want to retain reasoning the model produces. Some models may still emit unmarked reasoning that the widget cannot reliably separate from the answer.
- Request timeout (seconds) controls how long an HTTP backend may remain silent before the request times out. The default is 600 seconds, and the setting accepts 10 to 7,200 seconds.
- Unload after answering, available for local GGUF and ComfyUI text-encoder backends, releases the model after each response to free memory. Leaving it off avoids reloading the model for every question; ⏏ lets you release it manually.
For GGUF, Describe each media in its own pass first asks the model to describe each connected item, then uses those descriptions to answer your main question. This can take longer overall, but divides a large media request into smaller stages and gives you more opportunities to stop between them. If all descriptions fail, the widget warns you and falls back to attaching the media to the question.
ComfyUI-LLM-Widget Models
The extension does not come with one fixed set of named models. Instead, it lets you choose a backend: the service or local model that answers your question. The available choices differ in where the model runs and what files or service access they require.
| Backend | What it uses | When it may be useful |
|---|---|---|
| OpenAI-compatible chat completions | A service that accepts /v1/chat/completions requests | Use a compatible local server or hosted service already available to you. |
| OpenAI Responses API | A service that accepts /v1/responses requests | Use a Responses-compatible service, including OpenAI or another server that supports the format. |
| Gemini | Google’s native generateContent API | Use a Gemini model, including when you want to send a supported video file directly rather than selected frames. |
| GGUF | A local .gguf model through llama-cpp-python | Run a model locally. Image understanding also requires a compatible vision model, projector, and chat handler; without vision support, the model answers from text only. |
| ComfyUI text encoder | A compatible .safetensors model already in models/text_encoders | Reuse an LLM-based text encoder available in ComfyUI to write text from the editor. |
A ComfyUI text encoder must be capable of generating text. The documented compatible families include Qwen3-VL, Qwen3.5, and Gemma. CLIP-L and T5 are not text-generating choices for this feature. Some distributed encoder files omit the weights needed to produce meaningful text; those require a complete version of the model.
For GGUF, Vision projector can select a matching image projector automatically from beside the model, or none can force text-only use. Context size affects how much question and conversation history the local model can consider. GPU layers controls how much of the model is offloaded to the GPU; the documented value -1 offloads all layers.
The model you select affects the kind of answer you receive, its speed, and whether it can interpret images. For example, a text-only model can rewrite a prompt you provide, but it cannot independently describe an attached reference image.
What’s New with ComfyUI-LLM-Widget
The provided documentation describes these additions and changes, but does not give a complete version-by-version release history:
- OpenAI Responses API support: Adds
/v1/responsesas another way to connect to a compatible service. - ComfyUI text-encoder backend: Lets you use a suitable LLM-based
.safetensorstext encoder from ComfyUI directly in the widget. - Configurable HTTP timeout: Helps with slower models and media-heavy requests. The editor request no longer stops at its former 60-second frontend timeout.
- Image-format retry for compatible APIs: If an endpoint rejects an attached WebP image with the documented base64-image error, the widget converts eligible images to JPEG and retries once. This addresses a reported LM Studio image-format issue.
- Controls for model thinking and memory: Thinking settings help manage what appears in an answer, while unload controls help free local model memory before you run other work.
- Destination allowlist for HTTP requests: The installation owner controls which server destinations may receive questions, media, and API credentials.
Workflow compatibility note: Workflows saved with version 0.1.1 or earlier load question and answer in the wrong positions. Swap their contents once, then save the workflow again.
Troubleshooting ComfyUI-LLM-Widget
An Attached Image Is Skipped
- Look at the media indicators above the toolbar. A crossed-out red indicator means the widget cannot send that item from the editor.
- Check whether the image already exists as a file in ComfyUI’s
input/,output/, ortemp/folder. A connection fromVAE Decodealone does not create such a file before the workflow runs. - Use an already saved image, or turn on
generate_on_executeand run the workflow if you need to use an image generated by upstream nodes. - Confirm that Send the connected image / video is enabled if you want the model to receive media.
A Connected Question or System Prompt Is Ignored When I Press ✦
- Remember that ✦ does not run upstream nodes; it sends the text currently saved in the widget.
- Turn on
generate_on_execute. - Run the workflow to use the connected node’s actual output.
The Local Model Answers as Though It Cannot See the Image
- Confirm you selected a vision-capable model and, for GGUF, an appropriate vision projector rather than
none. - Check that the
llama-cpp-pythonbuild includes a compatible multimodal chat handler. A build without one can still answer text questions but cannot interpret the image. - If you need information about obtaining a handler-enabled build, consult the author’s referenced llama-cpp-python releases.
A Request Times Out or Appears to Take a Long Time
- Check the elapsed time in the toolbar.
- For an HTTP backend, review Request timeout (seconds). It limits how long the server can go without sending data, not the total duration of the entire exchange.
- For a large local GGUF media request, consider Describe each media in its own pass. Loading a model or evaluating a prompt may still take time before stopping is possible.
- Press ■ to stop an active editor request. A response that arrives after cancellation is discarded.
The API URL Is Rejected
The widget sends HTTP requests only to destinations listed in allowed_api_urls in llm_widget.json. The allowlist protects the installation from sending questions, images, or API credentials to an unapproved server. Its default destinations cover common local-server addresses and the official OpenAI and Gemini endpoints.
If you manage the ComfyUI installation and intentionally use another server or port, add its origin to allowed_api_urls in that file. The ⚙ dialog displays the list but cannot edit it. Changes take effect on the next request without a restart.
Frequently Asked Questions
Will pressing ✦ run my image-generation workflow?
No. It asks the model from the editor without queuing or running the graph.
Can I use the answer in another node?
Yes. Connect the text output. With generate_on_execute off, it passes the saved answer; with it on, the node generates a new answer during execution.
Will the model remember an image from an earlier conversation turn?
No. Previous exchanges are resent as text only. Reattach the image if the next question requires the model to inspect it again.
Where is my conversation saved?
The conversation log is the editable answer field, saved with the workflow. The widget does not keep a separate conversation history.
Why can’t I use the ComfyUI text encoder while a workflow is running?
The widget declines editor requests to that backend during workflow execution to avoid conflicting model-memory management. It is also best to avoid queueing a workflow while the encoder is generating an answer.
Can I stop a local-model response immediately?
The ■ button requests a stop, but model loading and some prompt-evaluation work cannot be interrupted immediately. The response stops at a point the backend can interrupt.
Why did an older workflow swap my question and answer?
This affects workflows saved with version 0.1.1 or earlier. Exchange the two values and save the workflow again.
Learn More about ComfyUI-LLM-Widget
- Visual reference: Compare the node screenshot and settings screenshot with the controls described above.
- Vision-capable GGUF support: See the referenced llama-cpp-python releases when checking for a build with multimodal chat handlers.
- LM Studio image error: The linked issue report provides background on the image-format rejection addressed by the widget’s JPEG retry.
