ComfyUI-QwenVL-F Introduction
ComfyUI-QwenVL-F adds image-understanding and text-generation nodes to ComfyUI. You can use it to describe an image, ask questions about visual details, analyze a sequence of video frames, or turn a rough idea into a more detailed prompt. For example, you could feed a reference image into a QwenVL node and ask it to describe the lighting, composition, and clothing before using that description elsewhere in your workflow.
This is a fork of 1038lab/ComfyUI-QwenVL. Its most important difference is local-only model discovery: the nodes list models already on your computer and never download one in the background. You choose where to obtain and store your models, then reload ComfyUI so the nodes can find them.
Before adding this extension to a new workflow: The author notes that recent ComfyUI versions can perform much of the same work with ComfyUI’s built-in TextGenerate node. Consider that built-in option first if you do not need this fork’s particular nodes or controls.
Example of the GGUF Advanced node
How ComfyUI-QwenVL-F Works
Think of a vision-language model as a visual assistant: you give it an image and a question, and it writes a response based on what it can interpret. A prompt enhancer works similarly with text: you give it a starting idea, and it generates a fuller text response. These outputs are text, which you can read or pass to later nodes in a ComfyUI workflow.
The extension offers two ways to run models:
- Transformers (HF) nodes use models stored as checkpoint folders. For image tasks, the folder must contain a vision-capable model.
- GGUF nodes use
.ggufmodel files throughllama-cpp-python. For image understanding, a compatible vision projector, often called an mmproj file, is also needed. The projector is the part that helps the model receive visual information.
The fork scans local model folders, including subfolders, to populate its model menus. By default, it scans ComfyUI/models/text_encoders and ComfyUI/models/LLM. You can change the scan locations through base_dirs in hf_models.json and gguf_models.json. Those locations can refer to a ComfyUI model-folder key, a subfolder of ComfyUI/models, or an absolute path. When two locations contain a model with the same listed name, the first match in scan order is used. Model entries elsewhere in those JSON files are reference lists, not download instructions or the source of the dropdown choices.
A typical image workflow is straightforward:
- Add a QwenVL node and connect an image source.
- Choose a locally available vision model.
- Select a preset prompt or write your own question, such as “Describe the color palette and camera angle.”
- Run the workflow and use the resulting text in your next creative step.
The Advanced nodes offer more control when you need to compare references or adjust how a response is generated.
ComfyUI-QwenVL-F Features
Six nodes for different tasks. The HF and GGUF backends each provide a standard QwenVL node, an Advanced QwenVL node, and a Prompt Enhancer. Choose a standard node for a quick visual description, an Advanced node for additional controls, or a Prompt Enhancer when your starting point is text rather than an image. The HF Prompt Enhancer can use a text-only Qwen model or a vision-language model in text mode.
Preset or custom prompts. Presets provide a quick starting point. A custom prompt lets you ask for a specific kind of answer, such as a concise subject description or a detailed account of lighting and composition. When supplied, the custom prompt overrides the preset.
Images, video frames, and multiple references. QwenVL nodes support image input and video frame sequences. The HF Advanced node includes frame_count, which controls how many frames are sampled from a video sequence. The Advanced HF and GGUF nodes also accept up to three image inputs, named image, image2, and image3. For example, you could provide two character references and ask the model to compare their outfits.
Thinking mode. The enable_thinking switch is off by default. It enables the <think>...</think> reasoning mode for Qwen3-VL Thinking models. Use an Instruct model or leave the switch off when you want a more direct response.
HF generation and performance controls. On supported HF nodes, max_tokens limits response length, while seed helps make repeated runs more consistent. The Advanced node adds temperature and top_p for sampling, num_beams for beam search, and repetition_penalty to discourage repeated wording. Higher temperature generally allows more varied wording; when num_beams is greater than 1, temperature and top_p sampling do not apply. It also offers device selection and an optional torch.compile setting.
HF nodes provide 4-bit, 8-bit, and unquantized loading options to balance memory use and performance; the quantization setting is ignored for models that are already pre-quantized, such as FP8 checkpoints. The attention_mode setting can automatically try SageAttention, then Flash Attention, then PyTorch SDPA. FP8 and BitsAndBytes models use SDPA for compatibility. Keep Model Loaded can speed up repeated runs, but turning it off may free memory for other ComfyUI nodes.
GGUF controls. The GGUF Advanced node lets you adjust settings such as context length and GPU layers. Its mmproj_name menu lists locally found projector files whose names match *mmproj*.gguf. Leave it on auto to use the projector beside the selected model, or choose a file explicitly when you need a different compatible projector. The GGUF Prompt Enhancer has a ctx control, defaulting to 8192, for how much context the model can consider.
The GGUF Advanced node and GGUF Prompt Enhancer also have mtp_draft_tokens for supported Qwen3.5, Qwen3.6, and Qwen3.8 MTP GGUF builds. 0 turns this speculative-decoding feature off; 2 is the author’s suggested starting point. It requires llama-cpp-python v0.3.48 or later, is skipped while a vision projector is loaded, and falls back to ordinary decoding if the chosen GGUF has no MTP data.
ComfyUI-QwenVL-F Models
Start with a model that matches your task. A vision-language (VL) model is for image questions; a text-only model is for prompt enhancement without an image. “Instruct” models suit direct requests, while “Thinking” models support the extension’s thinking mode. Larger models generally require more resources. FP8 and GGUF variants offer different storage and runtime options, but the best choice depends on your hardware and the files you have available.
| Model family | Listed models | When to use it |
|---|---|---|
| Qwen3-VL HF | 2B, 4B, 8B, and 32B, each listed in Instruct and Thinking versions and corresponding FP8 versions | Image understanding with the HF nodes. A 4B Instruct model is a practical starting point from the original README. |
| Qwen2.5-VL HF | 3B-Instruct and 7B-Instruct | Image understanding with the earlier Qwen-VL family. |
| Qwen3 HF text | Qwen3-0.6B; Qwen3-4B-Instruct-2507; qwen3-4b-Z-Image-Engineer | Text-only work with the HF Prompt Enhancer. |
| Qwen3 GGUF text | Qwen3-4B-GGUF, listed with Q4, Q5, Q6, and Q8 file variants | Text-only work with the GGUF Prompt Enhancer. |
| Qwen3-VL GGUF | 4B and 8B, each listed in Instruct and Thinking versions | Image understanding through the GGUF nodes when paired with a compatible mmproj file. |
The reference model repositories are Qwen3-VL-2B-Instruct, Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct, and Qwen3-VL-32B-Instruct, with Thinking and FP8 variants listed in the original model catalog. Other listed HF sources include Qwen2.5-VL-3B-Instruct, Qwen2.5-VL-7B-Instruct, Qwen3-0.6B, Qwen3-4B-Instruct-2507, and qwen3-4b-Z-Image-Engineer.
The listed GGUF repositories are Qwen3-4B-GGUF, Qwen3-VL-4B-Instruct-GGUF, Qwen3-VL-8B-Instruct-GGUF, Qwen3-VL-4B-Thinking-GGUF, and Qwen3-VL-8B-Thinking-GGUF. For a GGUF vision model, obtain its matching mmproj file as well.
The fork additionally describes GGUF handling for Qwen3.5, Qwen3.6, Qwen3.8, and Gemma 4. It does not provide a specific download catalog for those families in the supplied README. Model menus show files found locally, rather than every model named above.
What's New with ComfyUI-QwenVL-F
Fork changes (no version number supplied):
- Replaced automatic downloads across all six nodes with local-only model discovery and missing-model errors that list the folders checked. Older workflow model names, including
[local] relative/pathnames, continue to resolve. - Added explicit
mmproj_nameselection to GGUF Advanced nodes and up to three image inputs to the Advanced HF and GGUF nodes. - Added
enable_thinkingcontrols and actxcontrol for the GGUF Prompt Enhancer. - Added GGUF handling for Gemma 4 and newer Qwen3.5, Qwen3.6, and Qwen3.8 families, plus optional MTP speculative decoding for supported GGUF builds.
- Added
tools/install_helper.py, which reports dependency commands for a selected Python environment but does not run them. Its details are in the vision backend guide.
Upstream changes recorded in the original README:
| Version and date | Change | Why it matters |
|---|---|---|
| v2.1.1 — February 8, 2026 | Compatibility fix for Transformers 4.x and 5.x. Details | Helps HF nodes work across those dependency versions. |
| v2.1.0 — February 5, 2026 | Added SageAttention support, automatic attention selection, better FP8 handling, progress feedback, and improved cache clearing. Details | Offers more performance choices and clearer feedback while a run is in progress. |
| v2.0.0 — December 22, 2025 | Added GGUF nodes and Prompt Enhancer nodes. Details | Expanded model-format and text-only workflow options. |
| v1.1.0 — November 10, 2025 | Overhauled runtime controls, including attention selection, caching, quantization, and torch.compile options. Details | Made it easier to tune runs for available hardware. |
| v1.0.4 — October 31, 2025 | Added custom-model support. Details | Broadened model choice. |
| v1.0.3 — October 22, 2025 | Updated the model list. Details | Expanded the listed choices at that time. |
| v1.0.0 — October 17, 2025 | Initial release with Qwen3-VL and Qwen2.5-VL support, standard and advanced nodes, prompts, image/video input, and generation controls. | Established the core visual-analysis workflow. |
The upstream release notes mention automatic model downloading. That behavior does not apply to this fork.
Troubleshooting ComfyUI-QwenVL-F
A model is missing from the dropdown
- Confirm that the model is already on your computer; this fork will not download it.
- For an HF model, check that its checkpoint folder contains
config.jsonalongside its weight files. A QwenVL image node lists vision checkpoints, while the HF Prompt Enhancer also lists text-only checkpoints. - For a GGUF model, check that the
.gguffile is inside a scanned folder. - Check the configured
base_dirsinhf_models.jsonorgguf_models.json, then reload ComfyUI. If the node reports an error, use the folder list in that message to see where it looked.
A GGUF model loads but cannot process an image
- Check that you selected a vision-capable GGUF model, not a text-only Qwen3 GGUF.
- Check that its compatible mmproj file is available locally.
- On a GGUF Advanced node, try selecting that file with
mmproj_nameinstead ofauto. Theautooption looks for a projector beside the model. - If a required vision chat handler is missing, consult the llama-cpp-python vision guide.
A run uses too much GPU memory or feels slow
- On an HF node, try a smaller model or a lower-memory quantization option. A 4-bit setting can reduce memory use, although it may be slower on some systems.
- Turn off Keep Model Loaded if other nodes need the memory after the QwenVL step.
- For repeated runs with enough available memory, turn it back on to avoid loading the model each time.
- Leave
attention_modeonautounless you are investigating a specific compatibility or performance issue. FP8 and BitsAndBytes models use SDPA.
Frequently asked questions
- Will selecting a model download it? No. This fork discovers local models only.
- Can I use an older saved workflow? The fork retains resolution for older saved model names, including the
[local] relative/pathform, provided the corresponding model can be found. - Do I need an mmproj file for a text-only prompt? No. The mmproj is for GGUF vision input.
- Does a fixed seed guarantee identical wording every time? The seed is provided for more reproducible generation, but other model and generation settings also affect the response.
- Can MTP speed up an image-analysis run? The documented MTP feature is text-only and is skipped while a vision projector handler is loaded.
For a visual example of the node layout, see the GGUF Advanced screenshot. The upstream README also links an example workflow image and JSON workflow.
Learn More about ComfyUI-QwenVL-F
- Upstream ComfyUI-QwenVL repository: background on the original nodes and links to its release notes. Remember that its automatic-download instructions do not describe this fork.
- Example ComfyUI workflow: a starting point for seeing how a QwenVL node fits into an image workflow.
- Qwen model repositories: model pages for reviewing the Qwen checkpoints and GGUF files listed above.
- ComfyUI repository: background on the workflow application and its built-in capabilities.
- GGUF vision backend guide: technical reference when a GGUF vision model or chat handler does not work.
