Save 4 hours! We auto-setup your workflow! Free!

Drop your workflow.json — we handle every dependency, custom node, and model. Just open the link and run.

Auto-Setup Workflow Json (Free) Now!
ComfyUI > Nodes > ComfyUI_Qwen_chat_models > qwen_chat_model

ComfyUI Node: qwen_chat_model

Class Name

qwen_chat_model

Category
Comfyui_Qwen
Author
constantlygrowup (Account age: 440days)
Extension
ComfyUI_Qwen_chat_models
Latest Updated
2025-12-30
Github Stars
0.0K

How to Install ComfyUI_Qwen_chat_models

Install this extension via the ComfyUI Manager by searching for ComfyUI_Qwen_chat_models
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter ComfyUI_Qwen_chat_models in the search bar
After installation, click the Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • 16GB VRAM to 80GB VRAM GPU machines
  • 400+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 200+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

qwen_chat_model Description

Generates text replies from Qwen chat prompts using CUDA, with selectable models and quantization.

qwen_chat_model:

qwen_chat_model generates a text response from a Qwen instruction-tuned language model. It formats a conversation, runs the selected model, and returns only the newly generated reply, making it useful for drafting ideas, writing descriptions, or developing text for an AI art workflow. The node can download a missing model checkpoint and supports reduced-precision loading to help manage memory use. Its generation path requires CUDA; it is not a CPU-only text-generation node.

qwen_chat_model Input Parameters:

system

A required STRING containing the overall instructions for the conversation. Use it to establish the model’s role, writing style, or boundaries, such as asking for concise visual descriptions. The node places this text in a system message before your prompt, so it can influence how the reply is written. The field supports multiple lines. No default or length limits are specified in the source.

prompt

A required STRING containing the request you want the model to answer. Be specific about the subject, desired format, and level of detail to make the response more useful; for example, describe the scene you want turned into an image-generation prompt. The node sends this text as a user message after the system message. The field supports multiple lines. No default or length limits are specified in the source.

model

A required selection that determines which Qwen checkpoint produces the response. Options are Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Qwen2.5-32B-Instruct, Qwen3-8B-Instruct, Qwen3-4B-Thinking-2507, and Qwen3-4B-Instruct-2507. The default is Qwen3-4B-Instruct-2507. The node looks for the selection in its local LLM model folder and attempts to download it if absent. Larger checkpoints generally demand more memory; the source does not define a performance or quality ranking among these options.

quantization

A required selection controlling how the model is loaded: none, 4bit, or 8bit. The 4bit and 8bit options request bitsandbytes quantized loading, which can reduce model memory requirements; none does not request either quantization mode. This setting affects model loading, not the wording of your prompt. The source specifies no default.

temperature

A required FLOAT passed to text generation, with a source-defined default of 0.7 and no stated minimum or maximum. Temperature ordinarily controls variation in generated text, but this node does not explicitly enable sampling with do_sample=True. Depending on the installed Transformers version and generation configuration, changing temperature may therefore have no effect or produce a warning. Do not rely on this control alone to make responses more varied.

max_new_tokens

A required INT setting the maximum number of new tokens the model may generate, with a default of 512 and no source-specified minimum or maximum. Tokens are pieces of text rather than a fixed number of words. Raise the value when a response ends too early, or lower it when you want to cap response length and generation work. This limit applies to the reply, not to the text you provided.

seed

A required INT used to set PyTorch’s random seed before generation. The default is -1; at that value, the node does not set a seed. Any other value is passed to torch.manual_seed. A fixed seed can help when comparing runs involving randomness, but it does not by itself guarantee identical replies, particularly because sampling is not explicitly enabled. The source specifies no other numeric limits.

model_loaded_permanently

An optional BOOLEAN with a default of False. The value is passed to the model cache when the node obtains the selected checkpoint, requesting its permanent-loading behavior. Keeping a model available may reduce loading work on later runs, but can retain memory. The cache implementation is not provided, so its exact retention policy cannot be determined from this source.

offload_after_used

An optional BOOLEAN with a default of False. The value is passed to the model cache when generation ends, including when an error occurs during generation. Enable it when you want to request offloading after use, potentially freeing resources for other work. The cache implementation is not provided, so the exact destination and amount of memory released are not specified.

qwen_chat_model Output Parameters:

string

The node’s sole STRING output is the generated reply. It removes the input conversation tokens from the model’s result, then decodes the remaining tokens while skipping special tokens. Connect this output to a text display or another node that accepts text. It contains text, not an image or a structured conversation record, and it does not include the system message or prompt unless the model independently repeats them.

qwen_chat_model Usage Tips:

  • Put enduring instructions in system and the immediate task in prompt. For example, set a consistent visual-writing style in system, then describe the specific scene in prompt.
  • Start with the default Qwen3-4B-Instruct-2507 when testing a workflow. If loading runs out of memory, try 4bit or 8bit quantization, or select a smaller checkpoint.
  • Set max_new_tokens to match the amount of text you need. A short caption needs less room than several detailed scene variations.
  • Treat temperature and seed as experimental controls rather than guarantees of creative variation or repeatability: the node does not explicitly turn on sampling.
  • If you repeatedly run the same model and have sufficient memory, try model_loaded_permanently. When memory is needed elsewhere, try offload_after_used; their exact effects depend on the cache implementation.
  • Allow time and disk space for the first use of a checkpoint, because the node attempts a download when its local model folder does not exist.

qwen_chat_model Common Errors and Solutions:

AssertionError: Torch not compiled with CUDA enabled or Found no NVIDIA driver on your system

  • Explanation: The node moves its tokenized input to cuda. A setup without working CUDA support can fail even though part of model loading checks whether CUDA is available.
  • Solution: Run the node in an environment with a compatible NVIDIA GPU, driver, and CUDA-enabled PyTorch installation. This source does not provide a CPU fallback.

torch.OutOfMemoryError: CUDA out of memory

  • Explanation: The selected checkpoint, input text, or generated reply requires more GPU memory than is available.
  • Solution: Try a smaller model, select 4bit or 8bit quantization, shorten the prompt, or lower max_new_tokens. Close other GPU-intensive work before retrying.

OSError: ... is not a local folder and is not a valid model identifier or a checkpoint download error

  • Explanation: When the local model folder is missing, the node tries to download qwen/<selected model>. Access, connectivity, repository resolution, or storage problems can interrupt that step; the exact message depends on the underlying library.
  • Solution: Check your internet connection, access to the selected repository, available disk space, and write permission for ComfyUI’s model directory. Confirm that the selected checkpoint matches one of the node’s listed options.

ModuleNotFoundError: No module named 'bitsandbytes' or a bitsandbytes compatibility error

  • Explanation: The 4bit and 8bit settings request bitsandbytes quantization, which may be unavailable or incompatible with the installed environment.
  • Solution: Install a bitsandbytes version compatible with your PyTorch and CUDA setup, or select none if you have enough memory to load the model without that quantization.

ValueError during generate concerning temperature or generation settings

  • Explanation: The node passes temperature to generation without explicitly enabling sampling. The installed Transformers version or model configuration may reject or warn about a temperature setting that is not applicable.
  • Solution: Check the full error text and use a temperature accepted by your installed setup. Keep in mind that changing this field may not alter replies while sampling remains disabled in the node source.

ValueError: max_new_tokens must be greater than 0 or a similar generation-length error

  • Explanation: Generation needs a valid positive allowance for new tokens. Although the interface declares no minimum, the generation library may reject zero or negative values.
  • Solution: Set max_new_tokens to a positive integer, such as the source default of 512, and adjust it to the response length you need.

RuntimeError or ValueError while loading a checkpoint

  • Explanation: Model loading can fail because the selected checkpoint is incomplete, the installed Transformers stack cannot load it, or the available hardware cannot support the requested loading configuration. The precise error depends on the environment.
  • Solution: Read the first underlying error in the log, verify the checkpoint files and installed dependencies, then try a smaller model or another quantization option. If a download was interrupted, repair or replace the incomplete local checkpoint before retrying.

qwen_chat_model Related Nodes

Go back to the extension to check out more related nodes.
ComfyUI_Qwen_chat_models
RunComfy
Copyright 2025 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals. RunComfy also provides AI Models, enabling artists to harness the latest AI tools to create incredible art.

qwen_chat_model