qwen_chat_model:
qwen_chat_model generates a text response from a Qwen instruction-tuned language model. It formats a conversation, runs the selected model, and returns only the newly generated reply, making it useful for drafting ideas, writing descriptions, or developing text for an AI art workflow. The node can download a missing model checkpoint and supports reduced-precision loading to help manage memory use. Its generation path requires CUDA; it is not a CPU-only text-generation node.
qwen_chat_model Input Parameters:
system
A required STRING containing the overall instructions for the conversation. Use it to establish the model’s role, writing style, or boundaries, such as asking for concise visual descriptions. The node places this text in a system message before your prompt, so it can influence how the reply is written. The field supports multiple lines. No default or length limits are specified in the source.
prompt
A required STRING containing the request you want the model to answer. Be specific about the subject, desired format, and level of detail to make the response more useful; for example, describe the scene you want turned into an image-generation prompt. The node sends this text as a user message after the system message. The field supports multiple lines. No default or length limits are specified in the source.
model
A required selection that determines which Qwen checkpoint produces the response. Options are Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Qwen2.5-32B-Instruct, Qwen3-8B-Instruct, Qwen3-4B-Thinking-2507, and Qwen3-4B-Instruct-2507. The default is Qwen3-4B-Instruct-2507. The node looks for the selection in its local LLM model folder and attempts to download it if absent. Larger checkpoints generally demand more memory; the source does not define a performance or quality ranking among these options.
quantization
A required selection controlling how the model is loaded: none, 4bit, or 8bit. The 4bit and 8bit options request bitsandbytes quantized loading, which can reduce model memory requirements; none does not request either quantization mode. This setting affects model loading, not the wording of your prompt. The source specifies no default.
temperature
A required FLOAT passed to text generation, with a source-defined default of 0.7 and no stated minimum or maximum. Temperature ordinarily controls variation in generated text, but this node does not explicitly enable sampling with do_sample=True. Depending on the installed Transformers version and generation configuration, changing temperature may therefore have no effect or produce a warning. Do not rely on this control alone to make responses more varied.
max_new_tokens
A required INT setting the maximum number of new tokens the model may generate, with a default of 512 and no source-specified minimum or maximum. Tokens are pieces of text rather than a fixed number of words. Raise the value when a response ends too early, or lower it when you want to cap response length and generation work. This limit applies to the reply, not to the text you provided.
seed
A required INT used to set PyTorch’s random seed before generation. The default is -1; at that value, the node does not set a seed. Any other value is passed to torch.manual_seed. A fixed seed can help when comparing runs involving randomness, but it does not by itself guarantee identical replies, particularly because sampling is not explicitly enabled. The source specifies no other numeric limits.
model_loaded_permanently
An optional BOOLEAN with a default of False. The value is passed to the model cache when the node obtains the selected checkpoint, requesting its permanent-loading behavior. Keeping a model available may reduce loading work on later runs, but can retain memory. The cache implementation is not provided, so its exact retention policy cannot be determined from this source.
offload_after_used
An optional BOOLEAN with a default of False. The value is passed to the model cache when generation ends, including when an error occurs during generation. Enable it when you want to request offloading after use, potentially freeing resources for other work. The cache implementation is not provided, so the exact destination and amount of memory released are not specified.
qwen_chat_model Output Parameters:
string
The node’s sole STRING output is the generated reply. It removes the input conversation tokens from the model’s result, then decodes the remaining tokens while skipping special tokens. Connect this output to a text display or another node that accepts text. It contains text, not an image or a structured conversation record, and it does not include the system message or prompt unless the model independently repeats them.
qwen_chat_model Usage Tips:
- Put enduring instructions in
systemand the immediate task inprompt. For example, set a consistent visual-writing style insystem, then describe the specific scene inprompt. - Start with the default
Qwen3-4B-Instruct-2507when testing a workflow. If loading runs out of memory, try4bitor8bitquantization, or select a smaller checkpoint. - Set
max_new_tokensto match the amount of text you need. A short caption needs less room than several detailed scene variations. - Treat
temperatureandseedas experimental controls rather than guarantees of creative variation or repeatability: the node does not explicitly turn on sampling. - If you repeatedly run the same model and have sufficient memory, try
model_loaded_permanently. When memory is needed elsewhere, tryoffload_after_used; their exact effects depend on the cache implementation. - Allow time and disk space for the first use of a checkpoint, because the node attempts a download when its local model folder does not exist.
qwen_chat_model Common Errors and Solutions:
AssertionError: Torch not compiled with CUDA enabled or Found no NVIDIA driver on your system
- Explanation: The node moves its tokenized input to
cuda. A setup without working CUDA support can fail even though part of model loading checks whether CUDA is available. - Solution: Run the node in an environment with a compatible NVIDIA GPU, driver, and CUDA-enabled PyTorch installation. This source does not provide a CPU fallback.
torch.OutOfMemoryError: CUDA out of memory
- Explanation: The selected checkpoint, input text, or generated reply requires more GPU memory than is available.
- Solution: Try a smaller
model, select4bitor8bitquantization, shorten the prompt, or lowermax_new_tokens. Close other GPU-intensive work before retrying.
OSError: ... is not a local folder and is not a valid model identifier or a checkpoint download error
- Explanation: When the local model folder is missing, the node tries to download
qwen/<selected model>. Access, connectivity, repository resolution, or storage problems can interrupt that step; the exact message depends on the underlying library. - Solution: Check your internet connection, access to the selected repository, available disk space, and write permission for ComfyUI’s model directory. Confirm that the selected checkpoint matches one of the node’s listed options.
ModuleNotFoundError: No module named 'bitsandbytes' or a bitsandbytes compatibility error
- Explanation: The
4bitand8bitsettings request bitsandbytes quantization, which may be unavailable or incompatible with the installed environment. - Solution: Install a bitsandbytes version compatible with your PyTorch and CUDA setup, or select
noneif you have enough memory to load the model without that quantization.
ValueError during generate concerning temperature or generation settings
- Explanation: The node passes
temperatureto generation without explicitly enabling sampling. The installed Transformers version or model configuration may reject or warn about a temperature setting that is not applicable. - Solution: Check the full error text and use a temperature accepted by your installed setup. Keep in mind that changing this field may not alter replies while sampling remains disabled in the node source.
ValueError: max_new_tokens must be greater than 0 or a similar generation-length error
- Explanation: Generation needs a valid positive allowance for new tokens. Although the interface declares no minimum, the generation library may reject zero or negative values.
- Solution: Set
max_new_tokensto a positive integer, such as the source default of512, and adjust it to the response length you need.
RuntimeError or ValueError while loading a checkpoint
- Explanation: Model loading can fail because the selected checkpoint is incomplete, the installed Transformers stack cannot load it, or the available hardware cannot support the requested loading configuration. The precise error depends on the environment.
- Solution: Read the first underlying error in the log, verify the checkpoint files and installed dependencies, then try a smaller model or another quantization option. If a download was interrupted, repair or replace the incomplete local checkpoint before retrying.
