Save 4 hours! We auto-setup your workflow! Free!

Drop your workflow.json — we handle every dependency, custom node, and model. Just open the link and run.

Auto-Setup Workflow Json (Free) Now!
ComfyUI > Nodes > Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8

ComfyUI Extension: Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8

Repo Name

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8

Author
T8mars (Account age: 1793 days)
Nodes
View all nodes(1)
Latest Updated
2026-11-09
Github Stars
0.02K

How to Install Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8

Install this extension via the ComfyUI Manager by searching for Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 in the search bar
After installation, click the Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • 16GB VRAM to 80GB VRAM GPU machines
  • 400+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 200+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Description

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 adds native ComfyUI nodes for image generation and editing with eight structural control modes, reference-image guidance, and masked inpainting.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Introduction

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 adds native ComfyUI nodes for Alibaba PAI’s Qwen Image 2.1 Fun ControlNet Union. It helps you guide image generation and editing with visual information—not just text.

A text prompt describes what you want, but it may not reliably place a person, preserve a room’s layout, or follow a sketch. This extension lets you supply a control map: an image that describes structure, such as outlines, depth, or body pose. You can also provide photographs as editing references and masks to guide changes toward selected areas.

One Union checkpoint supports eight control types:

  • Canny
  • Depth
  • Grayscale
  • HED
  • Lineart
  • MLSD
  • Pose
  • Scribble

It also supports masked inpainting, where a mask indicates which parts of an image should be regenerated.

For AI artists, this makes it easier to:

  • Develop finished artwork from sketches or line drawings.
  • Explore different appearances while guiding composition with edges or depth.
  • Generate characters following a prepared pose.
  • Edit an existing photograph with additional visual references.
  • Guide local changes using a painted mask.

The extension uses ComfyUI’s native Qwen Image 2.1 model and conditioning pipeline rather than a separate image-generation system.

Example result using Canny control

The author’s pictured Canny example used the default 40 sampling steps and an output size of 800 × 1312 pixels. This is an example, not a required setting for every image.

Important: The model weights are licensed for non-commercial use only, unless separately licensed by Qwen. See the license section before using them in a project.

How Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Works

Think of the control map as a drawing guide

A useful analogy is an artist working with a sketch underneath a painting:

  • Your prompt describes the desired content and appearance.
  • Your control map provides structural guidance.
  • The Qwen Image 2.1 base model creates the image.
  • The Union ControlNet helps the model follow the supplied structure.

For example, you could use the same prepared line drawing with prompts for a watercolor illustration or a cinematic scene. The drawing guides the arrangement of shapes, while the prompt describes the intended treatment. The result is still generated, so exact correspondence is not guaranteed.

“Union” means several control types share one checkpoint

You do not need a different Union checkpoint for each supported control type. Instead, you select the mode that matches your prepared input.

The mode selector does not turn a photograph into a control map. Selecting Pose, for example, does not detect a person’s pose from an ordinary photograph. You must first prepare the appropriate map using a suitable tool or workflow.

The basic sequence is:

  1. Prepare a control map.
  2. Connect it to control_image on Apply Qwen Image 2.1 UNION.
  3. Choose the matching control mode.
  4. Supply your text conditioning and the required model connections.
  5. Run the workflow to generate an image.

Reference images and control images have different jobs

A control image describes structure. A reference image provides visual information for an edit.

When editing a photograph, use Qwen 2.1 UNION Image Reference Encode. This node prepares the reference information and produces:

  • Positive conditioning: information guiding what the model should create.
  • Negative conditioning: information used by the workflow to guide against unwanted results.
  • Latent: the compressed image representation used during generation.

Connect all three outputs to the sampler as shown in the example workflow.

Always use this encoder’s latent for reference-image editing. Replacing it with an unrelated empty latent can shift the edit or cause the reference guidance to be lost, especially when the aspect ratios differ.

Masks guide local editing, but do not lock pixels

For masked editing, provide the source photograph through inpaint_image and a mask through mask:

  • White guides regeneration.
  • Black guides preservation.

Think of the mask as an instruction about where to focus the edit, not as an absolute protective barrier. The mask conditions the model and does not guarantee unchanged pixels outside the white area.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Features

Eight structural control modes

Choose the mode according to the kind of visual guidance you have prepared.

ModeWhat the map communicatesExample artistic use
CannySharp edges and visible boundaries.Guide the contours of an object or the layout of a scene.
DepthRelative distance and spatial relationships.Guide foreground/background placement in an interior or landscape.
GrayscaleLight and dark tonal information without color.Use a monochrome image as tonal guidance while describing color in your prompt.
HEDSofter detected edges and boundaries.Guide forms with a soft-edge map rather than a sharp Canny map.
LineartDefined drawing lines and outlines.Develop a prepared line drawing into a finished illustration.
MLSDStraight-line structure.Guide architectural or geometric scenes.
PoseA prepared body-pose representation.Guide a character’s stance or gesture.
ScribbleLoose, simplified drawing marks.Explore a composition from a rough sketch.

These modes provide different kinds of guidance; they are not interchangeable labels. A pose map should use Pose, while a prepared depth map should use Depth.

Reference-image editing

The Qwen 2.1 UNION Image Reference Encode node lets you use an existing image as the target of an edit.

The inputs have distinct roles:

InputWhat to provide
control_imageA prepared map matching the selected control mode.
reference_imageImage 1: the original image to edit. It determines the output canvas size and aspect ratio.
reference_image_2 / reference_image_3Additional appearance or style references.
inpaint_imageThe source image for masked inpainting.
maskA mask using white for regeneration and black for preservation.

For a two-image edit:

  • reference_image is image 1, the edit target.
  • reference_image_2 is image 2, an additional visual reference.
  • Refer to them in your prompt as <image1> and <image2>.

For example:

Edit <image1> using the color palette and illustration style of <image2>.

This wording makes the intended roles clearer, but it does not guarantee an exact transfer. In particular, structural control does not guarantee that another person’s face or outfit will be reproduced exactly.

Masked inpainting

Masked inpainting combines a source photograph, a mask, and the workflow’s other guidance.

To prepare a local edit:

  1. Choose the photograph you want to change.
  2. Connect it to inpaint_image.
  3. Create a mask with white over the intended editing area and black elsewhere.
  4. Connect the mask to mask.
  5. Describe the desired change in your prompt.

For reference-based masked edits, also follow the reference encoder connections in the supplied workflow. After generation, inspect both the edited region and the surrounding image because preservation is guidance rather than a pixel-level guarantee.

Canvas and aspect-ratio handling

For reference editing, the first reference image determines the canvas. Prepare the control map with the same aspect ratio so its structure does not stretch.

For example, when editing a portrait-oriented photograph, use a portrait-oriented control map rather than a square map containing the same subject.

Extremely wide or tall inputs are rejected if they would produce an output side above 4096 pixels. You can address this by reducing resolution or cropping/padding the input.

Multiple variations from one control

You can explore different seeds while keeping the same structural guide. A seed is the starting randomness used during generation; changing it can produce different interpretations.

However, each connected condition input—control_image, inpaint_image, or mask—must contain one image. The native patch otherwise uses only the first item in each batch.

To sample several seeds from the same control, repeat the latent after the aspect-ratio node, rather than batching multiple condition images.

Native compatibility and checkpoint validation

The extension requires ComfyUI 0.37.0 or newer with native Qwen Image 2.1 base-model support.

It includes Union compatibility for releases before ComfyUI upstream PR #16519. Its loader also validates the expected 16-block Union checkpoint, helping detect an incompatible model file before generation.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Models

The supplied configuration uses one Union checkpoint alongside the Qwen Image 2.1 base-model components. These files perform different jobs; they are not alternative artistic styles.

ComponentDocumented fileRole
Union ControlNetQwen-Image-2.1-Fun-Controlnet-Union-ComfyUI.safetensorsAdds the supported structural controls and masked-inpainting guidance.
Base diffusion modelqwen_image_2.1_int8_convrot.safetensorsPerforms the image-generation process.
Text encoderqwen3vl_8b_int8_convrot.safetensorsEncodes prompt information for the model.
VAEqwen_image_2.1_vae_bf16.safetensorsConverts between visible images and the compressed latent representation used during generation.

Model resources

The converted Union checkpoint preserves the original tensor payload bytes; only the safetensors header metadata changes. In simpler terms, the conversion changes the file’s descriptive information rather than the stored model values.

The provided documentation does not list alternative Union sizes or performance comparisons between model variants.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Example Workflows

The workflow folder contains:

Start with a supplied example

  1. Make the sample images from assets/ available in ComfyUI/input/.
  2. Drag a UI workflow JSON from workflows/ onto the ComfyUI canvas.
  3. Check that the image inputs and model selections resolve correctly.
  4. Run the example before replacing its inputs.
  5. Change one element at a time, such as the prompt, control map, or seed.

Files ending in .api.json are API prompts, not workflows intended to be loaded onto the visual canvas.

Starting with an example is especially useful for reference editing because the encoder’s conditioning and latent connections must stay coordinated.

Troubleshooting Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8

The result does not follow the selected control mode

  1. Inspect the image connected to control_image.
  2. Confirm that it is already a prepared control map, not an unprocessed photograph.
  3. Select the mode matching that map.
  4. Compare your connections with the corresponding example workflow.

Remember: changing the mode selector does not preprocess the image.

A reference edit shifts, loses the source image, or uses the wrong framing

  1. Confirm that the photograph you want to edit is connected to reference_image.
  2. Find Qwen 2.1 UNION Image Reference Encode.
  3. Connect its positive and negative outputs to the sampler.
  4. Connect its latent output to the sampler.
  5. Remove any unrelated empty latent being used instead.
  6. Check that the control map matches the first reference image’s aspect ratio.

The second reference is additional guidance; it does not determine the editing canvas.

The control map looks stretched in the result

  1. Compare the width-to-height proportions of the target image and control map.
  2. Crop or pad the map to match the target canvas.
  3. Check that important structures still align after that adjustment.
  4. Run the workflow again.

Changes appear in the wrong masked region

  1. Inspect the mask visually.
  2. Make the intended editing area white.
  3. Make the area you want preserved black.
  4. Confirm that the correct source photograph is connected to inpaint_image.
  5. Check that the mask lines up with the source photograph.

If changes also appear outside the white area, this is not necessarily a wiring error: the mask does not enforce exact pixel preservation.

Only the first image in a batch affects the result

  1. Check control_image, inpaint_image, and mask.
  2. Supply a single image to each connected condition input.
  3. For multiple variations from the same guidance, repeat the latent after the aspect-ratio node and sample different seeds.

The native patch uses only the first item of a batched condition input.

An extremely wide or tall image is rejected

  1. Check the intended output dimensions.
  2. Reduce the input resolution, or crop/pad the image to a less extreme shape.
  3. Ensure the resulting output would not have a side above 4096 pixels.
  4. Update the control map to match the revised canvas.

A JSON file does not load as a visual workflow

  1. Check the filename.
  2. If it ends in .api.json, choose the corresponding UI workflow instead.
  3. Drag the UI workflow onto the canvas.

The Union checkpoint fails validation

  1. Check that you selected the documented converted Union checkpoint.
  2. Confirm that you have not selected the base diffusion model or another ControlNet by mistake.
  3. Check the download against the information in MODEL_SHA256.json if file integrity is in doubt.
  4. Verify that ComfyUI meets the documented version and native Qwen Image 2.1 support requirements.

Frequently asked questions

Do I need eight ControlNet checkpoints?
No. One Union checkpoint supports all eight documented control modes.

Can I connect an ordinary photograph and select Canny or Pose?
Not as a substitute for preprocessing. control_image expects a prepared map. Use the reference encoder when the photograph is an editing reference.

Can a second reference determine the output shape?
The first reference_image determines the canvas. Prepare your control map around that target.

Will a pose map copy someone’s face or clothing?
No exact transfer is guaranteed. Pose and other structural controls guide structure, not precise identity or outfit reproduction.

Are 40 sampling steps and 800 × 1312 required?
No. Those are the documented settings for the pictured Canny example, not a universal requirement.

Is this model suitable for commercial artwork?
The supplied model license permits non-commercial use only unless you obtain a separate license from Qwen.

Learn More about Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8

Extension-specific examples

General ComfyUI learning resources

The general ComfyUI resources are useful for learning the interface and troubleshooting workflows, but they are not dedicated documentation for this extension.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 License and Provenance

The extension code and model weights have different licensing terms:

MaterialLicense
Original adapter codeMIT
Bundled ComfyUI Union compatibility codeGPL-3.0, adapted from upstream PR #16519
Distributed extension packageGPL-3.0
Model weightsQwen Research License: non-commercial use only unless separately licensed by Qwen

For conversion details, see convert_union.py. File checksums are recorded in MODEL_SHA256.json.

The code license does not remove the model’s non-commercial restriction. Review the model license separately when deciding whether the model is appropriate for your intended work.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 Related Nodes

RunComfy
Copyright 2025 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals. RunComfy also provides AI Models, enabling artists to harness the latest AI tools to create incredible art.

Comfyui-Qwen-Image-2.1-Fun-Controlnet-Union-T8 detailed guide | ComfyUI