Nucleus-Image-comfyui-beta Introduction
Nucleus-Image-comfyui-beta adds custom nodes to ComfyUI for creating images with the Nucleus-Image 17B MoE diffusion model. It gives you a workflow for loading the model, turning written prompts into guidance, generating an image, and decoding the result into a viewable picture.
The extension is designed around Nucleus-Image’s own components. Its text encoder and image decoder differ from those used by common Stable Diffusion workflows, so the extension provides dedicated loaders rather than relying on ComfyUI’s standard CLIP and VAE loaders. It also includes optional controls for memory use and generation behavior.
This is a beta release. The author has tested image generation with the FP8 model, but has not run the bf16 model because of hardware limitations. Block Swap, CFG Rescale, and Model Shift have passed code review but have not been tested during generation. Treat their results as experimental. The author reports good results with the euler sampler and normal scheduler; other sampler and scheduler combinations have not been established as equally effective.
How Nucleus-Image-comfyui-beta Works
Think of the workflow as a small production line:
- Load the image model. The Transformer Loader supplies the part that gradually creates the image.
- Translate your prompt. The Text Encoder turns your words into information the image model can use. A positive prompt describes what you want; a negative prompt provides the sampler’s negative guidance.
- Generate the underlying image data. The Sampler starts from a seed and refines the result over a chosen number of steps. You can set the image size, guidance strength, sampler, and scheduler here.
- Make the image viewable. The VAE Decode node converts the sampler’s latent output, an internal image representation, into a regular image that you can connect to a ComfyUI image output node.
A basic prompted workflow is:
Transformer Loader ───────────────────────────────┐
Text Encoder Loader → Text Encode (Dual) ─────────┼→ Sampler → VAE Decode → image
VAE Loader ────────────────────────────────────────┘
Text Encode (Dual) produces separate positive and negative outputs for the Sampler. The optional Block Swap, Model Shift, and CFG Rescale nodes add controls around this basic flow.
The text encoder loads when a Text Encode node needs it and is released from GPU memory after encoding. The Transformer is offloaded to the CPU after sampling, and the VAE is offloaded after decoding. These behaviors help manage GPU memory, although they do not guarantee that every workflow will fit on every graphics card.
Nucleus-Image-comfyui-beta Features
Dedicated model loaders
- Transformer Loader selects a Nucleus-Image model from ComfyUI’s
diffusion_modelsdirectory. Itsprecisionsetting offersbf16,fp16, andfp32for computation; the documented tested configuration usesbf16. The loader detects and handles FP8 model weights automatically. Itsload_devicesetting chooses the initial location of non-expert parameters:offload_deviceormain_device. - Text Encoder Loader selects the Qwen3-VL text encoder from
text_encodersand offers the same computation-precision choices. You do not need to choose a loading device: the encoder is loaded when encoding begins and released afterward. - VAE Loader selects the matching image decoder from
vaeand offersbf16,fp16, andfp32computation precision.
The model file format and the precision setting are different choices. For example, the author’s tested configuration uses FP8 model files with bf16 selected for computation. The source README does not provide comparative image-quality results for the other combinations.
Prompt and conditioning nodes
- Text Encode turns one multiline prompt into conditioning for either the Sampler’s positive or negative input.
- Text Encode (Dual) accepts a positive and a negative prompt and produces both outputs while loading the text encoder only once. This is the author’s recommended choice for a typical prompted workflow. The negative prompt can be an empty string (
""). - Zero Conditioning creates a zero-valued conditioning input without loading the text encoder. The author describes it as an option for unconditional generation or as an alternative negative input. Its appearance may differ from an encoded empty negative prompt.
Generation controls
The Sampler combines the model and conditioning to generate a latent image. Its main settings are:
| Setting | Default | What it controls |
|---|---|---|
width and height | 1024 × 1024 | Output dimensions, adjustable in increments of 64 |
steps | 50 | How many denoising steps the sampler performs |
cfg | 4.0 | How strongly generation follows the conditioning |
seed | 0 | The starting random value; keeping it fixed helps you compare setting changes |
sampler_name | euler | The sampling method |
scheduler_name | normal | How sampling steps are scheduled |
The node offers ComfyUI’s built-in samplers and schedulers. The author specifically reports good results with euler and normal; that is a useful starting point, not a requirement.
The README suggests these image sizes for different compositions:
| Size | Shape | Possible use |
|---|---|---|
| 1024 × 1024 | Square | General-purpose images |
| 1344 × 768 | Landscape | Wide scenes |
| 768 × 1344 | Portrait | Tall compositions |
| 1280 × 720 | Widescreen | 16:9-style images |
Optional adjustments
- Block Swap sets
blocks_to_swapfrom0to29. At0, blocks stay on the GPU; increasing the value moves more MoE blocks to the CPU. This is intended to help when GPU memory is limited, but the author has not runtime-tested this feature. - Model Shift overrides the model’s sigma-shift settings with
base_shiftandmax_shift, which default to0.5and1.15. If you leave the node disconnected, the scheduler configuration supplies its defaults. The author has not runtime-tested this override. - CFG Rescale sets an adaptive guidance-normalization strength from
0.0to1.0;1.0is identified as the official default. The author has not runtime-tested this control, so change it cautiously and compare results with the same seed.
Finally, VAE Decode takes the Sampler’s latent output and the VAE Loader’s output and produces a viewable image.
Nucleus-Image-comfyui-beta Models
The extension works with three model components: a Transformer to generate the image, a Qwen3-VL text encoder to interpret prompts, and a VAE to turn latent output into an image. The source README lists these single-file options:
| Component | FP8 or standard file | Optional bf16 file |
|---|---|---|
| Transformer | nucleus_image_transformer_fp8.safetensors (about 16.9 GB) | nucleus_image_transformer_bf16.safetensors (about 31.7 GB) |
| Text encoder | nucleus_image_text_encoder_fp8.safetensors (about 8.2 GB) | nucleus_image_text_encoder_bf16.safetensors (about 16.5 GB) |
| VAE | nucleus_image_vae.safetensors (about 122 MB) | nucleus_image_vae_bf16.safetensors (about 243 MB) |
The author recommends FP8 because it uses fewer resources and is faster. The bf16 files are presented as optional higher-quality versions, but bf16 generation has not been run-tested by the author; the README does not provide a measured quality comparison. For a first workflow, use the FP8 files and the tested settings below:
| Node | Tested setting |
|---|---|
| Transformer Loader | FP8 transformer; precision: bf16; load_device: offload_device |
| Text Encoder Loader | FP8 text encoder; precision: bf16 |
| Text Encode (Dual) | Positive prompt and negative prompt |
| Sampler | steps: 50; cfg: 4.0; euler sampler; normal scheduler |
| VAE Loader | nucleus_image_vae.safetensors; precision: bf16 |
Use the matching Nucleus-Image VAE rather than a VAE from another image model. You can find the model’s official page at NucleusAI/Nucleus-Image on Hugging Face.
Troubleshooting Nucleus-Image-comfyui-beta
The image does not seem to follow the prompt
- Check that your positive Text Encode output is connected to the Sampler’s
positiveinput. - Connect a negative conditioning output to the Sampler’s
negativeinput as well. The author recommends Text Encode (Dual), even when its negative prompt is empty (""). - Start with the author’s tested
eulersampler,normalscheduler, 50 steps, andcfgof4.0. Keep the seed fixed while checking whether a changed connection or setting helps.
The image looks dark or gray
- Check the file selected in VAE Loader.
- Select the Nucleus-Image VAE, such as
nucleus_image_vae.safetensors, rather than a VAE from another model. - Run the workflow again and compare the decoded result.
Generation runs out of GPU memory
- If you are using bf16 model files, try the recommended FP8 files.
- Check whether Block Swap helps by increasing
blocks_to_swapfrom its default of0. The author has not runtime-tested Block Swap, so make small changes and watch for errors. - If needed, try a smaller image size while keeping both dimensions on the Sampler’s 64-pixel increments.
Frequently asked questions
Can I use ComfyUI’s usual CLIPLoader or VAELoader?
No. Nucleus-Image uses a Qwen3-VL text encoder rather than standard CLIP, and its VAE uses a 16-channel latent format. Use the extension’s dedicated loaders.
Why is there no device option on Text Encoder Loader?
The text encoder loads when a Text Encode node runs and is released from the GPU after encoding, so the loader does not require manual device selection.
Do I need to set FP8 quantization manually?
No. According to the source README, Transformer Loader detects FP8 model weights and handles them automatically.
Are all optional controls tested?
No. FP8 inference is the tested path. The author has not runtime-tested Block Swap, Model Shift, or CFG Rescale, and has not run-tested bf16 inference.
Learn More about Nucleus-Image-comfyui-beta
- Nucleus-Image model page: the model’s official page.
- Requirements reference: detailed software-dependency information for readers diagnosing a compatibility issue.
The author states that the node code may be modified, customized, and distributed without restrictions.
