ComfyUI Node: Flux Controlnet Sampler with Injection (CRT)

Flux Controlnet Sampler with Injection (CRT):

The FluxControlnetSamplerWithInjection node is designed to enhance the capabilities of AI art generation by integrating advanced sampling techniques with noise injection. This node leverages the power of ControlNet, a neural network architecture that allows for precise control over the generation process, to produce high-quality and diverse outputs. By injecting noise at specific stages of the sampling process, it introduces variability and creativity into the generated images, making it particularly useful for artists seeking unique and varied results. The node is capable of handling both image and latent inputs, ensuring flexibility in its application. Its primary goal is to provide a robust framework for generating art with controlled randomness, allowing for both structured and experimental outputs.

Flux Controlnet Sampler with Injection (CRT) Input Parameters:

model

This parameter specifies the model to be used for the sampling process. It is crucial as it determines the underlying architecture and capabilities of the generation process.

positive

This parameter represents the positive conditioning input, which influences the direction and style of the generated output. It is essential for guiding the model towards desired artistic features.

vae

The VAE (Variational Autoencoder) parameter is used for encoding and decoding images, playing a critical role in transforming latent representations into visual outputs.

control_net

This parameter refers to the ControlNet model, which provides additional control over the generation process, allowing for more precise and targeted outputs.

seed

The seed parameter is an integer that initializes the random number generator, ensuring reproducibility of results. It has a default value of 0 and can range from 0 to 0xFFFFFFFFFFFFFFFF.

seed_shift

This integer parameter offsets the main seed to introduce variation in the outputs. It ranges from -100000 to 100000, with a default value of 0, allowing for subtle or significant changes in the generated art.

steps

This parameter defines the number of sampling steps, influencing the detail and quality of the output. It ranges from 1 to 10000, with a default of 20 steps.

sampler_name

This string parameter specifies the name of the sampler to be used, with a default value of "dpmpp_2m_sde". It determines the sampling algorithm applied during generation.

scheduler

The scheduler parameter, a string, defines the scheduling strategy for the sampling process, with "karras" as the default. It affects the timing and sequence of sampling steps.

upscale_by

This float parameter determines the scaling factor for upscaling the output, ranging from 1.0 to 2.0, with a default of 1.5. It enhances the resolution of the generated images.

controlnet_strength

This float parameter controls the influence of ControlNet on the generation process, ranging from 0.0 to 10.0, with a default of 0.5. It adjusts the balance between control and randomness.

control_end

This float parameter specifies the endpoint for ControlNet's influence, ranging from 0.0 to 1.0, with a default of 1.0. It defines the duration of control during sampling.

enable_noise_injection

This parameter is a toggle option ("disable" or "enable") that determines whether noise injection is applied during sampling. It is disabled by default and can add creative variability to the output.

injection_point

This parameter specifies the stage at which noise is injected, allowing for targeted introduction of randomness at specific points in the sampling process.

injection_seed_offset

This integer parameter offsets the seed used for noise injection, introducing additional variability. It allows for fine-tuning the randomness introduced by noise injection.

injection_strength

This float parameter controls the intensity of the noise injected, affecting the degree of randomness in the output. It allows for balancing between structured and experimental results.

normalize_injected_noise

This toggle option ("disable" or "enable") determines whether the injected noise is normalized, ensuring consistent impact on the output. It helps maintain control over the noise's effect.

image

This optional parameter provides an image input for the sampling process. If not provided, a latent input must be used, serving as the starting point for generation.

latent

This optional parameter provides a latent input for the sampling process. If not provided, an image input must be used, serving as the starting point for generation.

Flux Controlnet Sampler with Injection (CRT) Output Parameters:

final_latent_tuple

The output is a tuple containing the final latent representation after the sampling and noise injection process. This output is crucial as it represents the generated art in its latent form, ready for decoding into a visual image. It encapsulates the effects of all input parameters and the transformations applied during the sampling process.

Flux Controlnet Sampler with Injection (CRT) Usage Tips:

• To achieve more varied and creative outputs, consider enabling noise injection and experimenting with different injection strengths and points.
• Adjust the controlnet_strength parameter to balance between structured control and creative randomness, depending on the desired outcome.
• Use the seed_shift parameter to introduce subtle variations in the output without changing the main seed, allowing for exploration of different artistic possibilities.

Flux Controlnet Sampler with Injection (CRT) Common Errors and Solutions:

"FluxControlnetSampler requires either an 'image' or a 'latent' input."

• Explanation: This error occurs when neither an image nor a latent input is provided to the node, which is necessary for the sampling process to begin.
• Solution: Ensure that you provide either an image or a latent input to the node before execution. Check your input connections and parameters to confirm that one of these inputs is specified.