ComfyUI Extension: ComfyUI prompt control

ComfyUI prompt control Introduction

The comfyui-prompt-control extension is designed to enhance the functionality of ComfyUI by making various operations prompt-controllable. This includes the ability to control LoRA (Low-Rank Adaptation) and prompt scheduling directly through the prompt itself. This extension is particularly useful for AI artists who want to streamline their workflow and achieve more precise control over their outputs without needing to manually adjust multiple samplers or conditioning nodes.

Key Features:

• Prompt Editing and Filtering: Modify prompts without the need for multiple samplers.
• LoRA Loading and Scheduling: Load and schedule LoRA models, including block weights.
• Prompt Masking and Area Control: Combine prompts, control specific areas, and interpolate between prompts.
• SDXL Parameters: Set specific parameters for SDXL models.
• Miscellaneous Controls: Various other functionalities to enhance your workflow. By using comfyui-prompt-control, you can achieve identical outputs to traditional ComfyUI workflows but with greater ease and flexibility.

How ComfyUI prompt control Works

The extension works by interpreting special syntax within your prompts to control various aspects of the image generation process. Think of it as giving instructions to ComfyUI through a more intuitive and centralized method—your prompt.

Basic Principles:

1. Prompt Scheduling: Allows you to specify changes in the prompt at different stages of the generation process.
2. LoRA Integration: Enables dynamic loading and application of LoRA models based on the prompt.
3. Masking and Area Control: Lets you define specific areas of the image to apply different prompts or effects. For example, you can use a syntax like [cat|dog:0.05] to switch between "cat" and "dog" at a specific point in the generation process.

ComfyUI prompt control Features

Prompt Editing and Filtering

• Syntax: Use brackets to define changes in the prompt.
• Example: a [large::0.1] [cat|dog:0.05] will start with "a large" and switch to "cat" or "dog" at 0.05 steps.

LoRA Loading and Scheduling

• Syntax: <lora:filename:weight>.
• Example: <lora:cats:1> will load the "cats" LoRA model with a weight of 1. - Block Weights: Use LBW syntax for more granular control.

Prompt Masking and Area Control

• Syntax: MASK(x1 x2, y1 y2, weight, op).
• Example: MASK(0 0.5, 0.3) will apply a mask to the specified area.
• Feathering: Use FEATHER(left top right bottom) to smooth the edges of the mask.

SDXL Parameters

• Syntax: SDXL(width height, target_width target_height, crop_w crop_h).
• Example: SDXL(1024 1024, 1024 1024, 0 0) sets the parameters for SDXL models.

Miscellaneous Controls

• Combining Prompts: Use AND to combine multiple prompts.
• Functions: Use FUNCNAME(param1, param2, ...) for various operations like SHUFFLE and SHIFT.

ComfyUI prompt control Models

The extension supports different models, each with specific use cases:

LoRA Models

• Standard LoRA: Use for general image generation tasks.
• Block Weight LoRA: Provides more control over specific parts of the model.

SDXL Models

• Standard SDXL: Use for high-resolution image generation.
• Custom SDXL: Allows for more specific parameter settings.

What's New with ComfyUI prompt control

Recent Updates:

• 2024-02-02: Automatic enabling of LoRA backup weights to CPU if VRAM runs out.
• 2024-01-14: Multiple CLIP_L instances are now joined with a space separator.
• 2024-01-09: Dropped AITemplate support; recommended to use Stable-Fast instead.
• 2024-01-08: In-place weight updates for improved performance.
• 2023-12-28: Updated MASK to use ComfyUI's mask_strength attribute. These updates improve the extension's performance and compatibility, making it easier for AI artists to achieve their desired results.

Troubleshooting ComfyUI prompt control

Common Issues and Solutions:

1. LoRA Not Loading:

• Solution: Ensure the filename is correct and matches the expected format.

2. Prompt Not Switching:

• Solution: Check the syntax and ensure the step percentages are correctly specified.

3. Out of Memory Errors:

• Solution: Use the --highvram switch or enable CPU offloading.

Frequently Asked Questions:

• Q: How do I load a specific LoRA model?
• A: Use the syntax <lora:filename:weight>.
• Q: Can I combine multiple prompts?
• A: Yes, use the AND syntax to combine prompts.

Learn More about ComfyUI prompt control

For additional resources, tutorials, and community support, check out the following links:

• These resources provide in-depth tutorials and community support to help you get the most out of comfyui-prompt-control.