ComfyUI Node: DP Prompt Token Compressor

DP Prompt Token Compressor:

The DP Prompt Token Compressor is a specialized node designed to intelligently compress text prompts to fit within specified token limits while preserving the most important semantic information. This node is particularly useful for AI artists who work with models that have strict token constraints, such as those used in text-to-image generation. By analyzing and scoring different parts of a prompt, the node ensures that the most critical elements are retained, allowing for effective communication of ideas within the token limits. The compressor is adaptable to different model types, each with its own token limit, and offers a balanced approach to maintaining the integrity of the original prompt. This functionality is crucial for optimizing the input to AI models, ensuring that the generated outputs are as close as possible to the user's creative vision.

DP Prompt Token Compressor Input Parameters:

target_tokens

The target_tokens parameter specifies the maximum number of tokens that the compressed prompt should contain. This parameter is crucial as it directly influences how much of the original prompt can be retained after compression. The node will attempt to fit the most important phrases within this limit, adjusting if necessary to comply with the model's token constraints. The minimum value is 1, and the maximum value is determined by the model's token limit, which is typically 77 for most models except for the Flux model, which allows up to 225 tokens.

model_type

The model_type parameter determines which model's tokenization rules and limits are applied during the compression process. Different models have different token limits, and this parameter ensures that the compression respects those limits. Available options include "sd1", "sd2", "sdxl", and "flux", with "sd2" being the default. Selecting the correct model type is essential for accurate token counting and effective prompt compression.

preservation_mode

The preservation_mode parameter dictates the strategy used to prioritize which parts of the prompt are retained during compression. The default mode is "balanced", which aims to maintain a good mix of semantic content and brevity. This parameter allows users to influence the compression process based on their specific needs, such as emphasizing certain keywords or phrases over others.

prompt

The prompt parameter is the text input that you wish to compress. It is the original prompt that contains the ideas or instructions you want to convey to the AI model. If no prompt is provided, the node will return default values indicating that no input was given. This parameter is essential as it forms the basis of the compression process.

DP Prompt Token Compressor Output Parameters:

compressed_prompt

The compressed_prompt is the result of the compression process, containing the most important parts of the original prompt that fit within the specified token limit. This output is crucial for ensuring that the AI model receives a concise yet meaningful input that aligns with the user's creative intent.

final_tokens

The final_tokens output indicates the number of tokens in the compressed prompt. This value is important for verifying that the compression process has successfully adhered to the specified token limit, ensuring compatibility with the chosen model's constraints.

info

The info output provides detailed information about the compression process, including the model type, token limits, original and compressed token counts, and the compression ratio. It may also list any phrases that were excluded during compression. This information is valuable for understanding how the prompt was altered and for making any necessary adjustments to achieve the desired outcome.

DP Prompt Token Compressor Usage Tips:

• Ensure that the target_tokens parameter is set according to the model's token limit to avoid unnecessary adjustments during compression.
• Use the preservation_mode parameter to tailor the compression strategy to your specific needs, such as prioritizing certain keywords or maintaining a balanced approach.
• Regularly check the info output to understand how the prompt was compressed and to identify any important phrases that may have been excluded.

DP Prompt Token Compressor Common Errors and Solutions:

Warning: Target tokens (<target_tokens>) exceeds <model_type> limit of <model_limit>. Adjusting...

• Explanation: This warning indicates that the specified target_tokens value exceeds the token limit for the selected model type.
• Solution: Adjust the target_tokens parameter to be within the model's token limit, which is typically 77 for most models except for the Flux model, which allows up to 225 tokens.

No input prompt provided

• Explanation: This message appears when no prompt is given to the node for compression.
• Solution: Ensure that a valid text prompt is provided as input to the node to enable the compression process.