ComfyUI Node: DiT Spectrum Patch

DiT Spectrum Patch:

The DiTSpectrumPatch node is a specialized tool designed to enhance the performance of diffusion models by applying a unique patching mechanism. This node focuses on optimizing the final layer of the model by forecasting the DiT (Diffusion Transformer) feature before the final layer is executed. By doing so, it allows for the re-execution of only the final layer or unpatchify operations on cached steps, significantly improving efficiency. This approach is particularly beneficial when integrated with a normal KSampler, KSampler Advanced, or Custom Sampler, as it bypasses the need for repeated sampling and does not incorporate additional guidance methods like mod guidance, DCW, SMC-CFG, or SPEED/SPD. The primary goal of the DiTSpectrumPatch is to streamline the model's workflow, reducing computational overhead while maintaining high-quality output.

DiT Spectrum Patch Input Parameters:

model

The model parameter represents the diffusion model that will be patched. It is the core component that the DiTSpectrumPatch will modify to enhance its performance. This parameter does not have a specific range or default value as it depends on the model being used in your workflow.

steps

The steps parameter defines the number of steps for which the patch will be applied. It influences how many iterations the model will undergo with the patch in place. The minimum value is 1, and the maximum is not explicitly defined, but a typical default is 30 steps.

window_size

The window_size parameter determines the size of the window used for forecasting the DiT feature. It affects the granularity of the forecast and can impact the accuracy and efficiency of the patch. The default value is 2.0, with a flexible adjustment range.

flex_window

The flex_window parameter allows for flexibility in the window size, providing a buffer to accommodate variations in the model's behavior. It is set to a default of 0.25, allowing for slight adjustments to the window size during execution.

warmup_steps

The warmup_steps parameter specifies the number of initial steps where the patch is not applied, allowing the model to stabilize before the patch takes effect. This is crucial for ensuring that the model's initial state is optimal for patching. The default value is 6 steps.

tail_actual_steps

The tail_actual_steps parameter indicates the number of steps at the end of the process where the patch is not applied, ensuring that the final output is accurate and not overly influenced by the patch. The default is set to 3 steps.

blend_w

The blend_w parameter controls the blending weight used during the patching process. It affects how much influence the patch has on the model's output. The default value is 0.3, providing a balanced blend between the patched and unpatched states.

cheby_degree

The cheby_degree parameter sets the degree of the Chebyshev polynomial used in the patching process. This influences the mathematical complexity and precision of the patch. The default degree is 3.

ridge_lambda

The ridge_lambda parameter is a regularization term used in the patching process to prevent overfitting. It helps maintain the stability of the model during execution. The default value is 0.1.

history_size

The history_size parameter defines the size of the buffer used to store past forecasts, which can be used to improve future predictions. The minimum value is 5, and the maximum is 10000, with a default size of 100.

enabled

The enabled parameter is a boolean that determines whether the patch is active. When set to True, the patch is applied; when False, the model runs without the patch. The default is True.

one_sampler_only

The one_sampler_only parameter is a boolean that restricts the patch to be applied only during the first sampler run within a workflow. This ensures that subsequent sampler runs, such as high-resolution fixes, are not affected by the patch. The default is False.

verbose

The verbose parameter is a boolean that, when enabled, logs detailed information about the patching process, including decisions on actual versus cached steps. This is useful for debugging and understanding the patch's impact. The default is False.

DiT Spectrum Patch Output Parameters:

MODEL

The output parameter MODEL represents the patched diffusion model. This output is crucial as it provides the enhanced model ready for further processing or sampling. The patched model is optimized for efficiency and performance, having undergone the DiT Spectrum patching process.

DiT Spectrum Patch Usage Tips:

• To maximize efficiency, use the DiTSpectrumPatch in workflows where repeated sampling is required, as it reduces computational overhead by caching certain steps.
• Adjust the window_size and flex_window parameters to fine-tune the balance between forecast accuracy and computational efficiency based on your specific model and task requirements.
• Enable the verbose option during initial setup to gain insights into the patching process and make informed adjustments to parameters.

DiT Spectrum Patch Common Errors and Solutions:

"DiT Spectrum Patch: cond_or_uncond=%s does not divide batch=%d; running actual forward without forecast update"

• Explanation: This error occurs when the condition or unconditional input does not evenly divide the batch size, preventing the forecast update from being applied.
• Solution: Ensure that the batch size is compatible with the condition or unconditional input, or adjust the batch size to be divisible by the input.

"DiT Spectrum Patch: one_sampler_only consumed after %d steps"

• Explanation: This message indicates that the one_sampler_only option has been consumed after the specified number of steps, meaning the patch will not be applied to subsequent sampler runs.
• Solution: If you need the patch to apply to more sampler runs, disable the one_sampler_only option or re-run the workflow to reset the patch state.