ComfyUI Node: MiniMax H3 Apply Reference Plan (Plan v2)

MiniMax H3 Apply Reference Plan (Plan v2):

The MiniMaxH3PlanV2ApplyReferencePlan node is designed to seamlessly integrate and execute a compiled Plan v2 package within ComfyUI's native H3 node framework. This node is pivotal for ensuring that the specified prompt and plan context are accurately matched and processed, thereby facilitating the routing of stored references in their native order. It leverages ComfyUI's official MiniMax H3 implementation to delegate conditioning tasks effectively. A notable feature of this node is its ability to handle Foley targets by constructing a locked-video and generated-audio latent, which is essential for per-token masking. This capability makes it an invaluable tool for AI artists looking to create complex audio-visual compositions with precision and ease.

MiniMax H3 Apply Reference Plan (Plan v2) Input Parameters:

clip

The clip parameter represents the video clip that will be processed by the node. It is crucial for defining the visual content that will be conditioned and transformed according to the plan. This parameter does not have specific minimum, maximum, or default values as it depends on the user's input.

vae

The vae parameter refers to the Variational Autoencoder model used for encoding and decoding the visual content. It plays a significant role in transforming the input video into a latent space representation, which is essential for further processing. Like clip, this parameter is user-defined without specific constraints.

h3_prompt

The h3_prompt is a string that serves as the guiding prompt for the plan execution. It is essential for aligning the plan's objectives with the user's creative intent. This parameter must be carefully crafted to ensure that the resulting output aligns with the desired artistic vision.

plan_context

The plan_context parameter provides the contextual information necessary for executing the plan. It ensures that the plan is executed in the correct environment and with the appropriate settings. This parameter is critical for maintaining consistency and accuracy in the plan's execution.

width

The width parameter specifies the width of the output video or image. It is important for defining the resolution and aspect ratio of the final output. Users should set this parameter according to their specific requirements for the project.

height

Similar to width, the height parameter defines the height of the output video or image. It works in conjunction with the width to determine the overall resolution and aspect ratio. Users should adjust this parameter based on their desired output dimensions.

ref_image_size

The ref_image_size parameter indicates the size of the reference image used in the plan. It is crucial for ensuring that the reference image is appropriately scaled and integrated into the final output. Users should specify this parameter based on the size of their reference image.

audio_vae

The audio_vae parameter is an optional input that represents the Variational Autoencoder model used for encoding and decoding audio content. It is necessary when the plan involves reference audio, ensuring that the audio is processed correctly alongside the visual content. If the plan requires audio processing and this parameter is not provided, an error will be raised.

MiniMax H3 Apply Reference Plan (Plan v2) Output Parameters:

positive

The positive output provides the native MiniMax H3 positive conditioning, which is essential for guiding the Basic Guider or another sampler path. This output is crucial for ensuring that the generated content aligns with the intended artistic direction.

latent

The latent output represents the native joint audio/video latent. For Foley plans, this output includes a VAE-encoded source video with a masked empty audio stream. This output is vital for creating synchronized audio-visual content with precise control over each component.

applied_prompt

The applied_prompt output is the exact prompt that was verified and sent with the plan context. It serves as a confirmation that the intended prompt was used in the plan execution, providing transparency and traceability in the creative process.

h3_length

The h3_length output indicates the native 17k+5 target frame count at 24 FPS. This output is important for understanding the duration and frame count of the generated content, allowing users to plan and adjust their projects accordingly.

adapter_report

The adapter_report output provides detailed information about the mode, checkpoint family, native implementation, target size, and applied routes. This output is essential for understanding the technical details of the plan execution and ensuring that the process aligns with the user's expectations.

MiniMax H3 Apply Reference Plan (Plan v2) Usage Tips:

• Ensure that the h3_prompt is well-defined and aligns with your creative goals to achieve the desired output.
• When working with Foley targets, provide a suitable audio_vae to ensure accurate audio processing and synchronization with the video content.
• Adjust the width and height parameters to match the resolution requirements of your project for optimal output quality.

MiniMax H3 Apply Reference Plan (Plan v2) Common Errors and Solutions:

"This plan contains reference audio. Connect the MiniMax H3 audio VAE to Apply Reference Plan.audio_vae."

• Explanation: This error occurs when the plan requires audio processing, but the audio_vae parameter is not provided.
• Solution: Ensure that you connect a suitable audio VAE model to the audio_vae parameter when the plan involves reference audio. This will enable the node to process the audio content correctly.