ComfyUI Node: H3 Audio Refine Sampler

H3 Audio Refine Sampler:

The H3AudioRefineSampler is a specialized node designed to enhance the audio component of a MiniMax H3 audio-visual (AV) latent while preserving the integrity of the video stream. This node is particularly useful for refining audio quality in scenarios where the video has already been fully processed and should remain unchanged. By freezing the video stream and applying additional denoising steps exclusively to the audio stream, the node ensures that the video remains bit-identical to its original form. This process leverages a noise mask that protects the video stream, allowing the model to focus on audio refinement in the context of the completed video. The node is efficient in terms of step arithmetic, reducing the total number of steps required for refinement without compromising on the quality of the audio output.

H3 Audio Refine Sampler Input Parameters:

model

This parameter specifies the MiniMax H3 model to be used for the refinement process. It should be the same model, including any LoRA or patch stack, that was used in the initial sampling pass. This ensures consistency in the refinement process.

positive

This parameter represents the positive conditioning input, which influences the refinement process by providing context or guidance for the desired audio output.

negative

This parameter represents the negative conditioning input, which helps in steering the refinement process away from undesired audio characteristics.

latent

The latent parameter is a sampled MiniMax H3 AV latent, containing both video and audio components from the initial pass. It serves as the input for the refinement process, where the audio will be enhanced while the video remains unchanged.

seed

The seed parameter is an integer value used to initialize the random number generator for the refinement process. It ensures reproducibility of results. The default value is 0, with a range from 0 to 0xffffffffffffffff.

steps

This parameter determines the number of refinement steps to be performed on the audio stream. The default is 6 steps, with a range from 1 to 100. These steps are executed at a denoise depth specified by the audio_denoise parameter.

cfg

The cfg parameter is a floating-point value that should match the value used in the first pass, typically 1.0 for Turbo LoRA passes. It influences the strength of the conditioning applied during refinement.

sampler_name

This parameter specifies the name of the sampler to be used, with options available from the KSampler.SAMPLERS. The default option is "euler".

scheduler

The scheduler parameter determines the scheduling strategy for the refinement process, with options available from the KSampler.SCHEDULERS. The default is "simple".

audio_denoise

This floating-point parameter controls how much the audio is re-noised on the shared schedule before refinement. It ranges from 0.01 to 1.0, with a default of 0.5. Values between 0.3 and 0.6 are typical for retaining pass-1 audio content, while 1.0 regenerates audio from scratch.

video_denoise

An optional parameter that defaults to 0.0, indicating that the video stream is completely frozen. Values greater than 0.0 allow partial reworking of the video during the refinement pass.

H3 Audio Refine Sampler Output Parameters:

LATENT

The output is a refined latent tensor that contains the enhanced audio stream while maintaining the video stream in its original, unaltered state. This ensures that the video remains bit-identical to the input, providing a seamless integration of refined audio with the existing video content.

H3 Audio Refine Sampler Usage Tips:

• Ensure that the model parameter matches the model used in the initial sampling pass to maintain consistency in the refinement process.
• Adjust the audio_denoise parameter based on the desired level of audio refinement. Lower values retain more of the original audio content, while higher values regenerate audio more extensively.
• Use the steps parameter to control the granularity of the audio refinement. More steps can lead to finer audio details but may increase processing time.
• Keep the video_denoise parameter at 0.0 to ensure the video remains unchanged unless partial video refinement is desired.

H3 Audio Refine Sampler Common Errors and Solutions:

H3-AudioRefine: latent is not a packed AV latent (expected a nested video+audio latent from a MiniMax H3 sampling pass, got a plain tensor).

• Explanation: This error occurs when the input latent is not a properly packed audio-visual latent tensor.
• Solution: Ensure that the input latent is derived from a MiniMax H3 sampling pass and contains both video and audio components.

H3-AudioRefine: nested latent has %d stream(s), expected 2 (video, audio).

• Explanation: The input latent does not contain the expected two streams (video and audio).
• Solution: Verify that the input latent is correctly structured with both video and audio streams.

H3-AudioRefine: unexpected stream shapes video=%s audio=%s (expected video [B,C,T,H,W] and audio [B,C,2,T]).

• Explanation: The shapes of the video and audio streams in the input latent do not match the expected dimensions.
• Solution: Check the dimensions of the video and audio streams in the input latent to ensure they conform to the expected shapes.