ComfyUI Node: WanDancerVideo

WanDancerVideo:

The WanDancerVideo node is designed to generate dynamic video content by leveraging advanced conditioning techniques. It allows you to create videos with a specified number of frames, using both visual and audio inputs to influence the final output. This node is particularly beneficial for artists looking to produce videos that incorporate specific visual styles or audio cues, offering a high degree of customization and control over the video generation process. By integrating various inputs such as initial images, masks, and audio features, WanDancerVideo provides a versatile platform for creating engaging and visually appealing video sequences.

WanDancerVideo Input Parameters:

positive

This parameter accepts conditioning inputs that positively influence the video generation process. It is used to guide the model towards desired visual outcomes based on the provided conditions.

negative

This parameter accepts conditioning inputs that negatively influence the video generation process. It helps the model avoid certain visual outcomes, ensuring the generated video aligns with the intended artistic vision.

vae

The VAE (Variational Autoencoder) input is crucial for encoding and decoding image data, allowing the model to process and generate video frames effectively. It plays a significant role in maintaining the quality and consistency of the video output.

width

This parameter sets the width of the generated video frames. It accepts integer values with a default of 480, a minimum of 16, and a maximum defined by the system's resolution capabilities. Adjusting the width impacts the aspect ratio and resolution of the video.

height

This parameter sets the height of the generated video frames. It accepts integer values with a default of 832, a minimum of 16, and a maximum defined by the system's resolution capabilities. Adjusting the height impacts the aspect ratio and resolution of the video.

length

This parameter determines the number of frames in the generated video, with a default value of 149. It accepts integer values with a minimum of 1 and a maximum defined by the system's resolution capabilities. The length of the video directly affects its duration and the complexity of the animation.

clip_vision_output

An optional input that provides CLIP vision embeddings for the first frame. These embeddings help in conditioning the video generation process based on specific visual features extracted from the initial frame.

clip_vision_output_ref

An optional input that provides CLIP vision embeddings for a reference image. These embeddings are used to guide the video generation process by referencing specific visual characteristics from the reference image.

start_image

An optional input that allows you to provide initial image(s) to be encoded. These images serve as the starting point for the video generation, influencing the initial frames and overall style of the video.

mask

An optional image conditioning mask for the start image(s). The mask determines which parts of the start image are retained (white) and which parts are generated (black), allowing for localized control over the video content.

audio_encoder_output

An optional input that provides audio features to influence the video generation process. It includes audio embeddings, frames per second (fps), and an audio inject scale, which collectively help synchronize the video with audio cues.

WanDancerVideo Output Parameters:

positive

The output includes the positively conditioned video data, reflecting the influence of the positive conditioning inputs on the final video.

negative

The output includes the negatively conditioned video data, reflecting the influence of the negative conditioning inputs on the final video.

samples

This output contains the latent samples generated by the model, representing the encoded video data that can be further processed or decoded into video frames.

WanDancerVideo Usage Tips:

• Ensure that the length parameter is set to 149 for optimal performance with the WanDancer model, as this is the recommended frame count for generating coherent video sequences.
• Utilize the mask input to selectively generate or retain specific parts of the start image, allowing for creative control over the video content and style.
• Experiment with different clip_vision_output and clip_vision_output_ref embeddings to achieve diverse visual styles and effects in the generated video.

WanDancerVideo Common Errors and Solutions:

"Invalid frame length"

• Explanation: The length parameter is set to a value other than 149, which is not supported by the WanDancer model.
• Solution: Adjust the length parameter to 149 to ensure compatibility with the model.

"Resolution exceeds maximum limit"

• Explanation: The width or height parameter exceeds the system's maximum resolution capabilities.
• Solution: Reduce the width and height values to fall within the supported resolution range.

"Missing VAE input"

• Explanation: The vae input is not provided, which is necessary for encoding and decoding video frames.
• Solution: Ensure that a valid VAE input is connected to the node to enable proper video generation.