ComfyUI Nvidia Cosmos Text & Image to Video Workflow
What is the Nvidia Cosmos Workflow
Turn your imagination into fluid videos using the newly released Nvidia Cosmos models in ComfyUI. This workflow demonstrates the strong AI capabilities of Nvidia Cosmos with its text-to-video and image-to-video generation features. Powered by Nvidia Cosmos's state-of-the-art 7B and 14B models, you can create high-quality videos from either textual descriptions or still images. The Nvidia Cosmos engine gives stellar results thanks to its ultra-efficient video processing capabilities.
Key Features of Nvidia Cosmos
- Dual Generation Modes: Nvidia Cosmos offers both text-to-video and image-to-video generation
- Guaranteed Motion: Always generates videos with movement when using 121 frames
- Effective Negative Prompts: Non-distilled model ensures better control through negative prompts
- Flexible Image Control: Generate from the last frame or create transitions between images
- Ultra-Efficient VAE: Nvidia Cosmos employs a refined VAE system for smooth, high-quality video generation
- High Resolution Support: Create videos at resolutions of 704x704 and above
- Precise Frame Control: Optimized for 121-frame sequences
- Smart Image Interpolation: Generate smooth transitions between reference images
How to Use the Nvidia Cosmos Workflow
Nvidia Cosmos workflow contains two main parts: _text-to-video_ and _image-to-video_ generation. By default, the _image-to-video_ group is bypassed. To switch between the two modes:
- For _text-to-video_: Keep the _image-to-video_ group bypassed (default setting)
- For _image-to-video_: Right-click the _image-to-video_ group and select
Set Group Nodes to Always
1. Text to Video Generation with Nvidia Cosmos
Setup and Requirements
- Choose your preferred Nvidia Cosmos model size (7B recommended for starting)
- Set resolution (Default 1280x704; minimum 704x704)
- Frame settings:
- Length: 121 frames (The model performs optimally with a length of 121; deviating too much from this can result in subpar video quality.)
- Frame rate: 24.00 (default rate for optimal quality) <img src="https://cdn.runcomfy.net/workflow_assets/1184/readme02.webp" alt="Nvidia Cosmos" width="350"/> <img src="https://cdn.runcomfy.net/workflow_assets/1184/readme03.webp" alt="Nvidia Cosmos" width="350"/>
Sampling Parameters for Nvidia Cosmos
- Sampler:
res_multistep(Nvidia's recommended sampler for Cosmos) - Scheduler:
karras(default for stability) - Steps:
20(higher = better quality but slower; lower = faster but less detailed) - CFG:
6.5(prompt guidance strength) - Denoise:
1.00(1.00 = complete transformation; lower values keep more original content)
Prompting Tips for Nvidia Cosmos
- Use detailed, multi-sentence prompts for better results
- Include comprehensive negative prompts
- Short prompts may generate coherent videos but might not strictly follow instructions
2. Image to Video Generation with Nvidia Cosmos
Setup and Requirements
- Same base requirements as Nvidia Cosmos text-to-video
- Supports
start_imageandend_imageinputs
Reference Image Options
- Set a
start_imageorend_image, or both at the same time - Images work best when similar in style and content (for smooth transitions)
Key Parameters
- Identical sampling settings to text-to-video mode
- Maintains same video quality standards
Advanced Tips for Nvidia Cosmos
- For higher quality results with more VRAM, try the Nvidia Cosmos 14B model
- Ensure prompts are descriptive and detailed for best results
- Experiment with different image pairs for unique transitions
More Information about Nvidia Cosmos
For more details and updates about Nvidia Cosmos, visit Nvidia Cosmos Official Page.
