ComfyUI Extension: ComfyUI-FL-VoxCPM

ComfyUI-FL-VoxCPM Introduction

ComfyUI-FL-VoxCPM is an innovative extension designed to bring advanced text-to-speech (TTS) capabilities to the ComfyUI platform. Powered by OpenBMB's VoxCPM model family, this extension offers a range of features that allow you to create high-quality, multilingual speech synthesis. Whether you're looking to clone voices, design new ones from text descriptions, or fine-tune custom voices, ComfyUI-FL-VoxCPM provides the tools you need. This extension is particularly useful for AI artists who want to incorporate realistic and expressive speech into their projects without needing extensive technical knowledge.

How ComfyUI-FL-VoxCPM Works

At its core, ComfyUI-FL-VoxCPM utilizes a tokenizer-free, diffusion autoregressive architecture. This means it can generate continuous speech representations directly, bypassing the need for discrete tokenization. Imagine it as a painter who doesn't need to sketch first but can directly paint a complete picture. This approach allows for highly natural and expressive speech synthesis. The extension supports multiple languages and can create voices from simple text descriptions, making it accessible and easy to use for artists.

ComfyUI-FL-VoxCPM Features

• VoxCPM V2 Model: This model boasts 2 billion parameters and produces 48kHz studio-quality audio across 30 languages. It's ideal for creating diverse and high-fidelity speech outputs.
• Voice Design: Create unique voices using natural language descriptions. For example, you can describe a voice as "a young woman with a warm and gentle tone," and the model will generate speech that matches this description.
• Voice Cloning: Clone any voice using a short audio reference. This feature is perfect for replicating specific vocal characteristics.
• Controllable Cloning: Modify the style or emotion of a cloned voice, allowing for creative expression while maintaining the original voice's timbre.
• Ultimate Cloning: Achieve maximum fidelity by using both reference audio and continuation audio, ensuring every vocal nuance is captured.
• LoRA Training: Fine-tune custom voices with a real-time training dashboard that provides insights into the training process, including loss charts and validation audio.
• Auto Transcription: Integrated Whisper technology transcribes audio to text, aiding in creating accurate reference texts.
• Audio Crop: Trim audio files to specific time ranges, making it easy to edit and manage audio content.

ComfyUI-FL-VoxCPM Models

The extension includes several models, each suited for different needs:

• VoxCPM2: With 2 billion parameters, this model is recommended for its high-quality output and support for 30 languages. It's perfect for voice design and controllable cloning.
• VoxCPM1.5: A stable model with 800 million parameters, offering high-fidelity TTS at 44.1kHz. It's suitable for projects requiring consistent quality.
• VoxCPM-0.5B: A legacy model with 500 million parameters, providing a lightweight option for basic TTS needs.

What's New with ComfyUI-FL-VoxCPM

Recent updates have introduced the VoxCPM2 model, which supports 30 languages and offers advanced features like voice design and controllable cloning. These enhancements allow for more creative and flexible use of the extension, enabling AI artists to produce even more realistic and expressive speech outputs.

Troubleshooting ComfyUI-FL-VoxCPM

If you encounter issues while using ComfyUI-FL-VoxCPM, here are some common solutions:

• Model Not Downloading: Ensure you have a stable internet connection. The models are downloaded automatically from HuggingFace on first use.
• Audio Quality Issues: Check your input settings, such as the cfg_value and inference_timesteps, to ensure they are optimized for your desired output.
• Voice Cloning Errors: Make sure your reference audio is clear and of good quality. Use the FL VoxCPM Transcribe node to generate accurate transcripts if needed.

Learn More about ComfyUI-FL-VoxCPM

To further explore the capabilities of ComfyUI-FL-VoxCPM, you can visit the VoxCPM GitHub repository for more detailed documentation and resources. Additionally, the Hugging Face page provides access to model weights and further technical details. For community support and discussions, consider joining the Discord server where you can connect with other users and developers.