ComfyUI-VoxCPM2 Introduction
ComfyUI-VoxCPM2 adds speech generation to ComfyUI. You can turn written text into spoken audio, describe a new voice in everyday language, or use a short recording as a reference for a voice clone. It also provides options for refining a particular voice with a LoRA adapter.
The extension uses VoxCPM2, a 2-billion-parameter text-to-speech model. It supports 30 languages without requiring you to select a language tag and produces 48 kHz audio. For an AI art workflow, this means you can create dialogue, narration, or character voice experiments alongside your other ComfyUI work.
ComfyUI-VoxCPM2 workflow screenshot
ComfyUI-VoxCPM2 is based on the original ComfyUI-VoxCPM project. The author has brought VoxCPM2 speech generation, voice cloning, and LoRA workflows into ComfyUI nodes.
How ComfyUI-VoxCPM2 Works
Think of the extension as a voice performer with three ways to prepare for a reading:
- Give it a script: Enter text in the VoxCPM2 TTS node, and the model generates speech.
- Give it a character direction: Add a
voice_description, such as “a calm, low voice with a measured pace.” The model uses that description when speaking your text; you do not need a reference recording. - Give it a voice reference: In VoxCPM2 Voice Clone, connect an audio clip. The model uses the clip as a guide for the new speech. Supplying an accurate transcript of that clip gives it more information to work with.
The speech is generated through a diffusion-based process. In practical terms, inference_timesteps controls how much work the model spends refining an output: fewer steps are useful for quick drafts, while more steps can improve quality but take longer. A seed lets you repeat a generation with the same settings; -1 chooses a random seed.
You can start with the default settings. Once you have a voice you like, adjust one setting at a time so you can hear what changed.
ComfyUI-VoxCPM2 Features
Text-to-speech and voice design
Use VoxCPM2 TTS when you have text but no reference audio. Enter the words in text, optionally add a voice_description, and queue the workflow. Descriptions can cover qualities such as age, tone, emotion, and pace. For example, “an older speaker with a gentle, unhurried voice” gives the model a different direction from “an energetic young presenter.”
Voice design can vary between runs. If the first result does not fit your character, try a more specific description or generate a few alternatives.
Two ways to clone a voice
Use VoxCPM2 Voice Clone with a connected Load Audio node and your new script in text.
- Controllable cloning: Leave
prompt_textempty. You can add avoice_descriptionsuch as “slightly faster, cheerful tone” to guide the delivery. - Ultimate cloning: Enter the exact words spoken in the reference clip in
prompt_text. The model uses both the recording and its transcript for higher-fidelity reproduction. If you do not have a transcript,enable_asrcan transcribe the clip and enter this mode automatically.
For best results, choose 5–15 seconds of clean, continuous speech. Reference clips longer than 50 seconds are rejected. In Ultimate cloning, an accurate transcript, including useful punctuation, helps the model follow the reference.
Only clone voices when you have the right to use them. Do not use generated speech for impersonation, fraud, or disinformation, and clearly label AI-generated content.
Audio cleanup and transcription
If a reference recording has background noise, enable_denoiser can clean it before cloning. This option uses ZipEnhancer and requires the optional modelscope dependency. When denoising is active, the extension also normalizes output loudness to -20 LUFS for a more consistent level.
enable_asr uses SenseVoiceSmall to transcribe reference audio and requires the optional funasr dependency. Its first use downloads an approximately 400 MB model. If you have already entered prompt_text, the manual transcript takes priority and automatic transcription is skipped.
Controls you can adjust
Both speech-generation nodes offer these controls:
| Setting | What it changes | A practical starting point |
|---|---|---|
cfg_value | How strongly the result follows the prompt or voice direction | Start at 2.0. Raising it can increase adherence; lowering it can allow more natural variation. |
inference_timesteps | Time spent refining the audio | Start at 10. Try 5–10 for drafts or 15–25 when prioritizing quality. |
max_tokens | The maximum generation length | Leave the default 4096 unless you need to limit or extend a generation. |
normalize_text | Processes ordinary text such as numbers, abbreviations, and punctuation | Keep it on for typical scripts. Turn it off for phoneme input such as {HH AH0 L OW1}. |
seed | Controls repeatability | Use a fixed number to revisit a result, or -1 for a random seed. |
retry_max_attempts and retry_threshold | Control automatic retries when a generation is detected as bad | Begin with the defaults unless you are troubleshooting inconsistent results. |
dtype, device, and force_offload | Control how the model uses your hardware and memory | Leave the automatic or default choices in place unless your system requires a change. |
torch_compile | Enables an optional inference optimization | Use it when it works well in your environment; it is not required to create speech. |
The extension manages model loading and unloading through ComfyUI. A selected LoRA checkpoint can also be applied in either generation node to use a voice style you have fine-tuned.
Supported languages
VoxCPM2 supports Arabic, Burmese, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, and Vietnamese. No language tag is needed for text in a supported language.
The model also lists these Chinese dialects: Sichuan, Cantonese, Wu, Northeastern, Henan, Shaanxi, Shandong, Tianjin, and Southern Min. Results may differ across languages and dialects, so listen to each output before using it in a finished piece.
ComfyUI-VoxCPM2 Models
| Model | When to use it | What to expect |
|---|---|---|
| VoxCPM2 | Text-to-speech, voice design, and reference-based voice cloning | The extension’s listed 2B-parameter model; supports 30 languages and produces 48 kHz audio. |
The model is downloaded automatically on first use. The following are optional supporting models or files, not alternative VoxCPM2 speech models:
- SenseVoiceSmall transcribes a reference recording when
enable_asris on. - ZipEnhancer cleans a reference recording when
enable_denoiseris on. - A LoRA adapter is a fine-tuned voice-style checkpoint that you select with
lora_name. You can use an existing adapter or create one with the extension’s training nodes.
Troubleshooting ComfyUI-VoxCPM2
My reference audio is rejected. What should I do?
- Check its duration. The extension rejects clips longer than 50 seconds.
- Choose a shorter section containing continuous speech; 5–15 seconds is the recommended range.
- Prefer a clear recording without music, overlapping speakers, or heavy background noise.
The clone does not sound close enough to the reference. How can I improve it?
- Start with a clean, high-quality reference clip.
- For Ultimate cloning, enter the recording’s exact spoken words in
prompt_text. - Check punctuation in the transcript, then listen to a new generation.
- If you are using a style description, try a simpler one so it does not pull the delivery away from the reference.
Automatic transcription is not working. What should I check?
- Make sure
enable_asris on and the optionalfunasrdependency is available. - Check whether
prompt_textalready contains text. If it does, the extension uses that text and skips transcription. - Allow time for the first-use SenseVoiceSmall download, which is approximately 400 MB.
The reference has noise. Can I clean it up?
Enable enable_denoiser in VoxCPM2 Voice Clone and make sure its optional modelscope dependency is available. A cleaner source clip is still preferable when you can record or select one.
Generation sounds unstable or takes too long. What can I try?
- Try a shorter or less expressive passage, especially if your input is very long.
- Return
cfg_valueandinference_timestepsto their defaults (2.0 and 10) to establish a baseline. - For faster drafts, try 5–10 inference steps. Increase the steps when you are ready to spend more time refining an output.
- Generate another take if voice design or style control varies; the author recommends trying 1–3 generations.
Do I need to specify the language?
No language tag is needed. Enter your script in one of the supported languages. Quality can vary by language, so review the spoken result for pronunciation and delivery.
Can I repeat an output?
Set seed to a specific number and keep your other settings the same. A value of -1 selects a random seed instead.
Learn More about ComfyUI-VoxCPM2
- VoxCPM project: Background on the speech model behind the extension.
- VoxCPM2 model page: Information about the model used for speech generation.
- LoRA Training Guide: The author’s guide to creating a custom voice adapter with the VoxCPM2 Train Config, VoxCPM2 Dataset Maker, and VoxCPM2 LoRA Trainer nodes.
- Original ComfyUI-VoxCPM project: The project on which this extension is based.
- Extension issues: Browse existing reports and discussions when you encounter a problem or have a feature idea.
The VoxCPM model and its components are provided by OpenBMB under the Apache-2.0 License.
