ComfyUI_YuE2 Introduction
ComfyUI_YuE2 brings the official YuE2 music-generation pipeline into ComfyUI. You can describe a song’s style, provide lyrics, generate music, and save the result as audio. You can also work with an editable musical score or use a source recording to guide a new cover-style performance.
The extension gives you two ways to work:
- A simple, four-node workflow for going from a description and lyrics to a finished audio file.
- A staged workflow that exposes the score, intermediate generation steps, and decoder when you want more creative control or a clearer view of what happened.
ComfyUI provides the visual workflow, audio connections, and progress display. The official YuE2 Python engine performs the music generation. The extension uses the official model weights rather than replacing the engine with a separate ComfyUI implementation.
Important distinction: An audio-to-cover workflow creates a new performance guided by a score extracted from the source. It does not clone the source singer’s voice or replace the vocals in the original recording while keeping its waveform intact.
How ComfyUI_YuE2 Works
Think of YuE2’s process as planning a song, rehearsing its musical content, and then recording it:
- Describe the song. In Official Text Encode, you provide a
styledescription andlyrics. The style can describe genre, instruments, tempo, language, and vocal character; the lyrics hold the words to be sung and any section tags. - Plan the music. YuE2 can create an ABC score—a text-based way to describe musical notes and structure. You can let the model create this plan or supply a score you have edited.
- Develop the performance. The model generates intermediate musical information, called semantic tokens, from the request and plan.
- Create and decode the sound. The synthesis stage produces acoustic information, and a VAE decoder turns it into listenable, stereo audio.
Official Generate Music runs these stages for you. If you use the separate stage nodes instead, you can inspect the ABC score, keep a generated plan unchanged between stages, or decode the same acoustic result with different VAEs.
For a cover, SheetSage2 Transcribe first converts a source recording into a structured score. You then use that score to guide YuE2 toward a new performance with your chosen style and lyrics. The transcription is a musical guide, not a copy of the original audio.
ComfyUI_YuE2 Features
The nodes appear in ComfyUI under audio/YuE2 Official. If you are new to the extension, load 01-text-to-music.json and start with its four connected nodes: Model Loader → Text Encode → Generate Music → Save Audio.
| Node | What you can do with it |
|---|---|
| Official Model Loader | Select the YuE2 model, decoder, device, runtime backend, and memory-related settings. The default quality path uses unquantized BF16 generation and FP32 VAE decoding with the listening VAE. FP8 is an optional choice, not an automatic response to running out of memory. |
| Official Text Encode | Enter style and lyrics, choose a planning mode and seed, and optionally provide an ABC score. Automatic CFG uses the official default for your planning mode; manual CFG lets you set it yourself. |
| Official Generation Options | Adjust sampling separately for the ABC-planning and semantic-generation stages. It also exposes acoustic synthesis steps and full or chunked VAE decoding. The defaults follow the official generation configuration, including 32 midpoint ODE steps. |
| Official Generate Music | Run the complete pipeline and receive stereo audio, readable ABC, and a JSON report. |
| Official Generate ABC | Generate a score and a saved musical plan. Its plan includes the exact token IDs needed by later stages. |
| Official Semantic Tokens | Continue from a saved plan without rebuilding it. |
| Official Synthesize | Turn semantic tokens into acoustic information using the official solver. |
| Official Decode Audio | Turn saved acoustic information into audio with your selected VAE. You can use it twice on the same saved result to compare decoders fairly. |
| ABC Score Input | Enter or edit an ABC composition before generating its performance. |
| Official SheetSage2 Transcribe | Extract a score from source audio for a cover workflow. Choose melody or full transcription, precision, presets, input-length limits, and other official transcription controls. |
Choosing a planning mode
fullplans melody and chords. It is the default for creating a new song.melodyplans the melody while leaving accompaniment more flexible. It is recommended for covers.offgenerates directly from style and lyrics, without an ABC planning stage.
A supplied ABC score requires full or melody. YuE2 expects its native two-voice Vocal/Ins notation; see the official ABC editing guide before making substantial score edits.
Adjusting length and generation
Official Generation Options includes temperature, top_p, top_k, repetition_penalty, penalty_window, min_tokens, and max_tokens for both the ABC and semantic stages. These are advanced ways to influence how the model chooses and repeats material; keeping the official defaults is a sensible starting point.
Semantic audio proceeds at 25 tokens per second. semantic_max_tokens sets a maximum, not a promised song length: the model may finish sooner. For example, 9,000 tokens allow approximately 360 seconds. If an output seems to end unexpectedly, check its report as well as this setting.
Working with audio and scores
The generated audio is 48 kHz stereo. Connect the AUDIO output to ComfyUI’s Save Audio (Advanced) node; the examples use FLAC. The extension does not add normalization, effects, or mastering, so you can make those finishing choices separately.
The staged workflow saves intermediate materials—including the score, plan, semantic tokens, and acoustic information—under ComfyUI/temp/yue2/. Generation reports include timings and whether a token limit truncated the result. Audio may still be playable when the report says truncated=true, but it should not be treated as a completed song without listening and checking the report.
For transcription, max_seconds=0 processes the whole source recording; a positive number limits how much of it is processed. Automatic window settings let the selected official preset choose its window sizes. Optional score-rendering outputs require separate SheetSage2 rendering assets; the inference model files alone do not provide them.
ComfyUI_YuE2 Models
The complete set of five model repositories and supporting files totals 11.08 GB, before runtime dependencies.
| Model or file | Purpose | When you need it |
|---|---|---|
| YuE2-3B | The shared model behind score planning, semantic generation, and acoustic generation. | For YuE2 music generation. |
qwen.tiktoken | The official text-tokenizer file used with YuE2-3B. | Required supporting file for the YuE2 model. |
| YuE2-Vae | The current listening decoder, which turns generated acoustic information into audio. | The default choice for listening to generated music. |
| YuE2-Vae-legacy | An alternative decoder used to reproduce the published evaluation protocol. | When you specifically want that comparison. Decode the same saved acoustic result with both VAEs to hear the decoder difference. |
| SheetSage2 | Converts source music into a structured ABC score. | For the audio-to-cover workflow. |
| MERT-v2-FullSong | The audio model required by SheetSage2. | Alongside SheetSage2 for transcription. |
The extension retains the verified official weights and configurations. It does not quantize or alter them as part of its model-folder adaptation. Models are used locally for inference; downloads occur when the downloader is explicitly run.
Troubleshooting ComfyUI_YuE2
A cover workflow will not run with its example audio selection. What should you do?
- Open 04-audio-to-cover.json.
- Upload and select a real source song in Load Audio.
select-source-audio.wavis a placeholder, not an included recording. - Enter your target style and lyrics in Official Text Encode.
- Keep transcription at
mode=melodyand text planning atcot=melodyif you want a cover guided by the melody with adaptable accompaniment.
Your generated song plays, but seems cut short. Is the audio file broken?
- Open the JSON report from Official Generate Music and look for
truncated=true. - Check the semantic token limit. It is a ceiling, and reaching it can leave a song incomplete.
- Remember that increasing the ceiling only permits a longer result; it does not force the model to use the full duration.
A score you supplied is not accepted. What should you check?
- Set the planning mode to
fullormelody; supplied ABC is not supported withoff. - Check that the score follows YuE2’s native two-voice
Vocal/Insformat in the ABC guide. - If you changed a generated composition, submit the edited ABC as a new request. An unchanged saved plan should be reused directly rather than edited between stages.
How can you compare the two VAEs without comparing two different songs?
Save the acoustic result from Official Synthesize, then connect that same result to Official Decode Audio with each VAE selection. Generating twice would also change the performance you are comparing.
Which device or backend setting should you choose?
- On Windows, use
backend=torch-eager. The official Windows Torch distribution lacks the FlashAttention kernel needed by the CUDA Graph backend. - On Apple Silicon, select
device=autoormpswithbackend=torch-eager. SheetSage2 transcription should usecpu; MPS BF16 requires macOS 14 or later. - On a CPU-only Linux setup, select
device=cpuafter preparing the CPU runtime. - On Linux with NVIDIA, select an available CUDA device. The official recommended starting point is a BF16-capable GPU with 24 GB of VRAM.
If memory is tight, the Model Loader exposes memory-related controls, including AR offload and VAE chunk size. The adapter also releases cached ComfyUI models on the selected device before launching the separate engine. FP8 must be chosen explicitly; it is not silently enabled after a memory failure.
Why does a staged workflow behave differently after an update?
Restart ComfyUI and rerun the earlier stages. Previously cached stage outputs may use an older artifact format.
Will transcription preserve the original singer or backing track?
No. SheetSage2 supplies a score that guides a newly generated performance. It does not extract a singer’s identity, clone a voice, or retain the source recording’s backing waveform.
Are results guaranteed to match a score or lyric exactly?
No. The example workflows and downloaded files have been checked for structure and integrity, but those checks cannot guarantee how faithfully a generated performance follows a score or lyrics. Listen to the output and revise your request or score as needed.
Learn More about ComfyUI_YuE2
Example workflows
Drag a workflow JSON from example_workflows/ onto your ComfyUI canvas:
- 01 — Text to Music: the shortest route from style and lyrics to saved audio.
- 02 — Staged Generation: inspect and control the score, semantic, synthesis, and decoding stages.
- 03 — Edited ABC Score: start with a short demonstration composition and deliberately change notes, durations, harmony, or tempo.
- 04 — Audio to Cover: transcribe a recording into a musical guide, then generate a new performance.
The text-to-music sample is based on the upstream song example. The score-editing workflow contains its own short demonstration composition.
Official guides and terms
- YuE2 generation guide — details of the official generation pipeline and controls.
- YuE2 ABC editing guide — the score format to use when editing a composition.
- YuE2 v0.1.6 release — the upstream release used by this extension.
- Official YuE2 model license — the exact conditions for using the weights and generated work.
The YuE2 weights use CC BY-NC 4.0 with additional permission for individual creators to publish and monetize generated outputs under the model’s terms. That permission does not authorize company commercial use of the weights; consult the official license before relying on a particular use case. The adapter has its own MIT license, while the upstream engine and transcription models retain their respective terms.
