YuE2-ComfyUI Introduction
YuE2-ComfyUI turns a musical idea, a style description, and lyrics into a complete song inside ComfyUI. It provides nodes for YuE2-3B, with 48 kHz stereo audio and an editable musical score as outputs. Generation runs on your own machine rather than through a hosted music-generation service.
You can start with as little as one sentence, write your own lyrics, bring in a MIDI melody, or transcribe an existing recording and sing its tune in a different style.
For AI artists, this creates several useful creative paths:
- Turn a concept into music: Describe a song and let a writing node prepare its style and lyrics.
- Work with your own words: Specify the genre, instruments, vocal character, language, and tempo separately from the lyrics.
- Shape the composition before rendering: Preview and edit notes, chords, tempo, and key in a visual score editor.
- Reinterpret a melody: Use a recording or MIDI file as the starting point for a new performance.
- Improve a finished result without replacing everything: Retake a section, replace words, change notes, insert a break, or extend a song.
- Create vocal-only or instrumental material: Separate vocals from a mix, or use a suitable instrumental LoRA adapter.
- Keep track of successful results: Save generation settings and reopen remembered songs for further editing.
The main generation node handles model discovery and the complete generation process. You do not need to choose a model file or fill in paths for a basic workflow. More detailed controls are available through the optional YuE2 Options node.
YuE2 Generate Song connected to lyrics and Save Audio, showing a completed 75-second song
In the author’s example, the style is entered on the generation node and the lyrics arrive from a text node. The 75-second song took 40.2 seconds to generate on an RTX 5090. Timing depends on your hardware and settings.
Licensing matters: The extension’s code is Apache-2.0, but the YuE2 and SheetSage2 model weights are licensed under CC BY-NC 4.0, which is non-commercial. Review NOTICE.md before planning commercial use.
How YuE2-ComfyUI Works
Think of the process as a small music production team: a composer plans the song, a performer interprets it, and an audio renderer turns that performance into a finished recording.
1. You describe the sound and provide the words
The two main text inputs have different jobs:
| Input | What it controls | Example |
|---|---|---|
style | How the song should sound | English, warm piano pop, expressive female voice, 88 BPM |
lyrics | What should be sung | Words organized under [Verse] and [Chorus] tags |
The style is not a substitute for lyrics, and lyrics do not reliably specify the production style. Keep musical direction in style and sung words in lyrics.
2. YuE2 plans a musical score
With the default cot: full setting, YuE2 first writes an ABC score: a compact text representation of musical notes, timing, sections, and harmony.
You do not need to understand ABC notation. The score editor displays the same information as:
- A piano roll, where notes appear as blocks on a time grid.
- Sheet music.
- The underlying ABC text, for users who want direct access.
This planning stage is useful because changing a composition before it is sung is much cheaper than rendering several complete songs.
3. YuE2 generates the performance
The model turns the score and lyrics into an internal representation of the performance. This describes the music before it becomes a normal audio file.
The seed influences this interpretation. Keeping the seed and settings fixed lets you repeat a result; changing the seed gives you another take.
4. The performance becomes sound
An acoustic stage builds the sound representation. By default, it uses 32 solver steps. A final decoder, called a VAE, converts that representation into 48 kHz stereo audio.
Style description + tagged lyrics
|
v
Editable ABC score
|
v
Internal performance
|
v
Acoustic sound representation
|
v
48 kHz stereo AUDIO
The cot setting changes the planning approach:
full: Plans melody and harmony. This is the default.melody: Plans the tune without chord guidance.off: Generates directly without a score. Score-based editing and transposition are therefore unavailable.
The role of the writing model
YuE2 Write Song uses a separate language model to turn your idea into a style description and lyrics. That model writes the instructions; YuE2-3B creates the music.
The writing node uses llama-cpp-python when it is already available. Otherwise, it can fetch official llama.cpp binaries and run the writer separately.
These are text-generation tools, not alternative music models. Their broader hardware support should not be interpreted as a hardware-support promise for the entire YuE2 song-generation pipeline.
YuE2-ComfyUI Features
Generate a complete song
YuE2 Generate Song is the simplest starting point.
[YuE2 Generate Song]
style
lyrics
seed
options optional
score_abc optional
|
+--> audio
+--> score_abc
+--> generation_info
Connect audio to Save Audio or another ComfyUI node that accepts AUDIO.
The node:
- Finds compatible weights already available on your machine.
- Downloads missing weights when downloads are allowed.
- Reports progress for each generation stage.
- Returns the score used for the song.
- Releases model memory after the run by default.
- Supports cancellation between generation tokens or decoding tiles.
For a first experiment, try:
Style:
English, warm piano pop, expressive female voice, 88 BPM
Lyrics:
[Verse]
Neon fades along the lane
Footsteps keep the time of rain
[Chorus]
Let the day come into view
Every road begins with you
Keep the seed on fixed while you learn what each setting does.
Write a song from one sentence
YuE2 Write Song prepares both inputs for the generation node.
[YuE2 Write Song]
idea: "a sad song about winter, female vocal"
|
+--> style --> [YuE2 Generate Song]
+--> lyrics -->
Its main controls are:
| Control | What you can do |
|---|---|
idea | Describe the subject, feeling, or intended voice in ordinary language. |
model | Use auto, a packaged model-list entry, or a compatible local GGUF model. |
language | Choose a language, or use auto to follow the language of your idea. |
length | Request approximately 8, 16, 24, or 32 sung lines. |
seed | Repeat the same written result under the same writing conditions. |
instructions | Add directions such as “first person,” “no chorus,” or “end on a question.” |
keep_model_loaded | Keep the writer in memory between runs; normally leave this off when song generation follows on the same GPU. |
The length choices are targets, not guarantees:
short: 8 lines.normal: 16 lines.long: 24 lines.very long: 32 lines.
The node reports how many lines it actually receives. Larger writing models can follow line budgets more closely, but the supplied context does not establish one universally best model for every song.
YuE2 Write Song with its idea, model, language, length, and instructions controls
Edit style and lyrics in a dedicated window
The Edit song... button opens a focused editor instead of requiring you to work in small node text boxes.
You can:
- Build a style from language, tempo, voices, instruments, and other sound descriptions.
- Reorder style parts or edit the final style line directly.
- Add, duplicate, move, and delete lyric lines and sections.
- Switch between structured sections and plain-text editing.
- Choose from 28 style examples drawn from actual demonstrations and gallery submissions.
- Resize the window or expand it to fill the browser tab.
Song editor showing style controls, lyric sections, and token counts
The final style line is the text the model reads. Naming two voices requests both, but does not reliably assign a particular singer to a particular section.
If you prefer ordinary ComfyUI text fields, enable Text on node. Inputs connected by wires remain controlled by their source nodes.
Writing useful lyrics
Put section tags on their own lines:
[Verse]
Your first line
Your second line
[Chorus]
Your chorus line
Another chorus line
The tags organize the song; they are not intended to be sung.
The editor can show how YuE2 divides text into tokens, the text pieces read by the model. Clicking a letter changes its capitalization, but this is not a reliable stress-marking system:
- A capital inside a word can split it into different tokens.
- A fully capitalized word may remain one token.
- Any capitalization change can produce a different song, even with the same seed.
For precise rhythmic emphasis, score editing is the stronger control.
Preview and edit the score
Edit score... opens the composition in a visual editor.
On YuE2 Generate Song, run the node once to obtain a score. On YuE2 Render Plan, you can write and edit the plan before generating audio.
Score editor with vocal notes, syllables, and metronome controls
The editor supports:
- Drawing, moving, stretching, and deleting notes.
- Selecting and moving groups of notes.
- Editing the voice line, instrumental line, chords, or all parts together.
- Copying, cutting, pasting, and duplicating phrases.
- Adding bars.
- Editing section names and boundaries.
- Undo and redo.
- Changing tempo.
- Moving the complete score into another key.
- Exporting the score with Save as MIDI....
Preview sound is only a guide
The editor can preview notes using piano and browser-generated sounds such as synth, bass, pluck, pad, strings, and drums.
These preview choices do not set the instruments in the generated song. YuE2 receives the score and your style description; the style determines the generated instrumentation.
The optional metronome helps you hear the beat. Its clicks never enter the generated song.
Fit lyrics to notes
The piano roll can display syllables under the vocal notes:
- White lines have the same number of notes and syllables.
- Amber lines have more or fewer notes than syllables.
Amber means “check this phrase,” not necessarily “this is wrong.” A singer can stretch a syllable over notes, but a large mismatch can cause words to run into later phrases or begin early.
These labels are a planning aid, not an exact transcript of where every syllable will be sung.
Understand what applying a score edit does
In the normal generation workflow, a score edit produces a new take of the whole song. It does not patch only the changed audio.
By default, an edit is associated with the style, lyrics, and planning mode it was made for. If those inputs change, the node avoids silently reusing an unsuitable edit.
Use:
- Reset score when you want a new composition.
- Keep for new words when you deliberately want to preserve a tune while changing the lyrics.
Small word substitutions tend to fit an existing tune more easily than adding whole lines. The score keeps its existing musical space.
Cover a recording
YuE2 Transcribe listens to a recording and outputs:
score_abc: The transcribed tune and, infullmode, its chords.lyrics: Section tags, recognized words when enabled, or your edited lyrics.
A basic cover workflow is:
[Load Audio] --> [YuE2 Transcribe]
|
+-- score_abc --> [YuE2 Generate Song]
+-- lyrics -->
[YuE2 Options: cot matches Transcribe mode] --> Generate Song
Write your own style description on the generation node to request a different genre, voice, or instrumentation.
YuE2 Transcribe with lyric and score editing controls
Important controls:
mode: fullretains the heard harmony as well as the tune.mode: melodyleaves the chords out, allowing accompaniment to follow the new style.lyrics_auto_recognitionenables word recognition; it is off initially.languagehelps prevent the speech model from mistaking the song’s language or writing a translation.second_hearingadds a Whisper transcription and asks the language model to choose words from both hearings.listenchooses between hearing the whole song in longer windows and hearing a minute at a time.
Set generation cot to the same value as transcription mode. A mismatch gives the model contradictory instructions.
Transcription is an interpretation, not a guaranteed exact score. Review both the lyrics and the musical timing before generating a cover.
Sing a MIDI or karaoke file
YuE2 Load MIDI reads existing note data rather than listening to audio. Reading a MIDI file requires no transcription-model download.
Supported formats are:
.mid.midi.kar.rmi
Use Choose MIDI file... or drop a file onto the node.
YuE2 Load MIDI showing tracks, automatic melody selection, and score information
You can choose:
- The track to use as the vocal melody.
- A separate instrumental track, or none.
melodymode without chords.fullmode with chords.- Whether to preserve the file’s sections.
The track list shows note ranges, instrument families, and why a track was selected. Drum tracks are not treated as singable melodies.
When a file contains simultaneous notes, the score’s melody line keeps one note at a time. Changing tempos are averaged into one score tempo, and the node reports relevant simplifications.
A track named Chords is read directly in full mode. Without one, chords are estimated from the arrangement and should be checked.
Put new lyrics onto an existing tune
without_sections is on by default. This sends the melody as a bare tune so the generation node can place lyric lines onto suitable phrases.
The extension looks for breathing spaces and phrases with note counts close to the syllable count. It can skip unsuitable riffs and repeat the tune when more space is needed.
This improves alignment, but a fast, densely packed instrumental theme may still be difficult to sing clearly.
For a karaoke file using its own words, turn without_sections off to preserve the file’s section structure.
Extract vocals or create an a cappella result
There are two ways to separate the voice from a song:
- Enable
vocals_onlyin YuE2 Options. - Place YuE2 Vocals Only after any audio source.
The separate node is useful when you want to keep both the complete song and its isolated voice from one generation.
The generated song is still created as a full mix first. A Mel-Band RoFormer separator then extracts the vocals.
For an a cappella workflow:
- Include “a cappella” in the style.
- Enable vocal separation.
The style request encourages the model to avoid long instrumental openings. Separation removes accompaniment that may still be generated.
Vocal separation preserves timing. If the original song has an instrumental intro, the extracted vocal can contain silence there.
Edit part of a finished song
YuE2 Edit Track changes selected material while retaining the original audio outside the edits, sample for sample.
It supports songs generated by this installation of the extension. It is not currently a general editor for arbitrary recordings.
Track editor with waveform, section markers, lyrics, and rendering controls
Available operations include:
| Operation | What it does |
|---|---|
| Retake | Sings selected bars again. |
| Cut | Removes a stretch and joins the remaining material. |
| New words | Replaces selected lyric lines and sings them again. |
| Notes... | Changes notes through the score editor and rerenders affected areas. |
| Go on... | Extends the song with more lyrics or a new ending. |
| Move | Repositions a section and regenerates material around the joins. |
| Break... | Inserts a 1–16-bar instrumental break. |
| Style... | Reinterprets a selected stretch with another style, optionally using a new melody. |
Select material by dragging over bars, or hold Alt for beat-level selection. Click a section label to select that section.
Edits are prepared first. Nothing is sung until you press Render.
Compare takes before committing
The takes control requests 1–4 alternatives for operations that regenerate material. The default is two.
You can:
- Compare alternatives at the same point in the song.
- Listen to As it was.
- Keep an already generated take without generating it again.
- Request more takes.
- Undo the last edit.
The edit controls include Seed, Variety, and Guide for the edit’s seed, sampling temperature, and text guidance.
Results remain model-dependent. A changed note may not be followed exactly, and an instrumental break may retain some voice.
Important editing limits
- Use the full mix, not a
vocals_onlyresult. - A WAV or FLAC saved from a remembered song can be recognized again.
- MP3 and other lossy formats change the samples and cannot be used to identify the song reliably.
- Songs made with
cot: offhave no score. They can be selected by time, but operations that require notes, bars, or sections are unavailable. - Changing a section’s style can cost substantially more than a short retake because the model generates a new-style performance from the beginning up to the selected area.
Reopen saved songs
The extension remembers each generated song’s score, internal performance data, and generation details in:
ComfyUI/user/yue2_comfy/songs
Saved songs... lets you search by style, lyrics, seed, note, or source, then reopen a song without singing it again.
Saved songs browser with search, song details, and previous edits
You can optionally retain the audio as FLAC for faster reopening. Remembered song data and retained audio each have a 4 GiB storage allowance; least-recently-used items are removed first.
Opening a saved song switches off the Edit Track node’s incoming audio connection. This avoids rerunning the upstream song generator just to edit an existing result.
Customize with LoRA adapters
YuE2 LoRA applies compatible adapters that modify YuE2’s behavior.
It distinguishes two parts of the music model:
- AR: Plans the score and performance, influencing melody, structure, and arrangement.
- NAR: Turns the performance into sound, influencing timbre, mix, and production.
YuE2 LoRA with separate AR and NAR strengths and adapter details
Each row shows which part an adapter affects. You can:
- Adjust AR and NAR strengths where supported.
- Enable or disable an adapter without removing it.
- Reorder adapters.
- Chain LoRA nodes to reuse a set.
- Search by filename or trigger word.
- Check compatibility information and warnings.
A trigger word is displayed but is not automatically inserted into your style.
Adapters must be supplied separately; the extension does not download them. It reads compatible files from ComfyUI’s configured LoRA folders and supports both ComfyUI and original m-a-p adapter layouts.
Generate instrumental music
Writing “instrumental” or “no vocals” in the style does not reliably stop the base model from singing.
The documented instrumental workflow uses Mothersuperior’s instrumental LoRA:
- AR strength: 1
cot:full- Lyrics:
[instrumental]or bare section tags without sung words
The author reported voice-free results in all 17 tested songs with that setup. This is a measured result, not a universal guarantee. Lower strength, sung lyrics, or cot: melody brought vocals back in the reported tests.
Adjust generation with YuE2 Options
You do not need this node for basic generation. Its defaults match the values used when the options input is empty.
| Setting | Artist-friendly explanation |
|---|---|
cot | Choose full score planning, melody-only planning, or no score. |
max_seconds | Set a duration ceiling; 0 estimates it from the lyrics. |
cfg_scale | Adjust text guidance. 0 selects the model’s released default. |
ode_steps | Set acoustic rendering steps. Default: 32. Fewer is faster but can sound thinner; more is not a guaranteed improvement. |
| Sampling controls | Change temperature, top-p, top-k, and repetition penalties for the generation stages. |
transpose | Move the complete score by semitones before it is sung. |
vae | Use standard for listening or legacy for reproducing published benchmarks. |
vocals_only | Return separated vocals instead of the complete mix. |
offload | Control how much of the model stays on the GPU at once. |
low_vram | Reduce GPU-memory use for smaller cards. |
attention_backend | Choose sdpa, fast, or flash; this affects speed and the resulting take. |
keep_model_loaded | Retain models between runs rather than releasing them. |
download | Allow automatic downloads, select a source layout, or disable downloads. |
quantization | Choose the build to fetch when appropriate. |
checkpoint | Explicitly choose compatible local YuE2 weights instead of automatic discovery. |
device | Choose where computation runs. |
For transposition, 2 means two semitones up, and -3 means three semitones down. This creates a new performance of the transposed score; it is not pitch-shifting the old recording.
Naming a key in the style is not reliable key control. Use transpose or the score editor’s key selector.
Plan first, render later
The advanced nodes separate composition from audio generation:
| Node | Purpose |
|---|---|
| YuE2 Plan | Writes a score without generating audio. |
| YuE2 Plan Batch | Writes several scores from consecutive seeds. |
| YuE2 Select Plan | Selects one plan without regenerating the batch. |
| YuE2 Render Plan | Sings a selected or edited plan and returns audio and latents. |
| YuE2 Decode Latents | Turns existing latents back into audio without generating another performance. |
Find these under YuE2/Advanced.
An untouched plan rendered with matching settings produces the same audio as the all-in-one generation node. A plan carries its own cot, because the score must be performed under the planning instructions it was written for.
Save generation information
Generate Song, Render Plan, and Decode Latents provide generation_info, a JSON text output containing details such as:
- Lyrics and style.
- The used seed.
- Loaded model name.
- Acoustic step count.
- Settings.
- ABC score.
- A format-version field.
Connect it to YuE2 Generation Info for separate outputs, including readable text, lyrics, seed, style, model name, and steps.
Absolute model paths are omitted. Older latent data without a recorded seed exports null; the field-extraction node requires a known seed.
YuE2-ComfyUI Models
Main music model and decoders
| Model or build | Role | When to use it |
|---|---|---|
| YuE2-3B | Main composition and song-generation model. Original backbone: approximately 6.76 GB, plus decoder and vocabulary. | Original m-a-p model layout or existing compatible files. |
| Comfy-Org YuE2 BF16 checkpoint | Repackages the standard model into one approximately 7.26 GB checkpoint. | Default automatic download and sharing weights with ComfyUI’s own YuE2 nodes. |
| Comfy-Org YuE2 INT8 ConvRot checkpoint | Approximately 3.69 GB quantized checkpoint. | Smaller downloads and disk use. It does not, by itself, reduce runtime VRAM. |
| YuE2-Vae | Standard audio decoder, approximately 0.49 GB. | Normal listening and production experiments. |
| YuE2-Vae-legacy | Alternative decoder, approximately 0.49 GB. | Reproducing published benchmark behavior or comparing decoder sound. |
The BF16 repack and original standard weights were checked by the author and produce equivalent results. INT8 quantization is lossy, so its restored weights can produce a different song.
INT8 download versus low_vram:
- The INT8 checkpoint is expanded to BF16 when loaded.
low_vramkeeps model layers quantized on the GPU during execution.
Use low_vram, not merely the smaller checkpoint, when the goal is reducing GPU-memory use.
An explicit checkpoint selection takes priority over quantization. Missing or incompatible explicit files are refused rather than silently replaced.
Writing models
These eight packaged entries are available to YuE2 Write Song and to the language-model stages of YuE2 Transcribe.
| Model | Download size | Approximate VRAM |
|---|---|---|
| Qwen3.5-4B Q4_K_M | 2.55 GB | ~4 GB |
| Qwen3.5-9B Q4_K_M | 5.29 GB | ~7 GB |
| Gemma 4 12B Q4_K_M | 6.63 GB | ~9 GB |
| Qwen3.8-27B Q4_K_M | 15.33 GB | ~17 GB |
| Gemma 4 26B-A4B Q4_K_M | 15.78 GB | ~19 GB |
| Gemma 4 31B Q4_K_M | 17.07 GB | ~20 GB |
| Qwen3.6-27B Q5_K_M | 18.17 GB | ~19 GB |
| Gemma 4 26B-A4B Q6_K | 21.58 GB | ~24 GB |
The author reports sizes using 2³⁰-byte units. VRAM figures were measured for selected models at an 8192-token context and estimated for the others.
Choosing a writer:
- Start with Qwen3.5-4B Q4_K_M if you want the smallest packaged default.
- Consider a larger model when line-count discipline or transcription cleanup matters and your hardware has room.
- The 26B-A4B entries are mixture-of-experts models: they store many weights but use only part of them for each generated token. Their larger file size does not necessarily mean slower writing.
- Treat model choice as a writing decision. It changes the style and words supplied to YuE2, rather than directly replacing the music generator.
auto reuses a suitable local GGUF and downloads the default Qwen writer only when none is available.
Compatible local GGUFs can also be discovered in ComfyUI model folders, the Hugging Face cache, and accessible Ollama stores. LoRA files, embedding models, and other files that cannot write independently are excluded.
Use Model list... to inspect, add, edit, or check entries.
Writing-model list with model sizes and availability
Supporting audio models
These are downloaded only when the relevant feature needs them, provided downloads are enabled.
| Model | Purpose | Approximate additional size |
|---|---|---|
| SheetSage2 from Comfy-Org/YuE2 | Transcribes a recording into a musical score. | 1.29 GB |
| Qwen3-ASR-1.7B | Recognizes sung words and evaluates lyrics in relevant editing operations. | 3.80 GB, plus small supporting files |
| Whisper large-v3 turbo | Provides the optional second hearing of lyrics. | 1.62 GB |
| Mel-Band RoFormer | Separates vocals and helps place track-editing bars on the recording. | 0.85 GB |
| Qwen3-ForcedAligner-0.6B | Locates where lyric lines are sung for word replacement and timing. | 1.84 GB |
Model discovery and downloads
The extension reuses compatible files already present in configured model folders or supported caches. Files generally do not need to be copied or renamed.
By default:
- The combined checkpoint downloads into
models/checkpoints. - SheetSage2 uses
models/audio_encoders. - Other supporting models use the registered YuE2 model folders.
- Writer downloads use a configured
LLMfolder.
Downloads can resume after interruption. Partial transfers use .part files so incomplete checkpoints are not mistaken for finished models.
Set download: off to prevent automatic downloads. The node then reports missing files, their links, and the expected locations.
YuE2-ComfyUI Workflow Templates
Open Workflow → Browse Templates and look under this node pack’s name.
Each template is self-contained, includes a Read me first note, and starts with fixed seeds.
| Template | What you can explore |
|---|---|
| A song from one line | Turn one idea into style, lyrics, and audio. |
| A song from style and lyrics | Generate directly from your own text. |
| A longer song | Use the options node to control the duration ceiling. |
| The score first | Write, inspect, and edit the score before singing. |
| Four scores, one song | Compare composition alternatives without rendering every candidate immediately. |
| One take, both decoders | Hear the same performance through standard and legacy decoders. |
| Sing a MIDI file | Use an existing melody. |
| Cover a song | Transcribe and reinterpret a recording. |
| An a cappella song | Combine an a cappella style with vocal extraction. |
| A song with LoRA | Add your own compatible adapters. |
| Edit a song | Make local changes to generated material. |
| An instrumental | Use the documented instrumental adapter setup. |
For a straightforward first session, choose A song from style and lyrics. Choose The score first when you want more control over composition before spending time on audio generation.
YuE2-ComfyUI Performance and Reproducibility
Memory expectations
The author reports the following song-generation measurements:
| Configuration | Approximate GPU memory |
|---|---|
| Full model kept on the GPU | ~10 GiB |
offload: on | ~4.4–4.5 GiB |
offload: on with low_vram | ~3.1 GiB |
These figures covered both short and four-minute songs. Supporting recognition and editing models have their own memory needs; for example, recognizing the words of a four-minute recording was reported at about 5.5 GiB, or 3.5 GiB with low_vram.
An NVIDIA RTX 30-series card or newer is the main documented GPU target. Older RTX 20- and GTX 16-series cards use FP16 for the final stage; their speed was not measured, and they cannot use fast or flash. CPU generation is supported but is described as roughly an hour per song.
Control song length realistically
max_seconds is a ceiling, not a requested exact duration.
0estimates a limit from your lyric lines.- A short lyric may end naturally well before a large ceiling.
- A large ceiling does not create more planned lyrics.
- A low ceiling can cut off a score before it finishes.
The score editor marks the stopping point with a dashed yellow line and dims bars that will not be heard.
For a longer song, provide more lyrics as well as enough duration. The model’s score budget is approximately five minutes of music, and the maximum generation ceiling is 360 seconds; neither guarantees a finished song of that length.
Repeat a result
Keep all of the following unchanged:
- Seed.
- Exact style text.
- Exact lyrics, including capitalization.
- Duration ceiling.
- Checkpoint and quantization behavior.
- Attention backend.
- Decoder and generation settings.
- LoRA files and strengths.
The author reports bit-for-bit repeatability under matching tested conditions. Important qualifications include:
sdpa,fast, andflasheach repeat consistently, but produce different takes from one another.- Changing
max_secondscan change the performance, not merely the ending. low_vramchanges the generated performance because quantization is lossy.- Very tight decoding memory can cause tiny audio differences between offload configurations.
- Version changes can affect sound; see the update notes below.
For the writing node, the in-process Python runtime and separate llama.cpp runtime can produce different words from the same seed. Repeatability applies within the same execution path.
What's New with YuE2-ComfyUI
The supplied documentation identifies the following version-related changes. For the complete release history, see CHANGELOG.md and the latest release.
Version 0.9.0: Faster, repeatable acoustic attention
The acoustic stage moved to cuDNN attention, improving speed while retaining repeatability.
A seed from version 0.8 keeps the same score and internal performance, but the rendered sound can differ slightly. This matters when comparing an older saved result with a new render.
Version 0.9.3: Key-spelling correction
The original documentation points to the 0.9.3 changelog for a correction involving difficult key signatures.
For example, writing a transcribed score in B-flat minor rather than the equivalent A-sharp minor avoids a documented YuE2 interpretation problem involving E-sharp and B-sharp. The practical benefit is avoiding unintended semitone errors in affected covers.
Other documented changes without a supplied version number
The current documentation also describes these improvements:
fastreplaces the oldercudnnattention choice. Older workflows namingcudnnnow run asfast.- Local checkpoint selection allows explicit use of compatible weights, with clear refusal of missing or incompatible files.
- Generation-information output makes it easier to preserve lyrics, seeds, settings, and score details.
- Visual song, score, and track editors provide more direct creative control.
- Chord editing and MIDI round-tripping support moving composition work between the extension and a DAW.
- Optional second-hearing recognition can improve lyric transcription, especially in the reported Russian-song tests.
- Saved-song browsing and local track editing reduce the need to regenerate complete songs.
- Configurable writing-model entries make existing local language models easier to reuse.
- Low-memory modes make song generation and recognition more accessible on smaller GPUs.
Release numbers are not assigned here where the supplied documentation does not provide them.
Troubleshooting YuE2-ComfyUI
Generation runs out of GPU memory
- Add YuE2 Options and connect it to the affected node.
- Set
offloadtoon. - If memory is still insufficient, enable
low_vram. - Leave
keep_model_loadedoff, including on the writing node when it precedes song generation. - Use Unload Models to release retained models and session takes.
- For a writing-model memory problem, select a smaller writer.
Downloading the INT8 checkpoint alone does not solve a runtime VRAM shortage.
The song ends before all lyrics are sung
- Open Edit score....
- Look for the dashed yellow stopping line.
- Check whether later sections fall beyond it.
- Increase
max_secondsif the ceiling is too low. - Check whether your lyrics fit the score’s phrases, especially when keeping a tune for new words.
Changing the ceiling can create a different take. It is best to settle the duration limit before searching for a favorite seed.
The song continues after the words run out
- Use
max_seconds: 0to restore the lyric-based ceiling. - Avoid giving a short lyric a needlessly large duration allowance.
- For MIDI workflows, keep the automatic phrase-placement behavior unless you specifically need the file’s own sections.
- Check warnings about a tune that fits the words poorly.
A longer ceiling gives the model permission to continue; it does not guarantee useful additional material.
My cover has the wrong rhythm or tempo
- Compare the transcription’s tempo with the recording.
- Change
listento a minute at a time. - Transcribe again.
- Inspect the score’s bars and meter changes.
- Compare the result with the whole-song transcription.
Shorter listening windows improved the reported vocal-song tests, but can introduce more stitching seams. They were not consistently beneficial for dense instrumental recordings.
Recognized lyrics are wrong or translated
- Enable
lyrics_auto_recognitionif it is off. - Set
languageexplicitly instead of leaving it onauto. - For multilingual material, list the languages separated by commas.
- Try
second_hearingif the first transcription remains poor. - Open Edit lyrics... and correct the words before generating the cover.
Recognition is not exact. A second hearing adds another model download and extra processing, and does not improve every song.
A cover warns about mode and cot
- Check
modeon YuE2 Transcribe or YuE2 Load MIDI. - Set
cotin the generation options to match it:fullwithfull.melodywithmelody.
- Run the generation again.
A score edit is ignored
- Check whether the style, lyrics, or
cotchanged after the edit. - Read the node’s message explaining which score it is using.
- Restore the original inputs, or deliberately enable Keep for new words.
- Confirm the edited bars are before the duration ceiling.
- Use Reset score if you intended to compose a new tune.
If a changed note is sung approximately rather than exactly, try another seed with the same edited score. YuE2 follows many score edits, but is not an exact sampled-instrument renderer.
MIDI lyrics are unclear or missing
- Confirm that the intended melody track is selected.
- Keep
without_sectionson for new lyrics over a bare tune. - Match
modeand generationcot. - Read warnings about note-to-syllable mismatches.
- Shorten or rewrite lines to fit the musical phrases.
- Inspect the score before rendering again.
A melody with few breathing spaces or a very fast note pattern may remain unsuitable for clear lyrics.
“Instrumental” still produces a singer
- Use the documented instrumental LoRA.
- Set its AR strength to 1.
- Use
cot: full. - Remove sung words from the lyrics.
- Use
[instrumental]or bare section tags.
The base model’s style prompt alone does not reliably suppress vocals.
My a cappella result still has instruments
- Include “a cappella” in the style.
- Enable
vocals_only, or add YuE2 Vocals Only after generation. - Keep the full mix separately if you plan to edit it later.
The style changes the arrangement request; the separator extracts the voice.
Edit Track cannot find my song
- Confirm the song was generated by this installation.
- Try Saved songs....
- If loading an audio file, use the original WAV or FLAC.
- Do not use MP3 or another lossy conversion.
- Use the full mix rather than an extracted-vocal result.
If the remembered song data has been deleted or aged out of storage, the file alone may no longer provide the required editing data.
A LoRA is unavailable or does nothing
- Use YuE2 LoRA and inspect its compatibility information.
- Check whether the adapter changes AR, NAR, or both.
- Confirm the row is enabled.
- Add any required companion adapter.
- Put the trigger word in the style yourself.
- Match the adapter’s documented
cotsetting.
An NAR-only adapter can change the sound without changing the melody or words. Some original m-a-p layouts are not applied by ComfyUI’s generic LoRA loader, so use the extension’s dedicated node.
Missing models or interrupted downloads
- Check whether
downloadis set tooff. - Read the node’s missing-file message.
- If downloads are allowed, run again to resume an interrupted transfer.
- Check available disk space.
- For an explicit checkpoint selection, confirm the file still exists and is compatible.
- Refresh node definitions after adding a checkpoint.
If the node reports missing tiktoken, follow the interpreter-specific instruction in its message rather than using an unrelated Python environment.
New controls or tokenizer marks do not appear
- Restart the ComfyUI server after updating the extension.
- Refresh the browser interface or node definitions as appropriate.
- For lyric token marks, confirm the YuE2 weights are present on disk.
The song, score, and track editor windows are currently English-only.
Audio playback freezes other ComfyUI previews on Windows
The original documentation identifies a Windows file-serving issue where a long audio request can stall other files.
- Stop ComfyUI.
- Start it with
AIOHTTP_NOSENDFILE=1in its environment. - Reopen the workflow.
The Edit Track window uses its own playback route and is not affected by that issue.
Model-list or saved-song management buttons are disabled remotely
These controls are restricted when the browser is on another machine.
You can use an SSH tunnel to access the server through localhost. The extension also provides YUE2_ALLOW_REMOTE_EDITS=1, but this enables management actions for anyone who can reach the server.
Do not enable that switch on an untrusted or publicly reachable server.
Frequently asked questions
Do I need music-theory knowledge?
No. You can start with a style and lyrics or a one-sentence idea. The piano roll provides a visual way to learn and adjust notes later.
Does generation require an online music service?
No. Generation runs locally. Network access is needed for model or runtime downloads when those files are not already available.
Can I assign one singer to the verse and another to the chorus?
Not reliably through voice labels or section tags. The author tested these approaches and did not find dependable per-section singer assignment.
Can I choose the key in the style prompt?
Use transpose or the score editor’s key control. Naming a key in the style was ineffective in the reported tests.
Can I export to a DAW?
Yes. Save as MIDI... exports voice, instrument, and chord tracks with musical timing information. Compatible files can be read back through YuE2 Load MIDI.
Why does the same seed sound different after I change one setting?
The seed is only one part of the generation conditions. Text, duration ceiling, backend, model build, adapters, and other settings also affect the result.
Can I edit a song from another application?
You can transcribe or extract vocals from external audio. Local Edit Track operations currently require a song generated and remembered by this extension.
Can I use the output commercially?
The extension code and model weights have different licenses. YuE2’s weights are non-commercial. Do not assume the Apache-2.0 code license authorizes commercial use of the model; review the model terms and NOTICE.md.
Learn More about YuE2-ComfyUI
Start with the artist-facing resources
- ComfyUI workflow templates: Open Workflow → Browse Templates and choose a YuE2 template. Each includes practical notes about inputs, downloads, and settings.
- Project repository: Extension documentation and current source.
- Changelog: Detailed changes across releases.
- Latest release: Current packaged release.
- Issue tracker: Existing problem reports and a place to describe reproducible failures.
When reporting an unexpected song or editing result, include the node involved, model selection, seed, relevant options, and the warning or error text. These details make it easier to distinguish a model limitation from a workflow problem.
Explore the models behind your workflow
- YuE2-3B model page: Main music-model information.
- Comfy-Org/YuE2: Combined checkpoints and SheetSage2 files.
- YuE2 standard VAE: Standard audio decoder.
- Instrumental LoRA: Adapter instructions for music without vocals.
- Default Qwen writing model: The small writer fetched when no suitable local writer is available.
Optional background on the writing runtime
You do not need these resources for ordinary song generation, but they explain the local text-generation tools used by the writing node:
- llama.cpp: The engine used to run GGUF writing models.
- llama-cpp-python documentation: Documentation for the optional in-process Python runtime.
License references
- LICENSE: Extension code license.
- NOTICE.md: Model, implementation, and third-party licensing details.
The code is Apache-2.0. YuE2 and SheetSage2 weights are CC BY-NC 4.0. The default writer, Qwen speech model, and forced aligner use Apache-2.0; Whisper and the vocal separator use MIT. LoRA adapters have their own terms, so check the license of every adapter you add.
