ComfyUI Node: FL YuE2 · Audio Adapter Trainer

FL YuE2 · Audio Adapter Trainer:

The FL_YuE2_AudioAdapterTrainer is a specialized node designed to facilitate the training of audio adapters within the FL YuE2 framework. This node is integral for users looking to customize and enhance audio processing capabilities by training acoustic adapters that can adapt to various audio styles and conditions. The primary goal of this node is to provide a robust and flexible training environment that allows for the fine-tuning of audio models, enabling them to produce high-quality audio outputs that align with specific artistic or technical requirements. By leveraging this node, you can effectively train models to handle diverse audio inputs, ensuring that the resulting audio is both stylistically and acoustically aligned with your creative vision.

FL YuE2 · Audio Adapter Trainer Input Parameters:

mode

The mode parameter determines the training approach for the audio adapter. It offers options such as "acoustic," "head," "joint," and "conditioned," each catering to different training needs. For instance, "conditioned" mode combines acoustic LoRA with continuous source audio across four decoder layers, while other modes focus on training the acoustic adapter or source head independently. This parameter is crucial as it defines the training strategy and influences the resulting model's adaptability and performance.

steps

The steps parameter specifies the total number of optimizer updates during training, including any resumed steps. It ranges from a minimum of 1 to a maximum of 100,000, with a default value of 1,000. This parameter directly impacts the training duration and the model's convergence, with more steps generally leading to better-trained models.

save_every

The save_every parameter sets the interval for checkpointing and held-out evaluation, with a default of 100 steps. It ranges from 1 to 100,000. Regular checkpoints are essential for monitoring training progress and ensuring that you can revert to a previous state if needed.

rank

The rank parameter defines the rank of the new acoustic LoRA, with a default value of 16 and a range from 1 to 128. This parameter affects the model's complexity and capacity, with higher ranks allowing for more detailed audio representations.

window_frames

The window_frames parameter determines the aligned training window size, measured in frames per second. It ranges from 25 to 512 frames, with a default of 200 frames, equivalent to 8 seconds. This parameter ensures that each audio pair used in training is sufficiently long, impacting the model's ability to learn temporal audio patterns.

head_learning_rate

The head_learning_rate parameter sets the learning rate for the head or joint mode, with a default of 0.00001. It ranges from 0.0000001 to 0.01. This parameter influences the speed and stability of the training process, particularly for the model's head components.

acoustic_learning_rate

The acoustic_learning_rate parameter specifies the learning rate for the acoustic LoRA and audio projection weights, with a default of 0.0001. It ranges from 0.0000001 to 0.01. This parameter is critical for adjusting how quickly the model learns acoustic features during training.

anchor_weight

The anchor_weight parameter is used in head/joint mode to preserve the starting encoder's source-token labels. It ranges from 0 to 1, with a default of 0.05. This parameter acts as a source-token anchor, ensuring that the model retains certain initial characteristics during training.

FL YuE2 · Audio Adapter Trainer Output Parameters:

YUE2_AUDIO_ADAPTER

The YUE2_AUDIO_ADAPTER is the primary output of the FL_YuE2_AudioAdapterTrainer node. This output represents the trained audio adapter, which can be used in subsequent audio processing tasks. The adapter is tailored to the specific training conditions and parameters set during the training process, making it a crucial component for achieving desired audio transformations and enhancements.

FL YuE2 · Audio Adapter Trainer Usage Tips:

• Experiment with different mode settings to find the best training strategy for your specific audio processing needs. Each mode offers unique benefits and can significantly impact the final audio quality.
• Adjust the steps parameter based on the complexity of your audio data and the desired level of model refinement. More steps can lead to better results but may require more computational resources.
• Use the save_every parameter to create regular checkpoints, allowing you to monitor training progress and revert to previous states if necessary.
• Fine-tune the learning_rate parameters to balance training speed and stability. Small adjustments can have a significant impact on the model's convergence and performance.

FL YuE2 · Audio Adapter Trainer Common Errors and Solutions:

Error: "Invalid mode selection"

• Explanation: This error occurs when an unsupported mode is selected for training.
• Solution: Ensure that the mode parameter is set to one of the supported options: "acoustic," "head," "joint," or "conditioned."

Error: "Steps value out of range"

• Explanation: The steps parameter is set outside the allowable range.
• Solution: Adjust the steps parameter to be within the range of 1 to 100,000.

Error: "Learning rate too high"

• Explanation: The learning rate for either the head or acoustic components is set too high, causing instability in training.
• Solution: Reduce the head_learning_rate or acoustic_learning_rate to a value within the recommended range to stabilize training.