ComfyUI Node: Sung Pitch

Sung Pitch:

The PlenioSungPitch node is designed to extract the vocal melody from a recording and represent it as a pitch curve, which can be visualized in a score editor. This node is particularly useful for music producers and AI artists who want to analyze the vocal line of a cover song and compare it with transcribed notes. By utilizing a separation model, the node isolates the vocal stem from the rest of the audio mix, allowing for a more accurate pitch tracking. The pitch curve is generated using a YIN pitch tracker, which is known for its precision in detecting pitch variations. Even if the separation model is unavailable or fails, the node will still complete its process and provide feedback on why the pitch curve could not be generated. This ensures that the workflow is not interrupted, making it a reliable tool for audio analysis.

Sung Pitch Input Parameters:

audio

The audio parameter is the source recording from which the vocal melody will be extracted. It is expected to be a dictionary containing the waveform and sample rate of the audio. This parameter is crucial as it provides the raw data needed for pitch extraction. The quality and clarity of the audio can significantly impact the accuracy of the pitch curve.

separation

The separation parameter determines the method used to isolate the vocal stem from the audio mix. It can be set to a specific separation model located in the models/audio_separation folder or to "whole mix" if no separation is desired. Choosing a specific model can enhance the accuracy of the vocal extraction, while the "whole mix" option may result in less reliable pitch tracking as it captures the loudest pitch, which could be an instrument. The available options for this parameter are dynamically generated based on the models present in the specified folder.

Sung Pitch Output Parameters:

sung_pitch

The sung_pitch output represents the extracted pitch curve of the vocal line, formatted for visualization in the Song Sheet's piano roll. This output is crucial for users who wish to see how the original singing aligns with or differs from the transcribed notes. It provides a visual representation of the pitch over time, allowing for detailed analysis and comparison.

Sung Pitch Usage Tips:

• Ensure that the audio input is of high quality and contains a clear vocal line to improve the accuracy of the pitch extraction.
• Select an appropriate separation model from the available options to enhance the isolation of the vocal stem, which can lead to more precise pitch tracking.
• Use the "whole mix" option only when a separation model is not available, as it may result in less accurate pitch detection due to interference from other instruments.

Sung Pitch Common Errors and Solutions:

"No audio came in."

• Explanation: This error occurs when the audio input is empty or not properly formatted.
• Solution: Verify that the audio input is correctly provided and contains valid waveform data.

"The separation model (none) is not in models/audio_separation; put it there or choose 'whole mix'."

• Explanation: The specified separation model is missing from the expected directory.
• Solution: Ensure that the desired separation model is placed in the models/audio_separation folder or select the "whole mix" option if no model is available.

"The separation failed (error); the editor shows no sung pitch."

• Explanation: An error occurred during the separation process, preventing the extraction of the vocal stem.
• Solution: Check the integrity of the separation model and ensure that it is compatible with the audio input. Retry the process or select a different model if necessary.

"gives no vocal stem."

• Explanation: The separation model did not successfully isolate the vocal stem from the audio.
• Solution: Verify the configuration of the separation model and ensure that it is suitable for the type of audio being processed. Consider using a different model if the issue persists.