ComfyUI  >  Nodes  >  ComfyUI-AudioScheduler

ComfyUI Extension: ComfyUI-AudioScheduler

Repo Name

ComfyUI-AudioScheduler

Author
a1lazydog (Account age: 4811 days)
Nodes
View all nodes (14)
Latest Updated
6/14/2024
Github Stars
0.1K

How to Install ComfyUI-AudioScheduler

Install this extension via the ComfyUI Manager by searching for  ComfyUI-AudioScheduler
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter ComfyUI-AudioScheduler in the search bar
After installation, click the  Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • High-speed GPU machines
  • 200+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 50+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

ComfyUI-AudioScheduler Description

ComfyUI-AudioScheduler enables loading mp3 files to drive animations and prompt scheduling using audio nodes, designed for integration with FizzNodes.

ComfyUI-AudioScheduler Introduction

ComfyUI-AudioScheduler is an extension designed to help AI artists integrate audio data into their creative projects. This extension provides a set of tools (nodes) that allow you to load audio files, analyze their properties, and use this information to control various aspects of your AI-generated animations or visualizations. By leveraging audio data, you can create more dynamic and synchronized multimedia experiences.

Key Features:

  • Loading Audio Files: Supports MP3 and WAV formats.
  • Amplitude Analysis: Reads and visualizes the amplitude of audio over time.
  • Frequency Analysis: Performs Fast Fourier Transforms (FFTs) to analyze frequency components.
  • Graphical Visualization: Generates graphs to preview amplitude and frequency data.
  • Dynamic Text Prompts: Creates text prompts driven by audio amplitude changes.

How ComfyUI-AudioScheduler Works

ComfyUI-AudioScheduler works by breaking down audio files into their fundamental components and using this data to influence other media elements. Here’s a simplified explanation:

  1. Loading Audio: The extension can load audio files in MP3 or WAV formats.
  2. Analyzing Audio: It reads the amplitude (volume) and frequency (pitch) information from the audio.
  3. Visualizing Data: The extension can create visual graphs of the amplitude and frequency data.
  4. Controlling Media: This data can then be used to control animations, generate text prompts, or influence other visual elements in your project. Imagine you have a piece of music and you want certain visual effects to sync with the beats or melody. ComfyUI-AudioScheduler can analyze the music and provide the necessary data to make this synchronization possible.

ComfyUI-AudioScheduler Features

LoadAudio

  • Input: List of audio file names (MP3, WAV).
  • Output: Loaded audio data.
  • Description: Loads audio files from a specified directory and prepares them for further analysis.

AudioToFFTs

  • Input: Loaded audio data, audio channel, frames per second.
  • Output: FFT data, total number of frames.
  • Description: Performs FFT on the audio data to extract frequency information. Useful for analyzing the pitch and tone of the audio.

AudioToAmplitudeGraph

  • Input: Loaded audio data, audio channel, lower and upper band range.
  • Output: Graph image of amplitude.
  • Description: Creates a visual graph of the amplitude within a specified frequency range. Helps in visualizing how the volume changes over time.

BatchAmplitudeSchedule

  • Input: FFT data, operation (avg, max, sum), lower and upper band range.
  • Output: Amplitude data.
  • Description: Calculates amplitude values from FFT data based on specified operations and frequency ranges. Useful for aggregating amplitude data.

ClipAmplitude

  • Input: Amplitude data, max and optional min amplitude.
  • Output: Clipped amplitude data.
  • Description: Clips amplitude values to a specified range. Ensures that the amplitude stays within desired limits.

TransientAmplitudeBasic

  • Input: Amplitude data, frames for attack, hold, and release.
  • Output: Adjusted amplitude data.
  • Description: Adjusts amplitude data with transient characteristics. Controls the attack, hold, and release behavior of the amplitude.

NormalizeAmplitude

  • Input: Amplitude data.
  • Output: Normalized amplitude data.
  • Description: Normalizes amplitude data, optionally inverting the values.

GateNormalizedAmplitude

  • Input: Normalized amplitude data, gating threshold.
  • Output: Gated normalized amplitude data.
  • Description: Gates normalized amplitude data based on a specified threshold.

NormalizedAmplitudeDrivenString

  • Input: List of text prompts, normalized amplitude data, triggering threshold, loop, shuffle.
  • Output: Dynamic text string.
  • Description: Generates text prompts based on changes in normalized amplitude. Can loop or shuffle prompts.

NormalizedAmplitudeToNumber

  • Input: Normalized amplitude data.
  • Output: Float and integer values of normalized amplitude.
  • Description: Converts normalized amplitude data to numerical values.

NormalizedAmplitudeToGraph

  • Input: Normalized amplitude data.
  • Output: Graph image of normalized amplitude.
  • Description: Generates a graph to visualize normalized amplitude data.

AmplitudeToNumber

  • Input: Amplitude data.
  • Output: Float and integer values of amplitude.
  • Description: Converts amplitude data to numerical values.

AmplitudeToGraph

  • Input: Amplitude data.
  • Output: Graph image of amplitude.
  • Description: Generates a graph to visualize amplitude data.

Troubleshooting ComfyUI-AudioScheduler

Common Issues and Solutions

  1. Audio File Not Loading:
  • Solution: Ensure the audio file is in MP3 or WAV format and located in the specified directory.
  1. No Output from FFT Node:
  • Solution: Check that the audio data is correctly loaded and the channel and frames per second are properly set.
  1. Graph Not Displaying:
  • Solution: Verify that the input data is correct and within the specified frequency range.

Frequently Asked Questions

  • Q: Can I use other audio formats?
  • A: Currently, only MP3 and WAV formats are supported.
  • Q: How do I adjust the frequency range for analysis?
  • A: Use the lower_band_range and upper_band_range parameters in the relevant nodes.
  • Q: What does normalizing amplitude do?
  • A: Normalizing scales the amplitude data to a standard range, making it easier to compare and use in visualizations.

Learn More about ComfyUI-AudioScheduler

For more detailed tutorials, documentation, and community support, you can explore the following resources:

  • ComfyUI Community Forums (https://community.comfyui.com)
  • These resources will help you get the most out of ComfyUI-AudioScheduler and connect with other AI artists who are using the extension in their projects.

ComfyUI-AudioScheduler Related Nodes

RunComfy

© Copyright 2024 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals.