ComfyUI Extension: Avatar Graph

Avatar Graph Introduction

image (https://avatech-avatar-dev1.nyc3.cdn.digitaloceanspaces.com/public-download/github-readme/270574817-201a005b-7e00-4671-85a1-54937bf0704e.png)

avatar-graph-comfyui is an extension designed to help AI artists create real-time interactive avatars. This tool leverages the Blender bpy mesh API and Avatech Shape Flow runtime to provide a seamless experience for animating and interacting with avatars. Whether you're looking to create eye blinks, lip-sync animations, or full-body movements, this extension simplifies the process, making it accessible even for those without a strong technical background.

How Avatar Graph Works

At its core, avatar-graph-comfyui uses a node-based system to manage and animate avatars. Think of nodes as building blocks that you can connect to create complex animations. Each node performs a specific function, such as moving a part of the avatar or changing its expression. By connecting these nodes in different ways, you can create a wide range of animations and interactions.

For example, to make an avatar blink, you would use a node that controls the eyelids and connect it to a trigger that activates the blink. This modular approach allows for great flexibility and creativity, enabling you to build custom animations tailored to your needs.

Avatar Graph Features

Custom Nodes

• Segmentation (SAM): Allows you to select and create multiple image segments directly.
• Create Mesh Layer: Generates a mesh object from input images, typically a segmented part of the entire image.
• Join Meshes: Combines multiple meshes into a single object.
• Match Texture Aspect Ratio: Rescales the mesh to match the original image's aspect ratio.
• Plane Texture Unwrap: Fills mesh faces and projects textures onto them.
• Mesh Modify Shape Key: Modifies the vertex transform based on shape keys.
• Create Shape Flow: Creates a runtime shape flow graph for interactive inputs.
• Avatar Main Output: The primary output node for the avatar file, updating the embedded Avatar View.

Shape Flow

Shape Flow is a runtime graph that allows interactive inputs to affect shape keys in real-time. This feature is particularly useful for creating dynamic animations that respond to user interactions.

Best Practices for Image Input

• Generate a New Character Image: Use prompts like "looking at viewer, detailed face, open mouth, [smile], solo, eye-level angle" to create images that the tool can easily recognize.
• Inpaint Existing Images: Modify existing images to open the mouth or change expressions while maintaining consistency with the base image.
• Pose Constraints (ControlNet): Use reference images to guide the pose and expression of the avatar.

Avatar Graph Models

Recommended Checkpoint Models

Anime Style SD1.5

• Flat 2D Animerge
• Mistoonanime

Realistic Style SD1.5

• Realistic Vision V5.1
• Am I Real
• MajicMix Realistic These models help in generating high-quality avatars that can be further animated using the extension.

Troubleshooting Avatar Graph

Common Issues and Solutions

1. Nodes Not Connecting: Ensure that the nodes you are trying to connect are compatible. Check the input and output types.
2. Avatar Not Animating: Verify that all necessary nodes are connected and that the input images meet the recommended specifications.
3. Texture Issues: Use the "Match Texture Aspect Ratio" node to ensure that textures are correctly applied to the mesh.

Frequently Asked Questions

• What is --enable-cors-header used for? It enables communication between ComfyUI and the Avatech editor for real-time animation updates.
• Can I use the extension without --enable-cors-header? Yes, but you will lose the real-time preview feature. You can still download the avatar file and view it in other software like Blender.

Learn More about Avatar Graph

For additional resources, tutorials, and community support, consider the following:

• Join our Discord for real-time help and discussions.
• ComfyUI Examples for workflow examples and inspiration.
• VanJS Documentation (https://vanjs.org/tutorial) for understanding the underlying web stack used in the extension. By leveraging these resources, you can enhance your understanding and make the most out of avatar-graph-comfyui.