Lipsync Video: Emotion-Aware Face Reanimation from Audio on playground and API | RunComfy

React-1 transforms a lipsync video through advanced video-to-video and audio-to-video processing. It not only syncs lip motion but also regenerates full facial performances—including micro-expressions, head, and eye movements—based on emotional cues. Compared to previous Lipsync-2 or Lipsync-2-Pro models, it excels in emotional realism and identity preservation.

Yes. React-1 is specifically designed for editing existing footage via video-to-video reanimation. You can upload a source lipsync video along with a separate audio track (audio-to-video input), select one of the six emotional presets, and the model regenerates facial performance while maintaining the original identity and scene style.

The current research preview of React-1 supports output up to 4K resolution but works most efficiently at 1080p. Maximum aspect ratio is 16:9, and it typically accepts one main video-to-video and one audio-to-video input per generation. Additional control sources such as ControlNet or IP-Adapter are not yet supported.

Yes. When producing a long lipsync video via the React-1 API or RunComfy playground, you may encounter memory limits beyond approximately 90–120 seconds of footage, depending on resolution. Exceeding this duration may result in partial frame drops during audio-to-video alignment.

After refining your lipsync video workflow in the RunComfy Playground, you can export configuration parameters (emotion, audio, video-to-video reference) and replicate them using RunComfy’s public API. The API mirrors the playground’s model settings and supports scripted generation for automated audio-to-video pipelines. Consult API docs or contact hi@runcomfy.com for integration keys.

React-1 employs fine-grained diffusion-based video-to-video transformation, aligning frame-level facial geometry with the given emotional prompt. During audio-to-video synthesis, emotional consistency in lip tension, eye focus, and head motion is maintained, ensuring realism in the reanimated performance.

React-1 performs best in dialogue-driven scenes and front-facing footage where facial details are visible. This enables precise audio-to-video mapping and expressive reanimation. It may struggle in cases with heavy occlusion, extreme lighting, or extreme profile angles affecting the video-to-video pipeline.

While most models generate entirely new content, React-1 enhances existing lipsync video performances through targeted video-to-video emotion editing. It focuses on identity retention, accurate timing, and emotional fidelity, whereas other generators prioritize scene diversity or stylization.

Commercial usage of any Lipsync or React model, including lipsync video outputs generated via video-to-video or audio-to-video processing, depends on Sync Labs’ official license terms. Always verify usage and attribution requirements on sync.so before deploying results in paid or public productions.

Traditional lipsync video models synchronize mouth movements via audio-to-video alignment only. React-1, however, applies emotion-driven video-to-video transformation, modifying subtle facial aspects such as brows and gaze to reflect emotional tone, offering a richer and more natural performance.