Difforum Introduction
Difforum helps you direct an AI video shot in ComfyUI. Instead of trying to describe every camera move and change of pace in a prompt, you can arrange scenes on a visual timeline, choose camera moves, mark important frames, and draw an energy curve. You can preview the result before committing to a full render.
The same direction can be used with different rendering approaches: MiniMax H3, LTX-2 / 2.5, image models such as SDXL and Flux, or a real-time sampler. Difforum can also export the camera motion for After Effects and Blender, helping you align titles, 3D elements, and compositing with the AI-generated shot.
Difforum is useful when you need a shot to move in a particular way, reach a particular image at a particular moment, or maintain a consistent visual idea across previews and renders. Its templates give you starting points for everything from a quick storyboard to a longer, multi-scene piece.
How Difforum Works
Think of the Director as a combination of a storyboard and an editing timeline. You tell it what happens when; a connected renderer decides what the finished frames look like.
- Set up the shot. Choose its duration and aspect ratio in Setup. Difforum can align the frame count with the requirements of the model you plan to use.
- Lay out the direction. In Director, add scenes with prompts and moods, choose camera moves, place keys at important moments, and shape the energy curve. A scene can also have its own picture and sound description.
- Preview the motion. Press ▶ to see the camera move on your picture. Every template also includes an Animatic, a quick preview of the whole shot with timecode, prompts, moves, energy, and keys shown on screen. Use Previz only when you want to queue that preview without running the render.
- Choose how to render. Connect Director’s
directionoutput to a supported rendering node. The same timeline can guide a video model, drive frame-by-frame image generation, or control a live performance. - Finish or export. Add a restyle or finishing pass if needed, or export the camera for use in After Effects or Blender.
For example, you could begin with a still photograph, set a slow inward camera move for the first scene, mark a new image at a musical beat, and raise the energy afterward. The Animatic lets you check that timing quickly. You can then render with a video model or use the Feedback Sampler for a more visibly changing, Deforum-style result.
You do not need to write camera expressions to use the visual controls. Expressions remain available for more advanced direction.
Difforum Features
Direct scenes, camera moves, and timing
The Director brings your shot’s main decisions into one timeline. You can choose from 25 visual camera presets, set exact frames with keys, and draw an energy curve to control how the shot develops. The mouse wheel zooms the timeline, making both short clips and longer shots easier to edit. You can also double-click the preview and use dragging, scrolling, and shift-dragging to frame a move by hand.
Scenes can carry their own prompts, moods, pictures, and sound descriptions. The Script view presents the timeline as readable frame ranges, such as 0-35 | calm | zoom_in slow | prompt | sound: rain. When you use a video model that supports audio, a scene’s sound description can inform its soundscape.
Preview before rendering
The Animatic shows the full sequence without waiting for the final model render. It displays the camera move alongside the timing information you entered, so you can spot a key that arrives too early or a transition that feels too fast. The Director’s Previz only button temporarily mutes render outputs when you queue the workflow; select it again when you are ready to render.
Keep a look across renderers
Director’s look setting offers these choices: cinematic, documentary, deforum_morph, animatediff_dream, disco_diffusion, vqgan_clip, flicker_experimental, psychedelic, music_video, stop_motion, and hand_drawn.
A selected look adjusts colour, detail, and energy for the Feedback Sampler and adds a corresponding style sentence to H3 or LTX prompts. It is a way to carry the same creative intention between rendering methods, although those methods will not produce identical images.
Place pictures at important moments
Use Keys with Keyframe Images when your video needs to reach specific pictures at specific times. The Feedback Sampler moves through those images, H3 and LTX guides use them as anchors, and the Animatic shows where they occur.
You can also connect pictures directly to Director for individual scenes. Difforum pins each picture to its place on the timeline and carries it through the direction connection to compatible guides, the Feedback Sampler, and the Animatic. This is helpful when planning an installation, a music-led piece, or a story in which a particular image must appear on a beat.
For a collection of prepared stills, Keyframe Assets can load images named by time, such as 0s_wide.png and 4.5s_sky.png. Shot Script can build a Director timeline from text, CSV, a file in input/, or another STRING-producing node. These options let you plan or style assets outside the timeline and bring them into a shot.
Choose a rendering style
- H3 Guides / H3 Shot turn your direction into frames and a camera prompt for MiniMax H3. Fill Reveal (AI) paints areas exposed by a camera move. H3 templates can use a small initial render followed by H3 latent upscaling and resampling for additional detail.
- LTX Guides place keyframes into an LTX-2 / 2.5 generation.
- Feedback Sampler repeatedly reinterprets the previous frame with an image model, producing the evolving Deforum-style look. Its depth handling supports parallax; without a depth map, 3D moves become pseudo-3D and the node indicates that limitation. A higher cadence means fewer frames are directly diffused, with intermediate frames blended between them, trading some frame-by-frame generation for speed. Energy can also affect how much sampling work a frame receives.
- Live Sampler plays feedback inside its node for real-time work, including turbo-model and webcam workflows, with Spout / OBS output.
Restyle and finish a clip
Restyle repaints frames from an H3, LTX, or existing video render with an image model. Its feedback carries the previous painted frame along the clip’s motion, allowing looks such as deforum morph, animatediff boil, disco flicker, and clean restyle while retaining the underlying motion. Look Mix can add grain or flicker cuts afterward.
Other finishing nodes include Upscale, Loop, Symmetry, Echo Trails, Flow Stabilize, and Detail Guard. Use the finishing passes that serve your shot rather than assuming every workflow needs all of them.
Move between Difforum and other tools
Camera Export writes camera data for After Effects (.jsx), Blender (.py), or JSON. Camera Import brings camera data back into a workflow. This is especially useful when a title, graphic, or 3D object must follow the movement of an AI shot.
The templates arrange work into recognizable blocks: Control, Direction, and Previz above Models → Render → Restyle → Upscale 2K → Output. Workflow Switches let you turn blocks on or off and jump to them, so you can work on one stage without losing the overall layout.
Difforum Models
Difforum is not itself a video model. It provides direction, guides, sampling, and finishing tools that work with the models used in your ComfyUI workflow. Choose a path based on the result you want and the models you have available.
| Model or rendering path | What it contributes | When to consider it |
|---|---|---|
| MiniMax H3 | Uses Difforum’s camera-derived frames and guides to generate motion. The first/last-frame and multiframe-reference workflows offer different ways to anchor a shot. | When you want a video model to perform a planned camera move or travel between important images. |
| LTX-2 / 2.5 | Receives Difforum keyframes through LTX Guides. | When you already use an LTX workflow and want to guide its timing and images from Director. |
| Image models, including SDXL, Flux, and SD1.5 | Power the Feedback Sampler, which re-imagines frames in sequence. | When the changing, painterly Deforum look is more important than the motion style of a video model. |
| Turbo image models | Support faster feedback workflows and the Live Sampler. | When responsiveness matters, such as live visuals or a quick look pass. |
| AnimateDiff motion modules | AnimateDiff workflows can process an existing video or travel between pictures with per-frame prompts. | When you specifically want AnimateDiff motion or its characteristic animated treatment. |
| Depth Anything 3 | Supplies depth for a parallax workflow. | When you want camera movement to respond to the apparent distance of objects in an image. |
Some templates require additional models or ComfyUI components. For instance, the AnimateDiff video templates call for AnimateDiff-Evolved, Advanced-ControlNet, an SD1.5 checkpoint, a motion module, and appropriate ControlNets. The faster AnimateLCM variation also calls for its motion module and LoRA. Check a template’s listed requirements before using it; the Models guide provides more model and settings guidance.
Difforum Templates
Open Templates → Difforum in ComfyUI to explore the included workflows. For a first look, 01_storyboard_no_model needs only an image: shape the timeline, preview the move, and queue the Animatic without selecting a generation model.
| Template | What you can explore | Main requirement |
|---|---|---|
01_storyboard_no_model | Direct and preview a shot without a generation model | An image |
02_feedback_sdxl | Deforum-style feedback with an image model | SDXL, SD1.5, or Flux |
03_parallax_3d_depth | Camera parallax informed by image depth | Depth Anything 3 |
04_audio_reactive | Camera movement that responds to music | An audio file |
05_live_turbo | Live feedback, webcam, and VJ output | A turbo model |
06_seamless_loop | A loop designed without a crossfade | A checkpoint |
07_ltx_guides | Director keyframes for LTX-2 / 2.5 | Your LTX graph |
08_h3_first_last_frame | A camera move anchored by first and last frames | MiniMax H3 fl2va |
09_camera_to_ae_blender | Camera export to After Effects or Blender and import back | No generation model |
10_h3_multikeyframe_guides | Multiple camera-derived anchors for H3 | MiniMax H3 ref2va |
11_h3_deforum_look | A feedback-generated look with H3 motion | Turbo SDXL and H3 ref2va |
12_long_shot_keys | A 30-second, five-scene shot with three image keys | A turbo model and three images |
13_restyle_any_video | Apply a Deforum, AnimateDiff, or Disco-style restyle to an existing clip | A turbo model and a video |
14_animatediff_on_video | AnimateDiff video-to-video with depth or edge guidance | AnimateDiff-Evolved, Advanced-ControlNet, SD1.5, a motion module, and ControlNets |
15_animatediff_lcm_fast | A faster AnimateLCM variation with a higher-resolution pass | Template 14’s components plus AnimateLCM module and LoRA |
16_h3_keyframe_assets | Drive H3 from prepared stills and a shot script | H3 ref2va and your stills |
17_story_three_shots_h3 | Make scene stills and play them as three H3 shots | An image model and H3 ref2va |
18_story_living_photograph | Keep one place grounded while the camera and light change gently | An image model and Depth Anything 3 |
19_story_painted | Build a growing subject in a 12 fps painted look | A turbo image model |
20_director_pictures_h3_sound | Give H3 a picture, camera direction, and sound description per scene | H3 ref2va; AnimateDiff-Evolved for the optional pass |
21_animatediff_travel_pictures | Use SparseCtrl image keys and changing prompts for AnimateDiff in-betweens | AnimateDiff-Evolved, Advanced-ControlNet, SD1.5, and SparseCtrl RGB |
Troubleshooting Difforum
The shot does not follow the camera move I expected.
- Preview the move in Director to check its framing and timing.
- Check the Animatic to confirm that the move appears in the intended scene and frame range.
- Confirm that
directionis connected to the guide or renderer you intend to use. For H3, the camera-derived guides are what turn the planned move into image anchors and a camera prompt.
A planned picture appears at the wrong moment.
- Check its position on the Keys track or its scene position in Director.
- If using Keyframe Images, make sure each intended key has a corresponding picture.
- If using Keyframe Assets, check the time in each filename, such as
4.5s_sky.png. - Run the Animatic to inspect the timing before rendering again.
A 3D move looks flat.
- Check whether the workflow supplies a depth map.
- If it does not, the Feedback Sampler uses pseudo-3D movement rather than depth-based parallax.
- For depth-based movement, examine the
03_parallax_3d_depthtemplate and the Bridges guide.
Queue produces a preview but not the final video.
- Check whether Previz only is active in Director.
- Turn it off when you are ready to render.
- Check Workflow Switches to make sure the relevant render block is enabled.
A template needs components that are not in my current workflow.
- Check the template’s Main requirement in the table above.
- Start with a template that matches the models you already use;
01_storyboard_no_modelis the model-free introduction. - Consult the Models guide and node reference for the relevant model path and node inputs.
Which renderer should I start with? Use the model-free storyboard to learn Director. Then choose H3 or LTX if you want to guide a video-model render, Feedback Sampler for the evolving Deforum look, or Live Sampler for a responsive performance workflow.
Do I have to write expressions? No. Director’s visual camera and timing controls are designed to work without them. The Expressions guide is available if you later want finer control.
Can I use a video I already made? Yes. The 13_restyle_any_video template is designed to restyle an existing H3, LTX, or live-action clip. The AnimateDiff video-to-video templates offer another approach with their listed model requirements.
Learn More about Difforum
- Recipes: Short projects and shot scripts, including settings for longer pieces.
- Bridges: How Difforum connects to H3, LTX, depth, keyframes, After Effects, and Blender.
- Node reference: Inputs and outputs for every node. Nodes also show descriptions inside ComfyUI.
- Models: Model choices and settings.
- Performance: Speed and quality tradeoffs, including Apple Silicon guidance.
- Prompt pack: Ready-made scene sets to adapt for your own work.
- Expressions: Optional camera-expression syntax for advanced control.
- Migrating from 0.x: Guidance for opening older workflows and finding their newer equivalents.
