ComfyUI-WanAnimatePreprocessV2 Introduction
ComfyUI-WanAnimatePreprocessV2 prepares images or video frames for Wan Video 2.2 Animate. It identifies a person’s body position, hand and facial landmarks, and eye movement, then turns those detections into data and images that an animation workflow can use.
For you, the practical benefit is steadier motion guidance. A pose detector can otherwise make a limb disappear for a frame, shift a face crop from side to side, or mistake image noise for eye movement. This extension offers controls to reduce those problems and a visual debug output so you can see what was detected before using it in a generation.
Its main outputs include:
- Pose data describing body, hand, and face positions.
- Face crops at 512 × 512 pixels, one for each frame.
- Iris and gaze data describing detected pupil positions and eye direction.
- Skeleton images from a separate drawing node, for workflows that need visual pose guidance.
- Debug images that mark detections directly on your source frames.
The author designed the extension for Wan 2.2 Animate. Compatibility with other Wan versions is not guaranteed.
How ComfyUI-WanAnimatePreprocessV2 Works
Think of the extension as making an annotated motion guide from your footage. It does not generate the finished animation by itself; it prepares information for the animation workflow.
- Find the person. A YOLO detection model locates the person in each frame.
- Mark body and facial positions. A whole-body ViTPose model estimates 133 keypoints across the body, hands, and face. A keypoint is a location such as an elbow, fingertip, or mouth corner.
- Keep useful detections. Confidence thresholds determine which person detections and keypoints are reliable enough to use.
- Make the face framing consistent. The extension finds the face region, can smooth its movement between frames, and produces a 512 × 512 crop from the original image.
- Estimate eye movement. It looks inside the detected eye regions for pupil positions and derives gaze direction. Optional smoothing reduces small frame-to-frame jumps.
- Show you the result. A debug image overlays landmarks, eye markers, gaze arrows, and detection boxes on the frame. If your workflow needs a skeleton image, Draw ViT Pose (V2) renders one from the pose data.
Contrast enhancement and blur can help the detection models read difficult footage. They do not blur the face crops: those are taken from the original frames to preserve facial detail.
A typical node chain is:
ONNX Detection Model Loader (V2)
│
▼
Pose and Face Detection (V2) ──► face crops / iris data / debug preview
│
▼
Optional pose editors
│
▼
Draw ViT Pose (V2) ──► skeleton images for Wan Animate
For a guide to which nodes you need and how to connect them, see NODE_GUIDE.md.
ComfyUI-WanAnimatePreprocessV2 Features
Person and whole-body pose detection
Pose and Face Detection (V2) accepts frames and produces body, hand, and face pose data. Two controls help balance missing detections against unwanted ones:
| Setting | What it changes | Practical starting point |
|---|---|---|
detection_threshold | How confident the person detector must be before accepting a person. | Start at 0.05. Lower it if the person disappears in some frames; raise it if background objects are being accepted. |
pose_threshold | Which detected keypoints are retained as visible. | Start at 0.3. Lower it toward 0.1–0.2 if obscured limbs keep disappearing. |
A low threshold preserves more uncertain information; a high threshold gives cleaner but potentially incomplete detections. For example, a dancer partly outside the frame may need more lenient settings than someone standing in a clear studio shot.
Help for noisy or difficult lighting
use_claheimproves local contrast for pose detection. Leave it on for low-light footage or when clothing and background have similar brightness.use_blur_for_posesmooths fine texture and compression noise before person and pose detection. It is on by default.blur_radiuscontrols how large an area is smoothed;blur_sigmacontrols the blur’s spread. The defaults,5and2.0, are a starting point for ordinary video. If a small person’s hands lose detail, try less blur; if grainy footage produces unstable detections, try more.
These adjustments are for analysis, not for softening your final face crops.
Stable face crops
A face crop is the square image passed along for expression-focused animation. Several controls affect how it behaves across a clip:
use_face_smoothingreduces side-to-side crop jitter while allowing the crop to follow movement. Its default is on.face_smoothing_strengthsets the amount of smoothing. Start at0.6; use less for fast action and more for a nearly still talking head.use_constant_face_boxkeeps the crop region at a fixed size instead of changing its zoom from frame to frame. This is on by default.face_box_size_pxcontrols the region selected before it is resized to 512 × 512. Start at224. Increase it if the forehead or chin is cut off; decrease it if the crop contains too much background.
For example, increasing face_box_size_px from 224 to 320 shows more of a close-up subject’s head. Decreasing it makes a small face fill more of the crop, but enlarging a tiny source region can look blurry. A fixed-size box is useful when the subject stays at a similar distance from the camera; a variable-size box may suit footage where they move much closer or farther away.
Iris and gaze tracking
The detector estimates each pupil’s position from the eye image and produces gaze direction data. Its debug overlay shows magenta pupil markers and orange gaze arrows, making it easier to judge whether the result follows the footage.
use_iris_smoothingreduces small jumps between frames; it is on by default.iris_smoothing_strengthdefaults to0.4. Increase it if the pupil marker jitters while the subject looks steadily ahead. Lower it if quick eye movements are being softened too much.
Eye tracking depends on visible, correctly detected eyes. It is less reliable when the face is very small, obscured, or poorly lit.
Skeleton drawing and optional edits
Draw ViT Pose (V2) converts pose data into coloured skeleton images. You can set its output width and height to match the resolution you intend to generate.
pose_draw_thresholdcontrols which joints appear in the image. If a joint is present in the detection but missing from the skeleton, try lowering this setting.retarget_paddingadds space around the figure. Increase it if limbs touch the edge.body_stick_widthandhand_stick_widthchange line thickness.-1chooses an automatic width;0omits that part of the drawing.draw_headincludes or omits the head representation.
The Face/Pose Delta Editor is an optional node between detection and drawing. It lets you move selected facial landmarks—such as an eyebrow, eye corner, or mouth corner—on chosen frames. Edits are stored relative to the face so they can follow its movement through the clip. See the face landmark editing guide.
For every node’s inputs, defaults, valid ranges, outputs, and ComfyUI tooltip descriptions, use NODE_REFERENCE.md.
Outputs you can inspect
| Output | What it is useful for |
|---|---|
pose_data | Pose information for drawing or downstream animation nodes. |
face_images | One 512 × 512 face crop per frame. Preview these to check framing. |
key_frame_body_points | Key body points from the first frame, provided as text data. |
bboxes | The person detection box from the first frame. |
face_bboxes | Face crop regions across the frames. |
iris_data | Per-frame pupil positions, confidence values, and gaze vectors. |
debug_image | An annotated view of the detections; connect it to a preview to diagnose problems. |
ComfyUI-WanAnimatePreprocessV2 Models
The main detection chain uses one person detector and one whole-body pose estimator, selected in ONNX Detection Model Loader (V2). The loader can run them through an NVIDIA GPU using CUDAExecutionProvider, or through the CPU using CPUExecutionProvider; CPU processing is substantially slower.
| Model | Role | When to choose it |
|---|---|---|
| YOLOv10m ONNX | Finds the person in each frame. | The recommended person detector in the original documentation. |
| ViTPose-Large whole-body ONNX | Estimates body, hand, and face keypoints. | Choose when faster processing matters. The listed model size is approximately 350 MB. |
| ViTPose-Huge whole-body ONNX | Estimates the same kinds of keypoints with a larger model. | Try when you want the more accurate option and can accommodate a model of approximately 1 GB. |
Use a whole-body ViTPose model: a body-only model will not provide the facial and hand landmarks this workflow expects. ViTPose-Huge is split into a .onnx file and a companion .bin data file; both must be together for it to load.
The wider node pack also includes features that can use other models, such as MediaPipe face landmarks, L2CS gaze weights, MiDaS depth, NLF 3D pose refinement, and optional face super-resolution models. These are not all required for the basic YOLO → ViTPose → Draw ViT Pose chain. If you use a feature that needs one of them, consult its node reference. Some nodes offer an allow_download switch, which is off by default; with it off, a missing optional model is reported rather than fetched.
What's New with ComfyUI-WanAnimatePreprocessV2
V2 compared with the original preprocessing approach
V2 adds contrast enhancement and detection-only blur to improve pose stability, along with a constant-size face box and frame-to-frame smoothing to reduce crop jitter. It also adds image-based iris and gaze estimation, iris smoothing, and a debug image that lets you inspect the results instead of guessing why an animation input looks wrong.
Dated changes noted by the author
- November 4, 2026 — Retargeting and FLUX-related fixes: The
use_fluxsetting is passed through, reference imagery reaches FLUX correctly, and edited results are re-detected. Failures in reference detection, FLUX, or retargeting now stop with a reason instead of silently using an unretargeted pose. Whenhires_imagesis supplied, it must contain the same number of frames asimages. FLUX requires its separate model weights and a GPU; the preprocessing node does not run FLUX by itself. - November 5, 2026 — Controls restored: The face super-resolution model selector lists available upscale models while remaining compatible with older saved workflow values. Controls for
jitterless,auto, andactioncrop modes are visible again. - November 6, 2026 — Optional model downloads made explicit: Nodes that use certain optional models expose
allow_download, off by default. If an optional model is absent, the log or error identifies the missing file and its expected location rather than downloading it without permission. - November 12, 2026 — More resilient downloads: MediaPipe face landmarker and NLF model downloads can resume after an interrupted transfer. Temporary network failures are retried, and an incomplete file is not presented as a finished model.
The original documentation also records fixes without a separate version number. These include correctly applying NLF pose refinement when there is no retarget, correcting facial landmark edits, drawing iris data in Iris ControlNet, measuring saccades from unsmoothed gaze, and keeping an unretargeted skeleton aligned when source and target image sizes differ. For artists, these fixes make previews and editing controls more dependable.
Troubleshooting ComfyUI-WanAnimatePreprocessV2
The quickest first step is to preview both debug_image and face_images. The debug image shows where the extension thinks the person, facial landmarks, and pupils are; the face images show what the animation workflow will receive as face crops.
The person disappears in some frames
- Check the debug image to see whether the body detection box disappears.
- Lower
detection_thresholdfrom0.05toward0.02or0.01. - Keep detection blur on; try CLAHE if lighting is difficult.
- If the person is tiny or mostly outside the frame, try footage with a larger, more visible subject. Lowering the threshold cannot recover detail that is not present.
Limbs or hands vanish from the skeleton
- Lower
pose_thresholdto around0.15–0.2to retain less-certain keypoints. - Lower Draw ViT Pose (V2)’s separate
pose_draw_thresholdto around0.05–0.1if points are detected but not drawn. - Check whether hands were deliberately hidden with
hand_stick_width: 0. - Inspect the source frame for occlusion or motion blur.
The face crop jitters or cuts off the head
- Turn on
use_face_smoothinganduse_constant_face_box. - For jitter, begin with
face_smoothing_strength: 0.6; increase it if a mostly stationary face still jumps. - For a clipped forehead or chin, increase
face_box_size_pxin steps, such as224→256→320. - Preview
face_imagesagain. If the subject moves quickly and the crop seems slow to follow, reduce the smoothing strength.
The iris marker or gaze arrow looks wrong
- In
debug_image, check the green eye landmarks first. If they do not outline the eyes, pupil detection has a poor starting point. - Check whether the eyes are large and clear enough in the source image. Distant faces provide little pupil detail.
- Keep CLAHE on for difficult lighting.
- If the marker is in roughly the right place but jitters, turn on iris smoothing or increase its strength slightly. Smoothing will not fix a consistently misplaced marker.
A model, CUDA, or retargeting error stops the workflow
- Read the complete ComfyUI error or log: missing optional models and retargeting failures are intended to report their reason.
- Check that the loader selects a whole-body ViTPose model and a YOLO ONNX model. For ViTPose-Huge, check that its companion
.binfile is alongside the.onnxfile. - If GPU execution fails, select
CPUExecutionProviderto test whether the workflow runs on CPU. CPU processing will be slower. - If using
retarget_image, resolve the reported reference, FLUX, or retargeting problem rather than assuming the workflow continued with a retargeted pose. - If using
hires_images, provide the same number of frames as inimages.
Frequently asked questions
Do I need Draw ViT Pose (V2)?
Use it when your Wan Animate workflow needs skeleton images. The model loader and detection node produce the underlying pose and face outputs; the drawing node turns pose data into images.
What is the difference between pose_threshold and pose_draw_threshold?
pose_threshold filters keypoint confidence during detection. pose_draw_threshold controls which available keypoints appear in the rendered skeleton. If you filtered a keypoint out during detection, lowering only the drawing threshold will not restore it.
Why are the face crops 512 × 512 even when I change face_box_size_px?
face_box_size_px changes how much of the source frame is selected—effectively the zoom. The selected region is then resized to the fixed 512 × 512 output.
Can I use a single image instead of a video?
Yes. Frame-to-frame face and iris smoothing offers no benefit when there is only one frame.
Which width and height should I choose?
Match the resolution you plan to use for generation, especially for skeleton images. The original documentation gives 832 × 480, 1280 × 720, and 512 × 512 as examples.
Will the extension automatically fetch every model it needs?
No. For optional features with allow_download, downloads are off by default. A missing model is reported so you can decide whether to enable its download.
Learn More about ComfyUI-WanAnimatePreprocessV2
- NODE_GUIDE.md: Start here to decide which nodes you need and how to connect the main detector → optional editors → pose drawing → Wan Animate chain.
- NODE_REFERENCE.md: Look up any node control, its accepted values, default, output, and matching ComfyUI tooltip.
- Face/Pose Delta Editor guide: Learn how to adjust individual facial landmarks in selected frames.
- Model pages: Explore the linked Wan YOLOv10m, ViTPose-Large whole-body, and ViTPose-Huge files when choosing between the documented detection models.
