Pose and Face Detection:
PoseAndFaceDetection analyzes a sequence of images to locate a person, estimate body and facial landmarks, and prepare face crops for later steps in an animation workflow. It uses a person detector to guide pose estimation, then derives a face region from the estimated facial landmarks. When a reference is supplied, it can retarget the estimated poses using that reference. This brings related pose and face preparation into one step while retaining the original pose estimates alongside any retargeted results.
Pose and Face Detection Input Parameters:
model
A required POSEMODEL containing the yolo person detector and vitpose pose estimator used by the node. The detector first finds a person in each image; the pose estimator then examines a crop based on that detection. The node expects both model components to be present and usable. No default, minimum, or maximum is defined for this input.
images
A required IMAGE batch to analyze. The node processes each image as a frame, estimating its pose and producing a corresponding face crop and face box. Image dimensions also determine the coordinate space of the returned face boxes. Supply at least one frame; an empty batch cannot produce the combined pose results. No default, minimum, or maximum is defined for this input.
width
A required integer described in the interface as the width of the generation. Its default is 832, minimum is 64, maximum is 2048, and step is 1. In the shown implementation, this value is used with height to set the target area when resizing an optional reference image. It does not resize the main images batch.
height
A required integer described in the interface as the height of the generation. Its default is 480, minimum is 64, maximum is 2048, and step is 1. Together with width, it determines the target area used to resize an optional reference image. It does not change the dimensions of the main images batch.
retarget_image
An optional IMAGE reference for pose retargeting; its default is None. When supplied, the node uses the first image in this input to estimate a reference pose and retarget the detected sequence. When omitted, it converts the detected poses without reference-based retargeting. No minimum or maximum is defined. Choose a reference in which the intended person can be seen clearly, since the reference pose must be estimated from that image.
face_padding
An optional integer controlling how many pixels the node adds around each detected face region before cropping, with the expanded region kept within the source image. Its default is 0, minimum is 0, maximum is 512, and step is 1. A larger value includes more surrounding context, such as hair or the edges of the head. Every resulting face crop is resized to 512 × 512 regardless of this setting; a value of 0 means no additional padding, not no resizing.
Pose and Face Detection Output Parameters:
pose_data
A POSEDATA bundle containing the poses prepared for subsequent processing, the original detected pose metadata, and reference-related data when a retarget_image was supplied. Without a reference, its prepared poses come from converting the detected pose metadata; with a reference, they come from the retargeting step. The bundle also holds the resized reference image and its estimated pose when applicable, or None for those entries when no reference was supplied.
face_images
An IMAGE batch with one face crop per input frame, in frame order. Each crop comes from the estimated facial landmarks and is resized to 512 × 512. If a calculated crop is empty, the node attempts a fallback crop; if that is also empty, it creates a black image. Check these images before using them downstream, because a fallback may not show the intended face.
key_frame_body_points
A STRING containing JSON-formatted body-point coordinates, such as [`{"x": 120, "y": 85}`]. The values are integer pixel coordinates derived from selected body landmarks in the original pose estimates. Although the node visits several key-frame positions, the returned string contains only the points assembled for the last position visited; it is not a list of points for every sampled frame. The number of points can vary if a selected landmark is unavailable.
bboxes
A BBOX output containing a one-item list with the first input frame’s detector box as four integers. This is the original detection result, not a box for every frame. If that result has fewer than four values, the node returns (0, 0, 0, 0) in the list instead. The implementation does not replace this output with the full-image box it may use internally when a detection is invalid.
face_bboxes
A BBOX, output containing one face box per input frame, in frame order. Each box is a four-value tuple in the order (x1, y1, x2, y2), identifying the left, top, right, and bottom edges of the face region used for cropping. The boxes include any requested face_padding and are bounded by the source image dimensions. If a crop proves empty and the node uses a fallback image, the recorded box still describes the originally calculated region.
Pose and Face Detection Usage Tips:
- Start with clear frames showing one prominent person. The node uses a single detector box per frame to guide pose estimation; when that box is missing or invalid, it estimates the pose from a crop based on the full image.
- Set
widthandheightto your intended generation dimensions when usingretarget_image, since their product controls the reference image’s resize target. Changing them does not resize the source frames or the returned face crops. - Begin with
face_paddingat0and increase it when the face crops cut off useful context. Inspectface_imagesafter changing it: additional padding changes what is included before each crop is resized to512 × 512. - If you need reference-based retargeting, put the intended reference first in the
retarget_imagebatch. The implementation reads only its first image. - Keep the outputs’ scopes distinct:
face_imagesandface_bboxescover the input frames, whilebboxescovers only the first frame andkey_frame_body_pointsdescribes only the last sampled key-frame position.
Pose and Face Detection Common Errors and Solutions:
KeyError: 'yolo' or KeyError: 'vitpose'
- Explanation: The supplied
modeldoes not contain one of the two components the node reads by name. - Solution: Connect a compatible
POSEMODELthat provides both the person detector and pose estimator; check the upstream model-loading node.
ValueError: need at least one array to concatenate
- Explanation: An empty
imagesbatch leaves the node with no estimated keypoints to combine. - Solution: Supply at least one image and confirm that an upstream frame-loading or filtering step has not produced an empty batch.
IndexError: list index out of range or an indexing error involving frame 0
- Explanation: The node accesses first-frame results, including the first detector box, so it cannot complete normally without a frame.
- Solution: Check that
imagescontains at least one valid image before running the node.
Empty face crop on frame ... , creating fallback image.
- Explanation: This is a warning from the node, not necessarily a stopping error. The face box produced an empty crop, so the node tries a fallback crop and, if needed, a black image.
- Solution: Inspect that frame and its
face_imagesresult. Use a clearer image with a visible face if the fallback is unsuitable; increasingface_paddingmay help when the calculated region is merely too tight.
An error at images.shape or images.numpy()
- Explanation: The node expects
imagesto be a ComfyUI image batch with batch, height, width, and channel dimensions that it can convert for processing. - Solution: Connect an
IMAGEoutput rather than another data type, and check that the upstream node produces a nonempty, valid image batch.
An error while processing retarget_image[0]
- Explanation: When a reference is connected, the node immediately reads its first image. An empty or incompatible reference batch cannot be used.
- Solution: Supply a valid
IMAGEbatch containing at least one reference image, or disconnectretarget_imageto run without retargeting.
IndexError while reading keypoints_body
- Explanation: The node requests specific body-landmark positions when building
key_frame_body_points. A pose result without those positions cannot supply all the requested values. - Solution: Confirm that the connected pose model is compatible with this node. Also try clearer frames in which the person’s body is visible.
cv2.error during image resizing
- Explanation: OpenCV could not resize an image or crop supplied during processing; an empty or invalid image region is one possible cause.
- Solution: Check the input and optional reference images for valid dimensions, then identify whether a particular frame triggers the error. Replacing a corrupt or empty frame is a practical first step.
