Run InstantHMR Prediction:
InstantHMRPredict, displayed as “Run InstantHMR Prediction,” estimates a three-dimensional human body from images using the InstantHMR model and the MHR body representation. It reconstructs body geometry and estimates pose, including body orientation, hands, shape, and camera-relative placement, so you can use photographed or filmed people in workflows that support MHR reconstruction data. The node prepares a square crop around each person, expands it to include surrounding context, and processes it at the model’s 288 × 288 resolution. It also accounts for the person’s position within the original image and an assumed or specified camera field of view, helping relate the reconstruction to the original scene. It can process individual images, sequences of frames, and multiple people, with optional tracking information to guide person selection. Predictions are made from individual person crops rather than through a temporal smoothing stage, so processing video does not by itself guarantee smooth motion. The underlying body rig intentionally omits MHR pose correctives because InstantHMR was trained without them.
Run InstantHMR Prediction Input Parameters:
instanthmr_model
A required connection of type INSTANTHMR_MODEL supplying the loaded InstantHMR model, normally provided by InstantHMRLoader. This model performs the reconstruction and provides the MHR body rig and mesh information used to organize the results. The node asks ComfyUI’s model manager to load it onto its configured execution device before prediction, with a memory estimate based on the selected crop batch size. Connect a compatible InstantHMR model rather than a general image-generation checkpoint or another body-estimation model, since the node expects InstantHMR-specific model methods and prediction fields. No default value, numeric range, or selectable options are defined for this connection.
image
A required IMAGE connection containing one image or a batch of images, such as video frames. Internally, the node reads this as a tensor arranged as batch, height, width, and channels, and processes each image as a frame. Person regions are cropped from these images, resized to 288 × 288, and normalized using the model’s expected ImageNet color statistics. Use standard ComfyUI RGB images; the normalization assumes three color channels. If no usable tracking boxes or detection boxes are supplied, the node uses the entire image as one candidate region for each frame, rather than automatically detecting everyone in the scene. Images within the tensor share the same height and width, and these dimensions are recorded with the result. No default value or numeric range is defined.
fov
A required FLOAT parameter specifying the vertical camera field of view in degrees, with a default of 0.0, minimum of 0.0, maximum of 180.0, and adjustment step of 0.1. It is marked as an advanced setting. For a positive value, the node calculates focal length in pixels as image height divided by twice the tangent of half the vertical field of view. This camera assumption affects estimated depth and absolute scale, so it is important when you want the reconstruction to match a known camera. A value of 0.0 means “use the default camera assumption,” not a literal zero-degree camera: focal length is then set to 1.05 times the image diagonal, matching the documented training default. If your camera is unknown, start with 0.0; if it is known, supply its vertical field of view rather than its horizontal field of view. Although 180.0 is permitted by the interface, values at or extremely near that limit produce a nearly zero focal length and are unsuitable for dependable camera calculations.
batch_size
A required INT parameter controlling how many person crops are processed in one model forward pass, with a default of 64, minimum of 1, and maximum of 1024. It is marked as an advanced setting. This is a crop count, not strictly an image or frame count: one frame containing five selected people contributes five prediction jobs. Larger values let the model process more crops together and may improve throughput when your execution device has sufficient memory, while smaller values reduce the working memory requirement. The node also uses this value when requesting model memory from ComfyUI’s model manager. Changing it controls how work is grouped; it does not change the model’s fixed crop resolution or intentionally act as a reconstruction-quality setting. If you encounter memory problems, reduce it progressively, for example from 64 to 16, 4, or 1.
track_data
An optional SAM3_TRACK_DATA connection for person tracks from SAM3 Video Track, intended especially for multi-person video. The node converts this information into per-frame person boxes and masks, removes absent track entries, and uses the remaining boxes to select regions for reconstruction. When masks are available, they are attached to person results and are also used in a final filtering stage to reject stray tracks, such as isolated limbs or blobs that do not sufficiently agree with the predicted body keypoints. The masks guide selection and filtering; the crop preparation code does not use them to remove the background from the image before prediction. Usable tracking-derived boxes take priority over bboxes. If tracking does not produce a usable box collection, the node tries bboxes next. No widget default, numeric range, or selectable options are defined; when disconnected, its execution value is None.
bboxes
An optional BOUNDING_BOX connection providing person detection boxes, for example from RT-DETR Detect configured for the person class. It is a connection-only input rather than a manually editable box widget. The node converts the supplied detections into per-frame boxes and predicts a body for each selected region, so correct person selection and frame alignment directly affect what gets reconstructed. Internally, boxes are handled as left, top, right, and bottom coordinates in the original image. Each region is expanded into a square crop with a side length of 1.2 times the box’s longer dimension. This input is used only when tracking has not supplied a usable box collection; supplying both inputs does not combine their boxes. If neither source supplies a box collection, the whole image becomes the fallback region. No numeric range or widget default is defined; when disconnected, its execution value is None.
Run InstantHMR Prediction Output Parameters:
mhr_pose_data
The sole output is an MHR_POSE_DATA object containing structured reconstruction data rather than a rendered image or a standalone exported character file. Its frames field contains one list per input frame, with each list holding that frame’s retained person predictions; a frame can contain multiple people or no retained predictions. Each person record includes the original bounding box, focal length in pixels, predicted 3D body keypoints, 2D keypoints mapped back into original-image coordinates, predicted mesh vertices, camera translation, global rotation, body and hand pose parameters, scale and shape parameters, facial-expression parameters, predicted joint coordinates and joint global rotations, and the combined MHR model parameters. A tracking mask is included when available, otherwise it is None. The separate pred_face_keypoints_3d and pred_face_keypoints_2d fields are explicitly set to None, so their presence should not be interpreted as available facial landmark predictions. At the top level, faces supplies mesh triangle connectivity, image_size stores the original dimensions as height followed by width, canonical_colors supplies model-derived canonical color information, and hand_vert_mask identifies hand vertices. Together, these fields allow compatible downstream nodes to interpret the body mesh, pose, and relationship to the source image; the estimates should not be treated as measured physical dimensions simply because camera and scale values are present.
Run InstantHMR Prediction Usage Tips:
- For a single person already filling most of an image, start without
track_dataorbboxes. The full-image fallback is convenient, but it represents one candidate region per frame and is not a replacement for multi-person detection. - For multiple people in a still image, supply person detections through
bboxes. Use detections from the same image and avoid passing boxes for unrelated object classes, since every selected region is sent to the human reconstruction model. - For multi-person video, supply SAM3 Video Track data through
track_dataand keep it aligned with the image sequence’s frame count, ordering, and resolution. Do not expectbboxesto add extra people when valid tracking boxes are already being used. - Leave
fovat0.0when you do not know the camera calibration. If you know the camera’s vertical field of view, enter that value to provide a better-informed depth and scale assumption; do not substitute horizontal field of view without converting it appropriately. - Start with
batch_sizeset to64if your hardware can accommodate it. Reduce the value when memory is limited, or cautiously increase it for large multi-person workloads while monitoring memory use; larger batches do not increase crop detail. - Use boxes that cover as much of the visible person as possible rather than just the face or torso. The automatic
1.2crop expansion adds context, but it cannot restore limbs that are missing from the source image or compensate reliably for a badly placed box. - Check tracking masks if expected people disappear from the result. Mask-based filtering can remove tracks judged inconsistent with the predicted body keypoints, so improving the person track can be more useful than changing inference batch size.
- Plan for a compatible visualization or export stage after prediction. This node returns reconstruction data, not an overlay preview, finished animation, or exported FBX file, and it does not perform an explicit temporal smoothing pass.
Run InstantHMR Prediction Common Errors and Solutions:
Missing required input: instanthmr_model or image
- Explanation: The node requires both a compatible loaded model and an image connection. ComfyUI may report a missing-input validation error before execution; exact wording depends on the installed version.
- Solution: Connect the InstantHMRLoader model output to
instanthmr_modeland a standard ComfyUI image output toimage. Keepfovandbatch_sizewithin their permitted ranges.
CUDA out of memory
- Explanation: Loading the model or processing the requested number of person crops can exceed available GPU memory. Multiple selected people increase the total work, while
batch_sizecontrols how many crops are processed together. - Solution: Lower
batch_size, for example to16,4, or1, and free memory used by other workflows or applications. If the model itself cannot fit, lowering the batch size alone may not resolve the problem.
ValueError: not enough values to unpack / too many values to unpack
- Explanation: A possible cause is an image tensor with the wrong number of dimensions. The node expects four dimensions: batch, height, width, and channels. These are possible Python runtime messages, not custom error strings defined by this node.
- Solution: Supply images through a standard ComfyUI
IMAGEconnection. If a custom conversion node produced the data, ensure that it preserves the batch dimension and the expected image layout.
RuntimeError: The size of tensor a must match the size of tensor b
- Explanation: A possible cause is an image with a channel layout incompatible with the three-channel RGB normalization. Incompatible shapes elsewhere in custom image or tracking data can also produce this general PyTorch error.
- Solution: Convert grayscale or RGBA content to a standard RGB image before prediction. If the error persists, check the traceback for the failing operation and verify that tracking data matches the supplied frames and image dimensions.
IndexError: index out of bounds
- Explanation: Inconsistent custom tracking or detection data can lead to invalid frame or person indexing. The node processes per-frame boxes and, when available, corresponding masks, so these collections must agree. This is a possible runtime symptom rather than an explicit error raised in the node source.
- Solution: Regenerate tracks or detections from the exact image sequence being processed. Avoid reusing tracking data after changing frame order, frame count, cropping, or resolution unless the data has been updated to match.
AttributeError or KeyError involving model methods or prediction fields
- Explanation: The node expects an InstantHMR-specific model with methods such as
memory_used_forwardand prediction fields such aspred_vertices. An incompatible model object or mismatched extension files may fail when these are accessed. - Solution: Use the model supplied by the compatible InstantHMRLoader. If the problem began after an update, make sure the loader, prediction node, and model implementation come from a consistent extension installation, then restart ComfyUI and reload the model.
No people returned for some frames
- Explanation: This can occur without an exception. A frame may have no selected boxes, absent tracks may be removed, or mask-based stray-track filtering may discard predictions. The full-image fallback applies when no box collection is available, not automatically whenever an individual frame has an empty box list.
- Solution: Inspect detections and tracking masks for the affected frames. Test a single frame with both optional inputs disconnected to check the full-image path, then reconnect person boxes or corrected tracking data.
Reconstruction has implausible depth or scale
- Explanation: An incorrect camera assumption can distort estimated placement and scale. A positive
fovis interpreted as vertical field of view, and values near180.0lead to a nearly zero focal length. This is a result-quality issue and may not produce an error message. - Solution: Reset
fovto0.0to use the training-default camera assumption, or enter a reliable vertical field of view for the source camera. Also verify that the person box covers the intended subject.
Multiple people produce only one reconstruction, or the wrong person is reconstructed
- Explanation: Without optional person boxes or usable tracks, the entire image is treated as one candidate region. The node does not run a separate person detector to discover all people in that fallback region.
- Solution: Supply person detections through
bboxesfor still-image workflows, or suitable SAM3 person tracks throughtrack_datafor video. If both are connected, remember that usable tracking boxes take priority.
