Save 4 hours! We auto-setup your workflow! Free!

Drop your workflow.json — we handle every dependency, custom node, and model. Just open the link and run.

Auto-Setup Workflow Json (Free) Now!
ComfyUI > Nodes > ComfyUI-InstantHMR

ComfyUI Extension: ComfyUI-InstantHMR

Repo Name

ComfyUI-InstantHMR

Author
jtydhr88 (Account age: 5623 days)
Nodes
View all nodes(3)
Latest Updated
2026-11-12
Github Stars
0.06K

How to Install ComfyUI-InstantHMR

Install this extension via the ComfyUI Manager by searching for ComfyUI-InstantHMR
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter ComfyUI-InstantHMR in the search bar
After installation, click the Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • 16GB VRAM to 80GB VRAM GPU machines
  • 400+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 200+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

ComfyUI-InstantHMR Description

ComfyUI-InstantHMR reconstructs 3D human poses, body shapes, hands, and meshes from photos, videos, or live cameras, supporting motion capture, pose rendering, character control, and animation export.

ComfyUI-InstantHMR Introduction

ComfyUI-InstantHMR brings InstantHMR human body reconstruction into ComfyUI. It estimates a person’s 3D pose, body shape, hands, and full body mesh from a photograph, video, or live camera feed.

For each person, it produces:

  • A 3D pose: how the body and limbs are positioned.
  • Body shape: an estimate of the person’s proportions.
  • 70 keypoints in 2D and 3D: reference points that describe the body and hands.
  • An MHR body mesh: a three-dimensional body surface built with Meta’s Momentum Human Rig.

The extension uses the same MHR body representation as ComfyUI’s native SAM 3D Body nodes. This means you can pass its results into existing nodes for smoothing motion, rendering pose references, and exporting animations.

InstantHMR reconstructing the bodies of four dancers

What can you use it for?

  • Create pose references from photographs. Turn a person’s pose into a mesh, silhouette, or OpenPose-style rendering.
  • Recover motion from video. Estimate body movement across frames, smooth the results, and export an animation.
  • Capture your own performance. Use a webcam or phone camera to preview reconstructed motion and record a clip.
  • Preview motion on a character. Drive a supported rigged character in the live 3D view.
  • Send live motion to another application. Stream the first detected person’s pose to an application that receives the VMC protocol.

InstantHMR is a compact model family, with approximately 20–32 million parameters. Its body-prediction stage is fast, but your overall workflow speed also depends on person detection, tracking, rendering, and hardware.

How ComfyUI-InstantHMR Works

Think of the workflow as finding a person, fitting a digital mannequin to them, and then choosing how to display or reuse that mannequin.

1. Identify each person

The prediction node needs to know which part of the image belongs to each person. It can use:

  • RT-DETR Detect, with the person class, to provide rectangular person boxes.
  • SAM3 Video Track to follow people through a video.
  • The entire frame, if neither boxes nor tracks are connected. In this case, the frame is treated as one person.

For a group photograph, boxes let the model process each person separately. For a video with several people, tracking also helps maintain a consistent identity for each person across frames.

2. Estimate the body

InstantHMR examines each person’s image region and predicts their pose, proportions, and keypoints.

The MHR body model then provides the corresponding 3D body. You can think of MHR as an adjustable mannequin: its joints describe the pose, while its shape settings describe body proportions.

This is an estimate from an image—not a direct measurement of the person. The author notes that full-body images generally align more closely than close-ups that cut off much of the body.

3. Account for the camera

The fov setting describes the camera’s vertical field of view, in degrees. This helps relate the 3D reconstruction to the camera image.

You can:

  • Connect MoGe through MoGe Geometry To FOV to estimate the camera field of view for a video.
  • Leave fov at 0 to use the documented fallback: a focal length equal to 1.05 times the image diagonal.

Field of view is a camera setting, not a pose-strength control. It affects how the reconstructed body is placed and projected into the image.

4. Render, smooth, or export the result

Run InstantHMR Prediction outputs MHR_POSE_DATA, a structured body-pose result rather than a finished image.

Compatible native ComfyUI nodes can then:

  • Reduce frame-to-frame jitter.
  • Render meshes, silhouettes, and pose skeletons.
  • Create supported 3D animation files.

A typical video pipeline is:

Video frames → person tracking → InstantHMR prediction → smoothing → rendering or animation export

ComfyUI-InstantHMR Features

Main nodes

NodePurposeWhen to use it
Load InstantHMR ModelLoads an InstantHMR checkpoint from models/detection.Select the model used for image or video prediction.
Run InstantHMR PredictionPredicts one MHR body per person box and outputs MHR_POSE_DATA.Process photographs or video frames.
InstantHMR Live CaptureProvides live camera and 3D previews, character driving, VMC streaming, and recording. Outputs the selected recorded clip as VIDEO when the workflow runs.Capture or preview a live performance.

Rendering and animation export

The prediction output works with these native nodes:

Native nodeWhat it adds
Smooth SAM3D Body Pose DataReduces jitter in reconstructed video motion.
Render 3D Body PoseRenders meshes, silhouettes, OpenPose 2D/3D, and SCAIL representations.
Create 3D Animation FileCreates animated GLB files in the openpose and scail styles. Other supported export options require an additional rig input, as explained below.

Important export requirement: the body_mesh and bones_only styles, as well as BVH export, read the MHR rig through a sam3d_body_model input. Connect a SAM 3D Body loader when using those options.

The InstantHMR checkpoint contains the rig needed for its own prediction, but those native export paths still require their documented SAM 3D Body model input.

Example ComfyUI workflow using InstantHMR

Multi-person video tracking

For videos containing several people, the author recommends using SAM3 tracks.

Smoothing relies on each person keeping the same index across frames. Tracking helps prevent the motion of one person from being mixed with another person’s motion.

Additional behavior and controls:

  • Tracks whose predicted body falls mostly outside their own SAM3 mask are dropped. This can reject false tracks, such as an isolated raised arm detected as a separate person.
  • If you know how many people are in the clip, set max_objects in Run SAM3 Video Track to that number.

Live camera preview

InstantHMR Live Capture runs RT-DETR and InstantHMR on a camera feed.

The preview shows:

  • Left: the camera image with 2D skeletons.
  • Right: reconstructed 3D bodies. Drag in this view to orbit around them.

Choose the InstantHMR model, RT-DETR detector, and camera, then press Start. These selections are saved with the workflow.

Live capture with camera and 3D body previews

Camera choices

  • This device: uses the webcam connected to the browser displaying ComfyUI. Both previews are mirrored, like a video call. Recordings and VMC output preserve the real left and right.
  • Phone / other device: displays a QR code. Scan it with a phone on the same network. The phone streams its camera and displays skeletons, while the ComfyUI node displays the 3D view.

The phone acts as the camera source; ComfyUI performs the processing on the computer running it.

Recording a performance

  1. Start the live preview.
  2. Press ● Record to record incoming camera frames.
  3. Recordings are stored in input/instanthmr_live/.
  4. The new clip becomes selected in the recording list.
  5. Run the workflow to output that clip as VIDEO.

To reconstruct the recorded clip, use:

InstantHMR Live Capture → Get Video Components → Run InstantHMR Prediction

The included live workflow demonstrates the rest of this pipeline.

Live smoothing and performance controls

SettingWhat it doesPractical effect
smoothingSteadies the pose between frames.Higher values reduce movement while you hold still, but add some lag to slow movements. 0 shows the raw per-frame result.
detect_everyRuns person detection every N frames and reuses the boxes between detections.Values of 2–3 can improve preview smoothness because detection is the slowest step.
fovSets the vertical camera field of view.Uses the same interpretation as the prediction node.

The live 3D view, driven character, and VMC output all use the smoothed pose.

For example, a held pose may benefit from more smoothing. If you are evaluating the model’s immediate response, setting smoothing to 0 lets you inspect the unfiltered result.

Driving a rigged character

The character selector lists supported rigged character files found in the ComfyUI input folder, including subfolders such as input/3d/.

Supported formats are:

  • .fbx
  • .glb
  • .gltf
  • .vrm

Selecting a character replaces the MHR body in the live 3D preview with that character, with one copy per detected person.

The extension matches bones:

  • By name for Mixamo, Unreal, and similar rigs.
  • Through the humanoid table for VRM characters.

Both T-pose and A-pose characters are supported. The character is scaled to match the performer’s leg length.

Streaming motion with VMC

The vmc field sends the first person’s humanoid pose to an application that supports the VMC protocol.

Examples include VSeeFace, Warudo, VNyan, VMC Link for Blender, and VMC4U for Unity.

Enter a destination in this format:

host:port

For example:

127.0.0.1:39539

ComfyUI sends the stream, so the address must be reachable from the computer running ComfyUI, not necessarily the device displaying the browser. Clear the field to stop streaming.

  1. Import a T-pose character, such as a Mixamo FBX.
  2. Apply its rotation and scale using Object → Apply → All Transforms.
  3. Select the character’s armature in the VMC Link panel.
  4. Press Start Receiver.
  5. Enter the receiver’s host and port in the live node.
  6. Use Start Recording in VMC Link to key the motion onto the armature.

For Mixamo rigs, the author specifies these manual bone mappings under VMC Link’s advanced settings:

Humanoid boneMixamo name
UpperLegUpLeg
LowerLegLeg
ToesToeBase
ChestSpine1
UpperChestSpine2
FingersHandThumb1–3, HandIndex1–3, and equivalent finger chains
Little fingerHandPinky

Adding facial expressions

InstantHMR itself predicts body, hands, and shape, not facial expressions.

For facial animation:

  1. Add Face Expression to SAM3D Body after Run InstantHMR Prediction.
  2. Supply a model from Load SAM3D Body Model.

The expression node uses MediaPipe to read faces and drive MHR’s facial expression blendshapes. The expressions can then be included in the rendered mesh and GLB/BVH export.

ComfyUI-InstantHMR Models

Five checkpoints are documented for this extension.

CheckpointBackboneParameters3DPW error, mm ↓Usage restriction
instanthmr_nano.safetensorsHGNetV2-B020.0 M46.6SAM License; non-commercial only
instanthmr_tiny.safetensorsHGNetV2-B120.3 M44.0SAM License; non-commercial only
instanthmr_small.safetensorsHGNetV2-B224.4 M41.7SAM License; non-commercial only
instanthmr_medium.safetensorsHGNetV2-B432.2 M40.0SAM License; non-commercial only
instanthmr_medium_commercial.safetensorsHGNetV2-B432.2 M50.7SAM License; commercial use permitted under its terms

The error column uses PA-MPJPE, a research measure of average joint-position error after alignment. Lower is better on that benchmark; it is not a guarantee of the error in your own footage.

Choosing a model

  • Nano: useful when prioritizing speed and responsiveness.
  • Tiny: a lightweight alternative with better reported accuracy than nano.
  • Small: a middle ground between model size and reported accuracy.
  • Medium: the strongest reported baseline accuracy among the listed checkpoints.
  • Medium commercial: the option for commercial projects, subject to the SAM License. It has the same model size as medium but a higher reported benchmark error.

Smaller models may make live capture more responsive, while larger models may provide more accurate poses. However, reducing model size will not necessarily produce a large overall speed improvement if detection or tracking is the bottleneck.

Model files and compatibility

Extension-ready checkpoints are available on Hugging Face. The model loader reads them from models/detection.

Each checkpoint includes:

  • The prediction network.
  • The MHR body rig it was trained on.

No separate body model is needed for InstantHMR prediction itself. Additional native export or facial-expression nodes may still require a SAM 3D Body loader.

The upstream project also distributes ONNX models. These are a different format; the extension provides scripts/convert_onnx.py for conversion to its checkpoint format.

Performance expectations

The upstream paper reports approximately 1 ms per person on a desktop GPU and approximately 5 ms per person on a phone NPU for the model stage, depending on model size and execution settings.

These are not end-to-end ComfyUI timings. They do not include all detection, tracking, frame handling, rendering, or export work. The phone NPU results also do not mean that the extension’s phone-camera mode runs inference on the phone.

Licensing

All five checkpoints are subject to the SAM License.

The nano, tiny, small, and medium checkpoints additionally carry a non-commercial restriction because of their training images. The commercial medium model uses commercially usable training images, but remains subject to the SAM License’s conditions.

Do not assume the upstream code’s Apache-2.0 license also applies to the model weights.

What’s New with ComfyUI-InstantHMR

The supplied documentation does not include a versioned changelog for the ComfyUI extension. The dated updates below belong to the upstream InstantHMR project and should not be interpreted as confirmed new controls in the ComfyUI nodes.

DateUpstream updateWhy it matters
November 6, 2026 — v1.0First release, including four model sizes, a commercially usable medium model, ONNX graphs, demo, training code, and dataset pipeline.Established the model choices used by the extension.
November 8, 2026 — v1.1Added an in-graph kinematic fit and two video pose filters.The fit adjusts the 3D pose toward the model’s predicted 2D keypoints; the filters aim to reduce motion jitter.
November 10, 2026Released PyTorch checkpoints and configuration files for all five models.Provides a starting point for advanced users who want to fine-tune models on their own data.

The upstream v1.1 options such as --fit and --pose-filter are documented for its own tools. The provided extension documentation does not establish that these options are exposed in ComfyUI. For documented ComfyUI smoothing, use the live node’s smoothing control or Smooth SAM3D Body Pose Data.

ComfyUI-InstantHMR Example Workflows

The included workflows demonstrate three starting points:

WorkflowWhat it demonstrates
Image workflowFinds people in a photograph with RT-DETR and renders reconstructed meshes over the image.
Video workflowUses SAM3 tracks, MoGe field-of-view estimation, smoothing, a rendered video, and SCAIL GLB export.
Live-recording workflowProcesses a clip recorded with InstantHMR Live Capture through the video pipeline.

Start with the image workflow to understand prediction and rendering. Use the video workflow when consistent tracking and smoothing are important, or the live workflow to reuse your own recorded performance.

Troubleshooting ComfyUI-InstantHMR

Only one body appears in a group image

Without boxes or tracks, the prediction node treats the entire image as one person.

  1. Check whether a detection or tracking input is connected.
  2. For a photograph, use RT-DETR Detect with the person class.
  3. For a multi-person video, use SAM3 Video Track.
  4. Confirm that the detector or tracker identifies the intended people.

The mesh does not line up closely with a cropped person

InstantHMR is trained on whole people. The author notes that chest-level close-ups can retain a good pose while the mesh sits a few percent away from the person in the image.

  1. Test a full-body frame to compare alignment.
  2. Check the person boxes or tracks supplied to prediction.
  3. Review the camera fov input.
  4. For video, use MoGe through MoGe Geometry To FOV when appropriate.

Video motion jitters or people become mixed during smoothing

  1. Use SAM3 tracks so each person keeps a consistent index across frames.
  2. Set max_objects to the known number of people.
  3. Pass the result through Smooth SAM3D Body Pose Data.
  4. For live capture, increase smoothing gradually.

More live smoothing makes held poses steadier, but introduces some delay during slow movements.

The live preview is slow

  1. Try nano or tiny.
  2. Set the advanced detect_every input to 2 or 3.
  3. Compare the preview again.

Detection is the documented slowest step in live capture, so changing detector frequency may help more than changing the body model alone.

The browser cannot open the camera

Browsers allow camera access only on localhost or HTTPS.

  1. If using a webcam on the same computer, open ComfyUI through localhost.
  2. If accessing ComfyUI over the network, use an HTTPS address.
  3. For a phone, confirm that it is on the same network as the ComfyUI computer.
  4. If the HTTPS setup uses a self-signed certificate, accept its warning on both the computer and phone.

A plain HTTP LAN address does not meet the documented camera-access requirement.

The preview looks mirrored

This is expected with This device camera mode. Both previews are mirrored for a video-call-style experience.

Recordings and VMC output keep the real left and right.

VMC motion does not reach the receiving application

  1. Confirm that the receiver is running.
  2. Check that the vmc field matches its host and port.
  3. Interpret the address from the computer running ComfyUI.
  4. If the receiver runs on another computer, do not use 127.0.0.1, which refers to the ComfyUI computer itself.
  5. In Blender, check that the correct armature is selected and Start Receiver has been pressed.

Only the first detected person is streamed through VMC.

A character’s motion looks incorrect

  1. Confirm that the file contains a rigged character in a supported format.
  2. Review bone names or the VRM humanoid mapping.
  3. For the documented Blender workflow, apply the character’s transforms.
  4. For Mixamo in VMC Link, review the manual bone mappings listed above.

The documentation describes support for Mixamo, Unreal-like naming, and VRM humanoid mappings; it does not promise automatic matching for every custom rig.

body_mesh, bones_only, or BVH export needs a model input

Connect a SAM 3D Body loader to the export node’s sam3d_body_model input. These export paths obtain their MHR rig from that input.

Frequently asked questions

Does InstantHMR predict facial expressions?
No. Add Face Expression to SAM3D Body with a SAM 3D Body model to estimate expressions separately.

Can I record a live session and process it afterward?
Yes. Select the recording and run the workflow to output it as VIDEO, then extract its frames with Get Video Components.

Do I always need a separate SAM 3D Body model?
No. InstantHMR prediction includes its own required rig. Certain native export options and the facial-expression workflow require an additional SAM 3D Body model input.

Can I use the models commercially?
Only the commercial medium checkpoint permits commercial use, subject to the SAM License. The other four checkpoints are non-commercial only.

Will a larger model always make the whole workflow slower?
Not necessarily by a noticeable amount. Detection and tracking can dominate processing time.

Learn More about ComfyUI-InstantHMR

Practical resources

Background and deeper reading

ComfyUI-InstantHMR Related Nodes

RunComfy
Copyright 2025 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals. RunComfy also provides AI Models, enabling artists to harness the latest AI tools to create incredible art.

ComfyUI-InstantHMR detailed guide | ComfyUI