InstantHMR Live Capture:
InstantHMRLiveCapture provides a live human-body capture and 3D preview experience using a camera on your device or a phone on the same local network. It combines an RT-DETR person detector, which locates people in the camera image, with InstantHMR, which estimates their body shape and pose for visualization. You can preview movement using the MHR body mesh or a supported rigged character, steady the motion to reduce visible jitter, and optionally stream the first detected person’s motion to a compatible VMC application. This makes it useful for checking poses, exploring character movement, and recording camera footage for later workflows. Live capture and workflow execution are separate: the interactive preview performs the live processing, while running the node loads an already recorded clip rather than starting a new capture or processing that clip through InstantHMR.
InstantHMR Live Capture Input Parameters:
recording
Selects the clip recorded through the live view that will be loaded when you run the workflow. Its options are populated dynamically by the extension’s recording-file list; the source does not specify a fixed default or a numeric range. This is the only setting needed for workflow execution itself: the node checks that the selected recording is available, resolves its file path, and opens it as a video. Changing the live-preview settings does not cause the selected clip to be reprocessed when the workflow runs. If the recording is no longer listed, execution fails with a recording-not-found message.
model
Selects the InstantHMR model used to estimate human body shape and pose during the live preview. Options come from ComfyUI’s registered detection model location and include only filenames containing instanthmr, matched without regard to letter case. No explicit default is defined. You need a valid selection for live inference, but this model is not loaded by the node’s workflow execution method, which only opens the selected recording. Accordingly, the node’s validation does not require the live model to be available when you are simply loading an existing clip on another machine.
detector
Selects the RT-DETR person detector used to locate people in the live camera image before body estimation. Options come from ComfyUI’s registered diffusion_models location and include only filenames containing rt_detr, matched without regard to letter case. No explicit default is defined. A valid detector is required for live inference, and its detection boxes help identify the image regions containing people. The frequency at which it runs is controlled by detect_every. Like the InstantHMR model, it is a live-preview setting rather than a requirement for opening an existing recording during workflow execution.
camera
Chooses the camera source for the live capture interface. The available options are Phone / other device and This device; no explicit default is declared in the source. Choose This device to use a camera attached to the device running the capture interface, or Phone / other device to display a QR code that you can open on a phone or another device on the local network. This setting changes how you supply live camera images, not which saved recording is loaded when the workflow runs.
character
Chooses the body displayed in the live 3D view. Options consist of MHR body followed by the character files discovered by the extension; supported character extensions are .fbx, .glb, .gltf, and .vrm. No explicit default is declared. The built-in MHR body provides a direct body-mesh preview, while a supported rigged character from the input folder lets you view the estimated movement on a character. This selection controls live visualization; the execution method does not use it to render or transform the recorded clip, and validation permits loading an existing recording even if the selected character is unavailable on the current machine.
vmc
Sets an optional destination for streaming the first detected person’s motion to an application supporting the VMC protocol, such as VSeeFace, Warudo, or VNyan. This is a text field with a default of an empty string, which disables streaming. When enabled, enter a host and port in the form host:port, for example 127.0.0.1:39539; this example is the field’s placeholder, not its default destination. The receiving application must be configured to listen at the matching address and port. This setting affects live motion streaming only and does not add motion data or another output to the workflow.
fov
Sets the vertical camera field of view in degrees for live body estimation. The default is 0.0, the minimum is 0.0, the maximum is 179.0, and the adjustment step is 1.0; this is marked as an advanced setting. A value of zero means the system uses its fallback focal length of 1.05 times the image diagonal, rather than treating the camera as having a literal zero-degree field of view. A nonzero value supplies a vertical field of view for interpreting camera perspective, so it is most useful when you know the camera’s actual vertical viewing angle. It affects the live reconstruction, not the loading of a saved recording.
detect_every
Controls how often the live person detector runs. The default is 2, the minimum is 1, and the maximum is 10; this is marked as an advanced setting. The detector runs every N frames and reuses its previous detection boxes between those frames. A value of 1 refreshes detection every frame, which can help when people move quickly or change position, but requires more frequent detector processing. Higher values reduce detector work, although reused boxes can become less well matched to rapidly moving people. This setting changes the live detection schedule, not the frame rate or contents of a recording when it is loaded by workflow execution.
smoothing
Adjusts stabilization of the live body motion. The default is 0.5, the minimum is 0.0, the maximum is 1.0, and the adjustment step is 0.05. Higher values make the body steadier, particularly when you hold a pose, while the smoothing is designed to let fast movements through. A value of 0 uses raw, unsmoothed motion estimates. Start with the default for general preview work, increase it if small unwanted movements are distracting, or reduce it to inspect the underlying estimates. It is a live-preview setting and does not apply a new smoothing pass to the saved clip when you run the node.
InstantHMR Live Capture Output Parameters:
VIDEO
Provides the selected recorded clip as a ComfyUI VIDEO value, created by opening the recording’s resolved file path. You can connect it to compatible downstream nodes that accept video for further handling or processing. This value represents a saved recording, not a continuously updating camera feed, and executing the node does not start live capture. The node does not expose body meshes, pose parameters, detection boxes, or VMC motion as separate workflow outputs. Its execution method opens the existing video without adding a new body-estimation pass or character render, so you should not interpret this output as exported 3D motion data.
InstantHMR Live Capture Usage Tips:
- Set up and test the live preview before running the workflow. Select an InstantHMR model and an RT-DETR detector, choose your camera source, record a clip through the live interface, and then select that clip in
recordingfor downstream use. - If a model selector is empty, check both the model’s registered location and its filename. The
modellist filters thedetectionlocation forinstanthmr, whiledetectorfiltersdiffusion_modelsforrt_detr; a file that does not match these filters will not appear. - Start with
detect_everyat2. Try1for faster movement or frequent changes in people’s positions; increase it gradually if detector processing is limiting live performance, watching for detection boxes that no longer follow movement well. - Use
smoothingaround its default of0.5for an initial test. Increase it for steadier held poses, or set it to0when you want to inspect raw tracking behavior without stabilization. - Leave
fovat0when you do not know the camera’s vertical field of view. If you have a reliable vertical field-of-view measurement, use that value rather than a horizontal or diagonal specification. - Start with
MHR bodyto check the basic 3D preview before testing a custom character. For character previews, use a supported rigged.fbx,.glb,.gltf, or.vrmfile from the input folder. - For phone capture, choose
Phone / other device, open the displayed QR code on your phone, and keep the phone and ComfyUI host reachable on the same local network. - For VMC streaming to an application on the same computer, enter its configured receiving address, such as
127.0.0.1:39539. Match the receiver’s actual port and remember that only the first detected person is streamed. - Avoid running a queued workflow while using live inference. The live service deliberately pauses during workflow execution to prevent competing operations on the same models.
- When reusing an existing recording on another machine, make sure that recording is available in the extension’s recording list. The live models and character do not need to be present just to load the saved clip.
InstantHMR Live Capture Common Errors and Solutions:
Recording not found: …
- Explanation: The selected
recordingis not present in the extension’s current recording list. This is the validation error explicitly returned by the node; it can occur if the clip was moved, deleted, or is unavailable on the current machine. - Solution: Select an available recording, restore the missing clip to its expected recording location, or create a new recording through the live interface. If you transferred the workflow to another machine, transfer the recording as well.
Pick an InstantHMR model and an RT-DETR detector in the InstantHMR Live Capture node
- Explanation: The live service has no selection for the InstantHMR model, the RT-DETR detector, or both, so it cannot process incoming camera frames.
- Solution: Select both models in the node. If either list is empty, make sure the appropriate model is available in its registered ComfyUI location and that its filename matches the selector’s filter:
instanthmrfor the body model andrt_detrfor the detector.
Paused while a workflow runs
- Explanation: Live inference is temporarily blocked because a ComfyUI workflow is running. The source explicitly protects against simultaneous live and workflow operations that could load or move the same models and crash the process.
- Solution: Wait for the workflow to finish, or stop it before continuing live capture. Treat this as a protective pause rather than a missing-model or camera failure.
Other live inference error messages
- Explanation: If live inference raises an exception, the service forwards the exception’s text to the capture interface and viewers. The supplied code does not define a fixed list of these messages, so the exact wording depends on the underlying failure.
- Solution: Read the displayed message and check the ComfyUI console for more detail. Confirm that your selected models are available and appropriate, then test with
MHR bodyand the default live settings to narrow down the issue. If the message identifies a resource or model-loading problem, resolve that specific issue before retrying.
Phone cannot connect or camera preview is unavailable
- Explanation: This is a troubleshooting symptom, not a fixed error string in the supplied code. Phone capture requires access to the live interface over the local network, while camera capture also depends on the device making its camera available.
- Solution: Confirm the selected
cameramode, allow camera access if prompted, and check that the camera works on the capturing device. For phone capture, reopen the displayed QR code and verify that the phone can reach the ComfyUI host on the local network.
VMC application receives no motion
- Explanation: This is a troubleshooting symptom rather than a predefined node error. Streaming is off when
vmcis empty, and an enabled destination must match a listening VMC receiver. Only the first detected person is streamed. - Solution: Enter a valid
host:portdestination, enable the receiver in your VMC-compatible application, and match its configured port. Use127.0.0.1only when the receiver runs on the same computer as the streaming service, check network access for a receiver on another computer, and confirm that the live preview is detecting a person.
