H3 Face Mask (SAM):
H3FaceMaskSAM creates face-shaped masks for a sequence of stabilized face crops. It uses SAM to identify the face in each original crop, guided by the face’s tracked location, so the masked area can follow the face more closely than a rectangular selection. The masks are made before enhancement, which helps cover the face as it appears in the source footage even if a generated replacement changes its shape. The node can also expand mask boundaries and average them across nearby frames to reduce visible edge flicker in video. When SAM does not produce a usable mask for a frame, the node uses that frame’s face rectangle as a fallback.
H3 Face Mask (SAM) Input Parameters:
crops
An IMAGE batch containing the original, stabilized face crops, with one crop per frame. Connect the crops output of H3 Face Track + Crop, not a refined or decoded result. SAM examines these source images to decide where the face is before any replacement is generated. Using enhanced images instead may leave parts of the original face outside the mask when the generated face has a smaller or shifted silhouette. No default, minimum, or maximum is specified for this input.
sam_model
A SAM_MODEL used to segment the face in each crop. The node prepares the model for prediction when the supplied model supports device preparation, then releases it afterward when supported. The model affects the quality and speed of the face-shaped masks; segmentation is attempted separately for every frame, so longer clips can take more time. No particular model choice, default, minimum, or maximum is specified by this node.
transform
An H3FACEXFORM containing face rectangles for the stabilized crops. For each frame, the rectangle supplies a box and center-point prompt that guide SAM toward the intended face. It also provides the rectangular fallback if that frame has no usable SAM mask. If a frame has no corresponding face_rect entry, the node uses a default rectangle beginning one-quarter of the crop width and height from the top-left and spanning half its width and height. No default input value, minimum, or maximum is specified.
threshold
A FLOAT passed to the SAM model’s prediction method for each frame. It may affect which predictions that model returns, but the supplied node source does not define the model’s internal interpretation of the value, so its precise visual effect depends on the connected SAM implementation. The default is 0.93, the minimum is 0.0, the maximum is 1.0, and the step is 0.01. This is separate from the node’s fixed check that converts a returned mask into a binary mask.
dilation
An INT controlling how far the mask grows outward after segmentation and rectangular fallbacks. Increasing it can give the replacement more coverage around the face, but too much growth may include nearby hair, background, or other areas you intended to preserve. A value of 0 leaves the boundary ungrown. The default and minimum are 0, the maximum is 128, and the step is 2.
temporal_smooth
An INT controlling how many neighboring frames are averaged to reduce changes in the mask edge from frame to frame. The default is 5, the minimum is 1, the maximum is 31, and the step is 2. A value of 1 disables smoothing; larger values can make edges steadier but may carry a mask’s shape across frames where the face moves or changes pose. Smoothing runs only when the batch has more than two frames, and the effective window is limited to an odd number that fits the batch.
H3 Face Mask (SAM) Output Parameters:
masks
A MASK batch with one mask per input crop, at the same height and width as the crops. Areas selected for pasting have higher values, while unselected areas have lower values. SAM results are converted to binary masks before optional dilation; temporal smoothing can then produce intermediate values along boundaries or in areas that change between frames. If a frame has no nonempty mask after the SAM pass, its face rectangle is filled instead, provided that rectangle has a valid area.
report
A STRING summarizing the pass. It shows the number of frames for which SAM returned an accepted, correctly sized mask, the number reported as falling back to face rectangles, the configured dilation and temporal_smooth values, and the final masks’ mean coverage as a percentage of the crop canvas. The segmentation count tracks accepted SAM results rather than checking whether each result remained nonempty; an empty accepted result can still receive a rectangle fallback later. Use the report as a quick diagnostic, and inspect the masks when exact per-frame behavior matters.
H3 Face Mask (SAM) Usage Tips:
- Connect the original stabilized
cropsdirectly from H3 Face Track + Crop. Make the masks from source footage before running enhancement so generated changes cannot alter the area selected for pasting. - Start with
dilationat its default of0. Increase it gradually only when you can see original-face edges remaining outside the pasted area. - Start with
temporal_smoothat5for video. Try a higher value if the boundary flickers; reduce it toward1if the mask seems to lag behind quick movement or pose changes. - Check the
reportafter a run. A high fallback count means many frames did not receive an accepted SAM result; inspect the face rectangles and the connected SAM model before relying on those masks. - Preview several frames, including profiles and rapid movement. A smooth-looking mask on one frame does not establish that it covers the original face throughout the clip.
H3 Face Mask (SAM) Common Errors and Solutions:
Required input is not connected
- Explanation: The node requires
crops,sam_model,transform,threshold,dilation, andtemporal_smooth. A missing required connection or value prevents the node from receiving what it needs to run. - Solution: Connect the three typed inputs and supply values for the three numeric controls. Use the original crop batch and its matching transform.
SAM masks: 0/N frames segmented
- Explanation: The report indicates that SAM did not return an accepted, correctly sized mask for any frame. The node attempts face-rectangle fallbacks, but these are rectangular rather than face-shaped. An exception raised by
predict()is caught by the node, so the report may appear instead of a detailed SAM error. - Solution: Confirm that a working
SAM_MODELis connected, the crops contain visible faces, and the transform corresponds to those crops. Preview the resulting masks to see whether the rectangle fallbacks cover the intended faces.
N fell back to the face rect
- Explanation: Some frames had no nonempty mask after the SAM pass and were filled using their face rectangles when those rectangles had valid dimensions. These frames may show a more rectangular paste boundary.
- Solution: Inspect the affected footage and the tracking results. Improve or correct face tracking where the rectangle misses the face, and check that the SAM model can segment the source crops.
Mask is empty or misses the face
- Explanation: A missing per-frame
face_rectcauses the node to use a generic central rectangle. A rectangle with zero or invalid area cannot fill the mask as a fallback. A prompt aimed at the wrong location can also make SAM select the wrong region or return no usable result. - Solution: Check that the transform contains a face rectangle for each crop and that the rectangles cover the intended face. Preview the crop and mask together, especially on frames where tracking is uncertain.
Original face remains visible around the replacement
- Explanation: The mask may be too small for the face in the source footage, or the masks may have been generated from enhanced images instead of the original crops.
- Solution: Reconnect the original
cropsoutput, inspect the corresponding face rectangles, and increasedilationin small steps only if additional coverage is needed.
Mask edge flickers between frames
- Explanation: SAM runs independently on each frame, so its boundary can shift slightly. With
temporal_smoothset to1, no temporal averaging is applied; smoothing also does not run for batches of two frames or fewer. - Solution: For a longer batch, try the default value of
5and increase it cautiously if the edge still shimmers. Keep the crops stabilized so neighboring masks remain reasonably aligned.
Mask appears to trail movement
- Explanation: A larger
temporal_smoothvalue averages more neighboring frames, which can blend together masks from noticeably different face positions or poses. - Solution: Lower
temporal_smoothand check whether the input crops are properly stabilized.
Masking takes much longer than expected
- Explanation: The node calls SAM once per frame, so processing time grows with the number of crops. The node asks supported SAM models to prepare their device before prediction, but actual speed still depends on the model and available hardware.
- Solution: Test a short crop batch first, confirm that your SAM setup can use the intended device, and allow more time for long clips. You can interrupt processing between frames.
Shape or size mismatch error
- Explanation: The node expects
cropsto be an image batch shaped as frames by height by width by channels, and it expects usable mask results to match each crop’s height and width. SAM results of another size are not accepted; an incorrectly shaped crop batch may fail before prediction. - Solution: Supply an
IMAGEbatch from the expected crop stage and a matching transform. Check upstream nodes for unexpected resizing, missing frames, or incompatible data.
