Follow Matte:
Follow Matte turns a SAM 3.1 tracking result into a consistent sequence of mattes for compositing. A matte is a black-and-white selection that determines where an edited or generated object will be included in a composite. Rather than accepting everything the tracker detects, this node follows the object’s established region and excludes tracked areas that lie too far away, even when they are connected to the object. It fills enclosed holes in accepted regions and prevents thin extensions from gradually pulling the search toward unrelated areas. When the object is not found, it holds the previous matte instead of leaving that frame without a selection; if tracking provides no usable selection, the initial selection supplies the fallback. When a matching background plate is available, the node can also include nearby areas changed by the generated shot, helping preserve generated details that tracking alone might miss. Spatial limits and a size-based safeguard help prevent those additions from spreading uncontrollably into the background.
Follow Matte Input Parameters:
masks
Required MASK input containing SAM 3.1’s tracked selections, normally one mask for each frame it tracked. These selections provide the object shapes that the node checks against the object’s established region. Regions outside the permitted neighborhood are excluded rather than automatically included just because SAM selected them. The tracking sequence may contain fewer masks than the shot has frames, or no tracked masks; the node is designed to maintain a matte through missing detections by holding the last available result, with the first-frame selection as the fallback. This socket is still required even though its mask sequence may contain no tracked results. No numeric default, minimum, maximum, or selectable options are defined for this input.
first
Required MASK input identifying the area on the first frame that SAM was asked to follow. This gives the node a starting reference for deciding which tracked region belongs to your intended object, rather than accepting unrelated detections elsewhere in the image. It also provides the documented fallback when no tracked mask is available. Choose a selection that clearly identifies the intended object or edit region; an overly broad selection can weaken the spatial separation between that object and nearby distractions. The optional element input can replace this selection as the starting object region. No numeric default, minimum, maximum, or selectable options are defined for first.
frames
Required IMAGE input containing the frames SAM tracked; for a paint workflow, supply the generated shot. The number of images in this batch determines how many mattes the node produces, even when the tracking sequence is shorter. When base is connected, these frames also supply the shot’s RGB pixels for comparison with the background plate so that nearby changed areas can join the matte. Without base, the node does not use frame-to-plate differences to expand the selection. Use the actual shot associated with the tracked masks, in the same frame order. No numeric default, minimum, maximum, or selectable options are defined for frames.
zone
Required FLOAT input controlling how far the accepted region may extend from the object’s established location, expressed as a fraction of image width. Its default is 0.0625, minimum is 0.0, maximum is 0.5, and adjustment step is 0.005. At the default setting, the nominal distance corresponds to 6.25% of the width—for example, approximately 120 pixels in a 1920-pixel-wide image. This neighborhood limits both the tracked regions that can be accepted and the nearby image changes that can join the matte. Smaller values make the node more selective but can reject fast movement or extended generated details; larger values allow more movement and expansion but increase the chance of admitting unrelated nearby content. When detections are lost, the tracking search neighborhood expands progressively, up to four times its normal reach, to help reacquire the object.
threshold
Required FLOAT input controlling how much a pixel must differ between the shot and the background plate before it can qualify as changed. Its default is 0.1, minimum is 0.01, maximum is 1.0, and adjustment step is 0.01. This setting affects change-based additions when base is supplied; it is not a SAM confidence threshold and does not directly control which tracked detections SAM produces. Lower values make smaller differences eligible, which can preserve subtle generated details but may also admit background variations. Higher values require stronger differences and can reduce unwanted additions, although subtle parts of the edit may then be missed. A qualifying change must still satisfy the node’s spatial and size safeguards, so a low threshold does not mean every changed pixel in the frame will be included.
base
Optional IMAGE input containing the background plate over which the shot was generated, at the same image size. Connecting it enables comparison between the plate and frames, allowing nearby changes to be added to the tracked object’s matte. This is useful when a generated object includes details that SAM does not fully recognize. The comparison is spatially restricted around the object, and unusually large additions are further restricted to reduce runaway selection. Use the actual corresponding plate rather than an unrelated background: framing, alignment, or appearance differences outside the edit can otherwise look like generated changes. The default is no connected image, represented internally as None; in that case, change-based additions are disabled. No numeric range or selectable options are defined.
element
Optional MASK input identifying where the edit changed the first frame. When provided, this establishes the starting object region instead of first, helping distinguish the actual generated element from a broader area originally used to guide SAM. It is particularly useful when your original tracking selection includes extra surrounding space but you have a more precise selection of the generated content. This input is a spatial starting reference, not a replacement sequence of tracked masks. Supply it in the same image coordinate system as the shot and tracking selections. The default is no connected mask, represented internally as None, so first supplies the starting reference. No numeric minimum, maximum, or selectable options are defined.
Follow Matte Output Parameters:
mask
MASK output containing one matte for every frame in frames. Each matte represents the accepted object region after spatial filtering, optional inclusion of nearby plate-to-shot changes, and filling of enclosed holes. Selected areas are represented as 1 and unselected areas as 0; these are region selections rather than automatically feathered edge mattes. If a usable region is not found on a frame, the preceding matte is retained instead of returning an empty selection, using the first-frame selection when necessary. This makes the result useful as an edit-selection mask for downstream compositing, but a held matte does not predict unseen motion or reconstruct the object’s hidden shape. Check any held sections and apply downstream edge treatment if your composite needs softer boundaries.
summary
STRING output containing a textual report returned by the matte-following operation. It provides diagnostic information alongside the mask sequence so you can assess how the node processed the shot. The supplied implementation tracks events such as frames with accepted tracked regions, held mattes, outlying tracked regions, and substantial change-based additions, but the exact wording and formatting of the returned report are not shown in the provided source. Treat this output as a human-readable diagnostic rather than a fixed-format data structure, and use it together with visual inspection of the mask sequence.
Follow Matte Usage Tips:
- Start with zone at 0.0625 and threshold at 0.1, then preview the entire matte sequence before making adjustments. Check frames with rapid movement, occlusion, or large generated changes.
- Connect the actual background plate to base when SAM misses parts of a generated element. Without base, lowering threshold cannot recover those details because plate-to-shot comparison is disabled.
- Increase zone gradually if the object moves beyond the accepted neighborhood. Reduce it if unrelated SAM detections near the object enter the matte; a larger zone is not automatically more accurate.
- Lower threshold gradually when subtle changed details are missing despite a correctly matched base. Raise it when small background differences are being added. This setting only affects additions based on image changes.
- Supply element when the original first selection is much broader than the generated object. A more precise starting region helps keep the tracking neighborhood focused on the intended edit.
- Keep frames, base, and masks spatially aligned. Matching image dimensions alone is not enough if the plate and shot have different crops, offsets, or camera positions.
- Inspect sections where the object disappears. Holding the previous matte avoids gaps, but it can leave a selection over the object’s last visible position; manually refine these frames if the intended composite requires the object to disappear completely.
- Add feathering or other edge preparation downstream when needed. Follow Matte fills holes and filters regions, but its output should not be assumed to have a soft compositing edge.
Follow Matte Common Errors and Solutions:
Required input is missing
- Explanation: This is a connection or validation issue, not a custom error string shown in the supplied code. The node requires masks, first, frames, zone, and threshold. Support for an empty tracking result does not make the masks socket optional.
- Solution: Connect the required image and mask inputs and keep both numeric controls within their supported ranges. If SAM returned no detections, provide its empty tracking result through the appropriate MASK connection rather than disconnecting the socket.
Image or mask dimensions do not match
- Explanation: This is a possible shape-related processing failure; the exact exception text depends on the operation that encounters it. The node expects spatially corresponding selections and images, and base is explicitly described as being at the same size as the shot. No resizing or alignment step is shown in the node’s execution code.
- Solution: Resize or prepare the shot, plate, and selections upstream so that they use matching dimensions and framing. Use ComfyUI IMAGE and MASK outputs rather than manually shaped arrays, and verify that first and element refer to the same image coordinates as the tracked masks.
The matte remains frozen on some frames
- Explanation: This is an expected fallback behavior rather than an exception. When no acceptable region is found, the node holds the previous matte. This can happen when tracking loses the object or its next detection falls outside the permitted neighborhood.
- Solution: Inspect SAM’s original tracking result on those frames. Improve the track or increase zone moderately if valid detections are being rejected because of movement. If the object genuinely leaves the scene, manually remove or adjust the held selection where appropriate.
Parts of the generated object are missing
- Explanation: This visual symptom can occur when SAM misses a detail, the detail lies outside the permitted neighborhood, or its difference from base is below threshold. Change-based recovery is unavailable when base is disconnected.
- Solution: Connect the matching plate, then try lowering threshold slightly or increasing zone enough to include the missing detail. Check that element is not incorrectly positioned. Very large change regions may still be restricted by the node’s size safeguard, so improve the track or refine the matte manually if necessary.
Unrelated objects or background changes appear in the matte
- Explanation: This visual symptom can occur when the accepted neighborhood is too broad or plate-to-shot differences qualify as changes near the object. A broad starting selection, a large zone, a low threshold, or a mismatched base can contribute.
- Solution: Tighten first or supply a precise element mask, reduce zone, and raise threshold if change-based additions are responsible. Verify that base is the correct aligned plate. Temporarily disconnect base to determine whether the unwanted area comes from tracking or from image differences.
The entire shot holds the initial selection
- Explanation: This is a possible fallback result, not necessarily a processing error. It can occur when no usable tracked region is available throughout the sequence and no acceptable change-based region is found.
- Solution: Confirm that masks contains the intended SAM tracking sequence and that its selections overlap the starting object region. Check first and element for incorrect placement, verify frame order, and inspect whether zone is too restrictive.
The matte has hard edges or filled interior openings
- Explanation: This follows the demonstrated processing rather than indicating a fault. The node generates region selections and explicitly fills enclosed holes, so it is not guaranteed to preserve every interior opening or produce a feathered edge.
- Solution: Apply edge softening downstream if required for compositing. If openings must remain transparent, restore them with additional mask editing or combine the result with a suitable detail mask before compositing.
