ComfyUI-Reference-Based-Video-Colorization Introduction
ComfyUI-Reference-Based-Video-Colorization adds color to black-and-white images and videos using a color image that you choose as a reference. Instead of asking a model to invent a palette entirely on its own, you give it a visual guide: for example, a color photograph showing the clothing, scenery, or overall mood you want in the finished footage.
The extension brings two colorization methods into ComfyUI: ColorMNet and Deep Exemplar. Each has nodes for a single image and for a sequence of video frames. For video, both methods are designed to help colors remain consistent as the scene changes from frame to frame. You can try the same reference with both methods, compare the results, and choose the look that suits your project.
A useful reference does not have to be an exact copy of your subject. It should, however, show similar objects or scenery and the colors you want to carry into your work. Try several references when a particular color choice matters.
<p align="center">
<img src="assets/Header Screenshot.png" alt="ComfyUI reference-based video colorization workflow" width="100%"/>
See it in action
<video width="100%" controls>
<source src="assets/ColorizationSample.mp4" type="video/mp4">
Your browser does not support the video tag. <a href="assets/ColorizationSample.mp4">View the colorization sample video instead.
The sample shows ColorMNet colorizing black-and-white footage with color-matching post-processing.
How ComfyUI-Reference-Based-Video-Colorization Works
Think of the reference image as a color guide and the black-and-white footage as the scene to paint. The extension looks for visual relationships between them, then applies colors from the reference to the grayscale image or frames. A reference featuring a green landscape, for example, can help guide the color of similar scenery in your footage.
Video introduces an extra challenge: if every frame is colored independently, an object may change color as it moves. The two methods address this in different ways:
- ColorMNet uses a memory-based approach. It draws on information across frames to help maintain a coherent result over time. It also uses DINOv2 image features to help interpret visual content.
- Deep Exemplar uses an exemplar, meaning your reference image, to guide colorization. Its video node can propagate information between frames to improve continuity.
Neither method guarantees historically accurate colors. The reference is an artistic guide, so the choice of image can substantially change the result.
A basic workflow
- Load a black-and-white video as frames, or load a single grayscale image.
- Load a color reference image.
- Connect both inputs to the corresponding ColorMNet or Deep Exemplar node.
- Set the target width and height, then run a short test.
- Review the colorized output and, for video, check whether colors stay steady across frames.
- Try another reference or method if the palette or consistency is not what you want.
For video, the supplied example is Workflows/Colorize_Video_Workflow.json. It uses a VHS Video Loader for input and VHS Video Combine for export. For one image, use Workflows/ColorMNet_Image_Workflow.json, which loads the target and reference images, runs ColorMNetImage, and saves the result.
ComfyUI-Reference-Based-Video-Colorization Features
ColorMNet video and image nodes
ColorMNet Video (ColorMNet/Video) accepts grayscale video_frames and a color reference_image. It outputs colorized_frames and a performance_report.
target_widthandtarget_heightset the output dimensions. A smaller test render can help you assess a reference before committing to a larger render.memory_modecontrols the quality and memory trade-off. Start withbalanced; uselow_memorywhen memory is limited, or tryhigh_qualitywhen you can devote more memory to the render.use_fp16enables half-precision processing and is on by default. The author recommends it as a speed-oriented setting.use_torch_compileenables a PyTorch 2.0+ optimization. It is off by default, and the first run can take longer while compilation warms up.
ColorMNet Image (ColorMNet/Image) accepts a grayscale image and a reference_image, then outputs a colorized_image and performance_report. It has the same target-size, FP16, and compilation controls, but no video memory-mode setting.
Deep Exemplar video and image nodes
Deep Exemplar Video (DeepExemplar/Video) accepts grayscale video_frames and a reference_image. It outputs colorized_frames and a performance_report.
frame_propagateis on by default and uses information from frames to support more consistent video color. Turn it off if you want frames processed independently.use_half_resolutionis on by default and reduces internal processing resolution to save memory. Turn it off when you want to test a quality-focused render and have enough memory.use_torch_compileanduse_sage_attentionare optional speed settings, both off by default. SageAttention applies only to Deep Exemplar.- Weighted Least Squares (WLS) filtering is available for smoother results. Its
lambdaandsigmasettings affect the filtering behavior; compare short previews before applying changes to a full video.
Deep Exemplar Image (DeepExemplar/Image) accepts image_to_colorize and reference_image, then outputs colorized_image and performance_report. It provides target-size, compilation, and SageAttention settings.
Progress and performance reports
The nodes provide progress feedback while processing. Their optional performance reports show measurements such as total time, time per frame, and frames per second (FPS), along with relevant settings. Connect performance_report to a text display node when you want to compare two runs. FPS is most useful when comparing similar clips and resolutions on the same computer.
The author reports speed improvements of 15–25% from torch.compile after its initial warmup and 20–30% faster attention operations with SageAttention where available. These are optimization estimates, not guaranteed improvements to an entire render.
ComfyUI-Reference-Based-Video-Colorization Models
The extension offers two colorization methods, not a list of interchangeable model variants:
| Method | Best reason to try it | Available nodes |
|---|---|---|
| ColorMNet (2024) | You want a memory-based approach to maintaining color across video frames, or you want to compare its result with Deep Exemplar. | Video and image |
| Deep Exemplar (2019) | You want the classic reference-based approach, with optional frame propagation and smoothing controls for video. | Video and image |
For a fair comparison, feed both methods the same grayscale content and reference image. Look beyond a single attractive frame: check faces, clothing, backgrounds, and moving objects throughout the clip. There is no single setting or method that will suit every reference and scene.
The models are downloaded automatically on first use according to the original documentation. The ColorMNet checkpoint, Deep Exemplar checkpoints, and supporting model files are stored within the extension’s checkpoints/ and data/ directories. The first run may therefore take longer than later runs.
Model-use rights matter: the extension’s implementation is licensed under MIT, but the original README identifies the ColorMNet model as CC BY-NC-SA 4.0, for non-commercial use, and says the Deep Exemplar model is subject to its original project license. Check the applicable model terms before using output in a commercial project.
What's New with ComfyUI-Reference-Based-Video-Colorization
The supplied README lists recent changes but does not provide version numbers or release dates. They are described here without assigning them to a particular release:
- Corrected ColorMNet pipeline normalization. The pipeline now matches the stated training setup more closely, using luminance-only input for the encoder and normalized color-component masks. This is a behind-the-scenes correction to how ColorMNet prepares its inputs.
- Automatic handling of two ColorMNet CUDA dependencies. The node now attempts to install
py-thin-plate-splineand the PyTorch correlation extension when it loads. The original README says the nodes can still run without them, though potentially more slowly. - A new single-image sample workflow.
Workflows/ColorMNet_Image_Workflow.jsonprovides a shorter starting point when you want to colorize a still image rather than assemble a video workflow.
Troubleshooting ComfyUI-Reference-Based-Video-Colorization
The colors do not match the look I wanted
- Try a reference with subjects or scenery more similar to your source.
- Choose a reference that clearly shows the palette you want, rather than one whose important colors are hard to see.
- Test a few references on a short clip or single image before processing the full video.
- Compare ColorMNet and Deep Exemplar; the same reference may produce different results.
Colors change noticeably between video frames
- Check several consecutive frames, not just the first and last.
- In Deep Exemplar Video, confirm that
frame_propagateis enabled. - Try ColorMNet Video as a comparison, since it uses memory across frames.
- If the reference is a poor visual match for objects in the footage, test a more similar image.
Processing runs out of memory or is too slow
- Lower
target_widthandtarget_heightfor a test render. The original README suggests 768 × 432 as a starting balance between speed and quality. - For ColorMNet, try
memory_mode=low_memory. - For Deep Exemplar Video, leave
use_half_resolutionenabled. - Process a shorter clip to test your settings.
- For later runs, consider
use_fp16where available oruse_torch_compileif your PyTorch version supports it. Remember that compilation can make the first run slower.
An optional acceleration feature is unavailable
- Turn off
use_torch_compileoruse_sage_attentionand try the render again. torch.compilerequires PyTorch 2.0 or newer. SageAttention requires a compatible CUDA-enabled setup and thesageattentionpackage.- For ColorMNet’s optional CUDA extensions, consult the project documentation if automatic setup fails. On Windows, the original README notes that compiling them may require Visual Studio Build Tools with Desktop development with C++ and a correctly configured
CUDA_HOME. - Run a short test after changing one setting at a time so you can tell which change helped.
Frequently asked questions
Can I colorize one photograph instead of a video?
Yes. Both methods include image nodes. The supplied single-image example workflow uses ColorMNet.
Do I need a color version of the exact same frame?
The reference is a guide, not necessarily an exact matching frame. A visually similar image with the desired palette is a good place to start.
Which method should I choose first?
Start with either method and a short test. ColorMNet offers memory modes; Deep Exemplar offers frame propagation and half-resolution controls. Judge the result on your own footage.
Why is my first run taking longer?
The models download automatically on first use, and enabling torch.compile adds an initial compilation warmup.
Can I use the results commercially?
Do not assume the extension’s MIT license covers its models. Review the ColorMNet and Deep Exemplar model licenses for your intended use.
For a visual reference while troubleshooting, compare your setup with the workflow screenshot and the sample video.
Learn More about ComfyUI-Reference-Based-Video-Colorization
- Quick Start — more guidance on getting a workflow running.
- Performance Guide — benchmarks and optimization details for comparing settings.
- Migration Guide — information for people moving from an older version.
- Architecture — a deeper technical explanation if you want to understand the implementation.
- ComfyUI — the application in which these nodes run.
- Original ColorMNet project and original Deep Exemplar project — the research implementations behind the two methods.
- Project issues — a place to check whether another user has encountered the same problem or to report a reproducible issue.
