ComfyUI-DLSS5 Introduction
<p align="center">
<img src="docs/images/icon.png" width="160" alt="Experimental DLSS Neural Rendering project icon">
ComfyUI-DLSS5 is an unofficial, experimental Windows extension that brings NVIDIA DLSS-based image and video processing into ComfyUI. It provides tools for increasing image resolution, applying an experimental neural-rendering pass, and optionally generating extra frames for video.
You can use it to:
- Upscale images and video by 2×, 3×, or 4× with DLSS Super Resolution.
- Apply experimental Neural Rendering at the original resolution or after upscaling.
- Start with one-node presets for still images, short videos, longer videos, and quick previews.
- Create depth and motion guides to help processing understand the apparent structure and movement in your footage.
- Choose video-processing modes to balance continuity and native processing memory.
- Generate intermediate video frames through the optional DLSS Frame Generation worker.
For AI artists, the extension offers a way to explore larger outputs and alternative rendering treatments without rebuilding an entire generation workflow. Its video tools also provide guidance based on neighboring frames, rather than treating every frame as an unrelated image.
Experimental preview: The documented extension version is v0.5.0. Results depend on your source material, settings, GPU, and runtime. The author reports that measured 4K output throughput remains below 30 source frames per second; do not assume real-time processing. See the measurements and limits.
Important limitations
- This is an experimental alpha project, not an official NVIDIA or ComfyUI product.
- “DLSS5” is the historical project name. It does not mean this extension is an official NVIDIA DLSS 5 SDK integration.
- Depth and motion are estimated from your images. They are not the accurate scene data that a game engine can supply.
- Upscaling, neural rendering, and frame generation are different operations. Increasing resolution does not automatically increase the video’s frame rate.
How ComfyUI-DLSS5 Works
The extension connects ComfyUI images and video batches to VapourSynth and Direct3D 12 bridge wrappers. In simpler terms, it passes your images to a separate processing system that can use the NVIDIA runtime, then returns the results to ComfyUI.
A typical guided workflow has four parts.
1. Start with an image or a sequence of frames
A still image is processed as a single frame. A video is loaded as a batch: an ordered collection of images.
The supplied video workflows use ComfyUI-VideoHelperSuite to load footage. Some also include a video encoder to turn the processed frames back into a video file.
2. Estimate depth and movement
The extension can prepare two kinds of guide:
- Depth: An estimate of which parts of the picture are closer to or farther from the viewer. Think of it as a rough stage layout—not a true 3D reconstruction.
- Motion: An estimate of where visible details moved between frames. Think of it as arrows connecting a detail in the current frame to its earlier position.
For a still image, there is no previous frame, so motion is zero. Video presets use optical flow or RAFT to estimate movement.
Temporal depth tools can use information across frames to make depth estimates more consistent. This is useful because a depth model working on individual images may interpret the same object slightly differently from frame to frame.
3. Choose the processing operation
| Operation | What it does | Example use |
|---|---|---|
| Super Resolution, or SR | Increases image dimensions by 2×, 3×, or 4× | Turn a 960 × 540 frame into a 1920 × 1080 frame at 2× |
| Experimental Neural Rendering, or NR | Applies a neural pass without changing the current resolution | Explore a different rendering treatment while keeping the same dimensions |
| SR + Neural Rendering | Upscales first, then applies NR | Create a larger output with a subsequent neural-rendering treatment |
| Optional Frame Generation | Inserts generated frames into a sequence | Use the supplied 24→48 FPS example workflow |
A scale setting applies to both width and height. A 2× output therefore contains four times as many pixels; a 4× output contains sixteen times as many. Larger settings can substantially increase processing and storage requirements.
4. Preview and save the result
Inspect still images at 100% zoom, so you can judge actual pixels rather than the browser’s resized preview.
For video, inspect both individual frames and playback. An attractive still frame does not necessarily mean the entire clip is temporally consistent.
Unlike a game integration, this extension cannot recover genuine geometry, material buffers, exposure, jitter, or engine-authored motion from ordinary footage. Its estimated guides are useful approximations, not replacements for that information.
ComfyUI-DLSS5 Features
Easy one-node processing
Easy Upscale & Render is the simplest entry point. It provides presets for:
- Still images.
- Short videos.
- Long videos.
- Fast previews.
For an initial test, the author recommends these settings in the supplied Easy workflow:
Auto (recommended)Upscale + neural rendering2xQualityNeutral / faithful
These provide a reference point before you explore other controls. Test one change at a time so you can identify which setting caused a visible difference.
Easy uses Depth Anything V2 by default. The separate workflow 04 uses Video Depth Anything Small for temporally consistent depth.
Easy one-node workflow in ComfyUI
Super Resolution and neural-rendering controls
The DLSS Super Resolution node offers 2×–4× upscaling with color, depth, and motion guides.
The Experimental Neural Rendering node provides a same-resolution neural pass with style and structure controls. The combined DLSS SR + Neural Rendering node runs SR followed by NR.
For visual comparisons:
- Begin with a neutral treatment to establish a baseline.
- Compare alternative style settings against the original at the same zoom level.
- Check important details such as faces, text, edges, and repeating patterns.
- For video, look for changes over time as well as changes in individual frames.
The supplied images illustrate representative results, not guaranteed improvements for every source.
Original, neutral rendering, stronger style, and difference heatmap
Original, 2×, and 4× comparison
A difference heatmap shows where pixels changed, not whether the change improved the image.
Selectable depth guides
DLSS Depth Guide (Model Selection) provides model explanations and selectable depth backends, including DA-V2, optional YOLO26 Nano/Small, and VDA-S.
It also exposes FP16 and inference-size controls:
- FP16 uses lower numerical precision during model inference. Its suitability and benefits depend on the selected backend and hardware.
- Inference size controls the working size used for depth estimation. It is separate from the final SR output scale.
These controls let you experiment with guide-processing cost and detail. Use the fast depth selection and benchmarks to compare supported options rather than assuming one setting is always fastest or most accurate.
Motion and temporal depth tools
- RAFT Motion Guide estimates dense current-to-previous motion for reprojection.
- Temporal Depth Stabilizer reprojects and blends depth across frames.
- Video Depth Anything produces temporally consistent depth over video windows.
Reprojection means using estimated movement to align information from an earlier frame with the current one. When motion is estimated well, this can help keep guidance more consistent. Difficult motion or inaccurate alignment can still lead to trails or ghosting.
Full-sequence and bounded video processing
| Mode | How it works | When to consider it |
|---|---|---|
| Persistent full sequence | Keeps one native context for the complete batch, avoiding native chunk resets | Short or medium clips whose complete input and output fit in RAM |
| Bounded overlap-add | Processes native windows of chunk_size + history_overlap frames and blends overlapping regions | When you need a smaller native working set |
Bounded mode is not fully streaming video processing. The complete ComfyUI IMAGE input and output batches still occupy memory.
Easy’s long-video and fast-preview presets apply bounded windows to all selected operations.
A separate, optional persistent NR execution feature reuses matching NR initialization. This is distinct from choosing a full-sequence workflow: SR remains isolated, and the existing process execution mode remains the default. See persistent NR execution.
Progress, cancellation, and runtime reports
Progress reporting covers guide generation, native windows, and native frames. Some stages—particularly model downloads and VDA or FlashDepth calls—may have long intervals without visible updates.
Use ComfyUI’s interrupt control to cancel. SR/NR child processes are stopped before temporary files are cleaned up. VDA and FlashDepth check cancellation at stage boundaries, so an active model call may finish first.
Runtime reports show the resolved scenario, active settings, dimensions, frame counts, and an estimated temporary-storage requirement. That estimate excludes model weights, extra copies, and file overhead.
Optional Frame Generation
DLSS Frame Generation uses the separate open-source DLSSG-Stream-Worker to produce intermediate video frames.
This can increase the number of frames available for playback; it does not increase image resolution. The supplied example targets 24→48 FPS.
Important limitations include:
- It has separate runtime requirements and a dedicated status check.
- Hardware-accelerated GPU scheduling, or HAGS, is required by the worker.
- GPU and driver compatibility must match the supplied DLSS-G runtime.
- The documented v1 worker receives color and motion, but not real depth or camera matrices. It supplies constant depth and identity camera transforms.
- It cannot match a game integration with accurate scene and camera information.
ComfyUI-DLSS5 Models
The selectable models primarily create guides for processing. They are not a collection of interchangeable artistic image-generation checkpoints.
| Model or backend | Role | When to use or evaluate it |
|---|---|---|
| Depth Anything V2 Small / DA-V2 | Estimates depth from image pixels | The default Easy guide and a starting point for still images |
| YOLO26 Nano | Optional selectable depth-guide backend | Compare with other guide options using the documented benchmarks |
| YOLO26 Small | Optional selectable depth-guide backend | Evaluate alongside Nano and DA-V2 for your material and hardware |
| Video Depth Anything Small / VDA-S | Estimates temporally consistent depth for video windows | The author’s recommended temporally consistent depth workflow, supplied as workflow 04 |
| RAFT | Estimates motion between frames rather than depth | Video workflows that need current-to-previous motion guidance |
| FlashDepth | Optional high-resolution video depth backend | High-resolution depth workflows using its separate Python environment |
Depth Anything V2, VDA-S, and RAFT weights download on first use. The first guide-model run downloads Depth Anything V2 Small when using the default guide.
Changing a guide can change the structure information supplied to SR or NR, which may affect the output. The provided context does not establish one model as best for every image or clip.
For source links, model downloads, and licensing, consult runtime sources and licenses. For YOLO26 and selectable depth controls, see fast depth selection and benchmarks. FlashDepth details are documented separately in the FlashDepth guide.
ComfyUI-DLSS5 Workflow Guide
| Supplied workflow | Best starting use | Important notes |
|---|---|---|
| 00 — Easy one-node 2× | Your first still image or simple clip | One node selects the preset and operation |
| 01 — Guided still 2× | Direct control over depth, motion, and the full pipeline | Does not require VideoHelperSuite |
| 02 — Persistent video 2× | Short or medium clips that fit in memory | Includes a VideoHelperSuite loader and encoder |
| 03 — Bounded video 2× | Reducing the native video-processing working set | Complete ComfyUI batches still use RAM; includes an encoder |
| 04 — VDA-S temporal 2× | Temporally consistent video depth | Ends at Preview Image; add an encoder to export video |
| 05 — FlashDepth 2× | Optional high-resolution depth processing | Uses an isolated environment; ends at Preview Image |
| 06 — DLSS-G 24→48 | Optional frame generation | Uses the separate worker and includes an encoder |
All supplied video workflows use VideoHelperSuite for loading. See the workflow notes for defaults and dependencies.
Choosing a manageable video sample
Before processing a long clip, use VideoHelperSuite’s loader controls to select a short test:
frame_load_caplimits the number of frames loaded.skip_first_framesmoves the start point.force_sizeresizes the input.force_ratediscards or duplicates source frames to reach a chosen rate;0disables it.select_every_nthsamples frames at an interval.
Changing force_rate is not DLSS Frame Generation: it drops or duplicates frames rather than synthesizing new intermediate ones.
For ordinary processed video, the encoder’s frame_rate should match the intended processed sequence rate. In a frame-generation workflow, use the intended higher output rate. Increasing playback FPS without adding frames makes the clip shorter and faster.
VDA-S temporal workflow in ComfyUI
What’s New with ComfyUI-DLSS5
v0.5.0 — Experimental Preview
The documented update introduces:
- Selectable guide models: Optional Easy overrides and the model-selection depth node add YOLO26 Nano/Small choices alongside the documented depth options.
- Model explanations: Descriptions help you understand the purpose of the available guides.
- FP16 and inference-size controls: Additional options let you evaluate guide-generation settings for your hardware and source material.
- Optional persistent NR execution: Matching NR initialization can be reused. SR remains isolated, and the existing process mode stays the default.
These changes provide more control over guide selection and NR execution, but they do not establish real-time 4K performance. The author’s matched local measurement remains below 30 source FPS at 4K output.
See depth benchmarks and native-worker measurements and limits.
Troubleshooting ComfyUI-DLSS5
Check the supported environment first
The documented requirements are:
- Windows 10 or 11, 64-bit.
- A supported NVIDIA RTX GPU and a recent NVIDIA driver.
- A working ComfyUI environment with PyTorch/CUDA.
- At least 16 GB system RAM; 32 GB or more is recommended for video.
- Fast temporary storage with sufficient free space.
- The pinned VapourKit runtime.
Frame Generation has additional worker, GPU/runtime compatibility, and HAGS requirements.
Runtime Status reports FAIL, MISSING, or LFS POINTER
- Run DLSS 5 Runtime Status.
- Identify which check failed: files, Python interpreter, imports, or plugin probes.
- Remember that
PRESENTmeans only that a file exists. It does not prove the component works. - Use the matching instructions in the troubleshooting guide.
- Consult runtime documentation for verified runtime details and hashes.
Do not replace a missing DLL with an arbitrary download that merely has the same filename.
Runtime Status says Inference: UNTESTED
This is expected before an image has been processed.
- Run the Easy still workflow with a small test image.
- Check the output and console messages.
- If processing fails, follow the inference diagnostics in the troubleshooting guide.
A successful status probe is not certification of GPU inference or image quality.
A video run exceeds memory or temporary storage
- Reduce
frame_load_capand test a shorter section. - Lower the input resolution or upscale factor.
- Try Bounded overlap-add to reduce the native working set.
- Free additional temporary disk space.
- Review the runtime report’s storage estimate, allowing extra room for excluded overhead.
Remember that bounded mode still holds complete ComfyUI input and output batches in RAM.
Processing appears stuck
- Check whether the current stage is a first-use model download, VDA call, or FlashDepth call.
- Inspect the console for progress or errors.
- If needed, use ComfyUI’s interrupt control.
- Allow an active VDA or FlashDepth model call to reach its cancellation boundary.
- Follow the stuck-process guidance if processing does not recover.
You see ghosting, flicker, or depth trails
- Select a short section where the issue is easy to reproduce.
- Compare the original and processed frames at the same scale.
- Check the selected depth and motion guides.
- Compare with workflow 04 if temporal depth consistency is the concern.
- Consult the ghosting and depth-trail guidance.
Estimated guides cannot guarantee clean results for every motion pattern or source.
Video playback is too fast or too slow
- Check the loaded sequence’s frame rate and any
force_rateor frame-sampling changes. - Check Video Combine’s
frame_rate. - For normal processing, match the intended source-sequence rate.
- For Frame Generation, match the intended generated output rate—for example, the supplied 24→48 workflow.
Video Combine’s FPS controls playback timing; it does not create missing intermediate frames.
The workflow shows previews but saves no video
Workflows 04 and 05 end at Preview Image.
- Confirm which workflow you are using.
- Add a video encoder, such as VideoHelperSuite’s Video Combine.
- Connect the processed frames.
- Set the intended output frame rate, format, and save behavior.
Frame Generation does not run
- Run DLSS Frame Generation Runtime Status, not only the base status node.
- Check the worker and runtime diagnostics.
- Verify HAGS and compatible GPU/driver/runtime support.
- Confirm the workflow’s source and output FPS settings.
- Review the runtime documentation and worker documentation.
DLSS Frame Generation workflow in ComfyUI
Frequently asked questions
Does this work on macOS, Linux, or non-NVIDIA GPUs?
The documented extension is Windows-only and requires a supported NVIDIA RTX GPU.
Does a 4× setting mean four times as many video frames?
No. It multiplies width and height by four. Frame Generation is a separate feature.
Will it reproduce official in-game DLSS results?
Not necessarily. Ordinary images and videos lack the accurate scene buffers available to a game renderer.
Can I judge quality from an animated browser preview?
Use previews to navigate, but inspect the actual output too. VideoHelperSuite’s Advanced Previews are lower-quality viewing copies.
Can I share the exported media safely?
Inspect metadata first. Outputs can contain workflows, prompts, filenames, model names, and paths. Never publish runtime/config.json.
What should I include in a bug report?
Provide a redacted Runtime Status report, console traceback, workflow JSON, expected and actual frame count/resolution/FPS, and runtime hashes. Do not include private media or proprietary DLLs.
Learn More about ComfyUI-DLSS5
- Workflow notes: Defaults, workflow dependencies, and guidance for the supplied examples.
- Troubleshooting guide: Runtime failures, memory problems, ghosting, depth trails, and stuck processes.
- Depth selection and benchmarks: Guide-model choices and depth-processing controls.
- Native execution and measurements: Persistent NR behavior and measured limits.
- FlashDepth guide: The optional high-resolution depth backend.
- ComfyUI-VideoHelperSuite: Video loading, frame selection, previews, audio, and export.
- DLSSG-Stream-Worker: Optional Frame Generation capabilities and limitations.
- Runtime sources and licenses: Component provenance, model sources, and redistribution terms.
- Runtime reference: Runtime file details and verified hashes.
- Screenshots and examples: Additional interface captures.
- Asset provenance: History and attribution for supplied visual assets.
- Project repository: Source code and project updates.
ComfyUI-DLSS5 Licensing and Safety
The extension source code is GPL-3.0. NVIDIA components, model weights, VapourKit, VapourSynth, ComfyUI, worker binaries, and example media retain their own licenses. Review the runtime sources document before redistribution.
Native DLLs and workflows run with ComfyUI’s permissions. Review their origin, verify hashes where available, and inspect imported workflows before running them.
The project credits ComfyUI, VapourSynth, VapourKit, NVIDIA DLSS runtime components, TorchVision RAFT, Depth Anything, Video Depth Anything, FlashDepth, ComfyUI-VideoHelperSuite, and DLSSG-Stream-Worker.
The project is not affiliated with NVIDIA, ComfyUI, RenoDX, or VapourKit. DLSS and NVIDIA are trademarks of NVIDIA Corporation.
