SUPIR | Photo-Realistic Image/Video Upscaler

1. ComfyUI SUPIR for Image Resolution | ComfyUI Upscale Workflow

This ComfyUI Upscale workflow utilizes the SUPIR (Scaling-UP Image Restoration), a state-of-the-art open-source model designed for advanced image and video enhancement. In this workflow, you will experience how SUPIR restores and upscales images to achieve photo-realistic results.

2. ComfyUI SUPIR Overview

SUPIR, the forefront of image upscaling technology, is comparable to commercial software like Magnific and Topaz AI. Our tutorial encompasses the SUPIR upscaler wrapper node within the ComfyUI workflow, which is adept at upscaling and restoring realistic images and videos.

For image upscaling, this workflow's default setup will suffice. To modify it for video upscaling, switch from “load image” to “load video” and alter the output from “save image” to “combine video” to cater to video files.

3. Introduction to SUPIR Model

The Scaling-UP Image Restoration technology is a groundbreaking enhancement and upscaling model introduced by the paper Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild. SUPIR innovates with a photo-realistic image restoration method using a generative prior coupled with model scaling, enriched by multimodal techniques allowing image restoration guided by textual prompts, which broadens its application spectrum significantly.

4. How to Use ComfyUI SUPIR for Image Resolution

4.1. SUPIR Compatible Models

Before diving into SUPIR’s usage, ensure the checkpoint models are accessible:

• Two versions of SDXL CLIP Encoder from OpenAI and LAION, respectively.
• SDXL and LLaVA base models, crucial for the initial stages of image processing.
• Optional models like Juggernaut-XL versions which can replace the SDXL base in specific scenarios for enhanced photorealistic outcomes.

4.2. SUPIR Models

Two key versions of SUPIR are available:

• SUPIR-v0Q: Optimized for high generalization and quality, suitable for a wide range of images.
• SUPIR-v0F: Tailored for images with light degradation, preserving more details in such conditions

4.3. Key Parameters of SUPIR

• scale_by: The upscaling ratio for given inputs determines how much the image size is increased during restoration.
• steps: This parameter specifies the number of steps for the EDM Sampling Scheduler, likely influencing the detail and quality of the restoration process.
• cfg_scale: This is the classifier-free guidance scale for prompts, affecting how strongly the output adheres to the provided textual prompts.
• positive-prompt & negative_prompt: These parameters allow users to guide the restoration towards desired qualities (positive prompt) and away from undesired characteristics (negative prompt).
• s_churn & s_noise: Representing original hyperparameters of EDM, this controls aspects of the noise model within the diffusion process, impacting the final image texture and clarity.
• color_fix_type: This parameter allows for the selection of color correction methods post-restoration, with options including 'None', 'AdaIn', and 'Wavelet'.

4.4. Performance Tips for SUPIR

• Hardware Requirements: To achieve optimal results in higher resolution upscaling with the SUPIR upscaler, it is essential to have a sufficiently powerful hardware setup. We recommend using a machine equipped with at least 48GB of VRAM, such as the Extra Large Machine available at RunComfy, to handle the intensive computational demands of advanced image detailing.
• Maximizing Image Detail with Text Prompts: Furthermore, to maximize the potential of the SUPIR's advanced AI algorithms, make full use of the detailed prompt feature. This allows you to guide the restoration process more precisely, enhancing the detailing and realism of the upscaled images. By effectively leveraging these prompts, SUPIR can produce outputs that are not only larger in size but also superior in quality.

5. More Details About the SUPIR

Image restoration technology has grown tremendously, now delivering results that are visually stunning and smarter. This growth is largely due to the introduction of the SUPIR Upscaler, which utilizes advanced generative models to enhance images.

5.1. Core Capabilities of the SUPIR model

• Robust Models: The heart of the SUPIR Upscaler is the StableDiffusion-XL (SDXL), a powerful generative model with 2.6 billion parameters. It's supported by an adaptor model adding another 600 million parameters, enabling the SUPIR Upscaler to restore images with exceptional detail and fidelity.

5.2. Data-Driven Excellence

• Extensive Training Data: The SUPIR Upscaler is trained on a dataset of over 20 million high-quality images, each annotated with detailed descriptions. This dataset trains a 13-billion-parameter multi-modal language model, enhancing the SUPIR Upscaler’s capability to produce precise content prompts for targeted image restoration.

5.3. Innovative Technology and Strategic Implementation

• Advanced Design: The SUPIR Upscaler includes several strategic enhancements like the ZeroSFT connector, which improves efficiency and reduces computational demands. Additionally, its image encoder is fine-tuned to better handle image degradation, increasing the accuracy of restoration results.
• Comprehensive Training: Beyond high-quality images, the dataset also includes lower-quality, negative examples. This helps the SUPIR Upscaler learn to identify and correct visual flaws, enhancing the overall restoration quality.

5.4. Balancing Enhancement and Fidelity

• Sophisticated Techniques: Despite using generative models, the SUPIR Upscaler employs a novel sampling technique to balance the enhancement quality with the fidelity of the original images. This ensures that while the visual quality is boosted, the authenticity of the original images is preserved.

For a deeper dive into the SUPIR Upscaler’s capabilities and more technical details, explore resources on its GitHub page or the foundational research paper. These resources provide comprehensive insights into the technologies and strategies that establish the SUPIR Upscaler as a leader in image restoration.