ComfyUI Node: Metric3D Normal Map

Metric3D Normal Map:

The Metric3D-NormalMapPreprocessor node is designed to generate high-quality normal maps from input images, leveraging advanced deep learning models. This node is particularly useful for AI artists who need to extract detailed surface normals for 3D reconstruction, rendering, or other visual effects. By utilizing a pre-trained Metric3D model, the node ensures accurate and efficient normal map generation, which can significantly enhance the realism and depth of your digital artwork. The node's primary function is to process an input image and output a corresponding normal map, which represents the orientation of surfaces within the image. This can be particularly beneficial for applications in computer graphics, game development, and virtual reality, where understanding the geometry of a scene is crucial.

Metric3D Normal Map Input Parameters:

backbone

The backbone parameter specifies the architecture of the pre-trained model to be used for normal map generation. You can choose from three options: "vit-small", "vit-large", and "vit-giant2". The default value is "vit-small". The choice of backbone affects the model's capacity and performance, with larger models generally providing more accurate results but requiring more computational resources.

fx

The fx parameter represents the focal length of the camera in the x-direction. It is an integer value with a default of 1000, a minimum of 1, and a maximum defined by MAX_RESOLUTION. This parameter influences the scaling of the input image and can affect the accuracy of the normal map generation.

fy

The fy parameter represents the focal length of the camera in the y-direction. Similar to fx, it is an integer value with a default of 1000, a minimum of 1, and a maximum defined by MAX_RESOLUTION. Adjusting this parameter can help fine-tune the normal map output based on the specific characteristics of the input image.

Metric3D Normal Map Output Parameters:

IMAGE

The output of the Metric3D-NormalMapPreprocessor node is an IMAGE parameter, which contains the generated normal map. This normal map is a visual representation of the surface normals of the input image, encoded in RGB format. Each pixel in the normal map corresponds to a surface normal vector, which indicates the direction perpendicular to the surface at that point. This output can be used in various applications, such as enhancing the realism of 3D models, improving lighting effects, and facilitating more accurate 3D reconstructions.

Metric3D Normal Map Usage Tips:

• To achieve the best results, choose the appropriate backbone model based on your computational resources and the desired accuracy. For most applications, "vit-small" provides a good balance between performance and accuracy.
• Adjust the fx and fy parameters to match the characteristics of the camera used to capture the input image. This can help improve the accuracy of the normal map generation.

Metric3D Normal Map Common Errors and Solutions:

Model file not found

• Explanation: The specified pre-trained model file could not be located.
• Solution: Ensure that the model file metric_depth_<backbone>_800k.pth is correctly placed in the expected directory and that the backbone parameter is set correctly.

CUDA out of memory

• Explanation: The GPU does not have enough memory to load the model or process the image.
• Solution: Try reducing the resolution of the input image or using a smaller backbone model. Alternatively, you can run the process on a machine with more GPU memory.

Invalid input image

• Explanation: The input image is not in a valid format or is corrupted.
• Solution: Verify that the input image is correctly formatted and not corrupted. Supported formats typically include JPEG, PNG, and BMP.

Unexpected keyword argument 'depth_and_normal'

• Explanation: The function call includes an unexpected keyword argument.
• Solution: Ensure that the function signature matches the expected parameters and that no extra arguments are passed. This might require checking the implementation of common_annotator_call.

Torch device not found

• Explanation: The specified Torch device (e.g., GPU) is not available.
• Solution: Verify that the machine has the required hardware and that the correct device is specified in the code. You can check the available devices using torch.cuda.is_available().