ComfyUI Node: FisheyeDepthEstimator

FisheyeDepthEstimator:

The FisheyeDepthEstimator is a sophisticated node designed to generate a comprehensive 180-degree fisheye depth map from an image. This node is particularly beneficial for applications requiring a wide field of view, such as virtual reality or panoramic imaging. It achieves this by initially estimating the depth of the entire fisheye image and then extracting multiple 90-degree pinhole views from different orientations (front, left, right, up, and down). Each view undergoes depth estimation, and the results are reprojected back into the fisheye perspective. The node then renormalizes overlapping areas and merges all six depth maps using a selectable mode, ensuring a seamless and accurate depth representation. This process allows for a detailed and expansive depth map that captures the nuances of a scene from multiple angles, enhancing the depth perception and realism in digital art and imaging projects.

FisheyeDepthEstimator Input Parameters:

image

This parameter represents the input image for which the fisheye depth map is to be estimated. It is crucial as it serves as the basis for all subsequent depth calculations and transformations.

model_name

The model_name specifies the depth estimation model to be used. This choice impacts the accuracy and style of the depth map generated, with different models potentially offering varying levels of detail and performance.

depth_scale

The depth_scale parameter adjusts the scale of the depth values in the output. It has a default value of 1.0, with a range from 0.0 to 1000.0, allowing you to fine-tune the depth perception to suit specific artistic or technical needs.

pinhole_fov

This parameter defines the field of view for the pinhole camera projections, with a default of 90.0 degrees. It can range from 1.0 to 179.0 degrees, affecting how much of the scene is captured in each pinhole view and influencing the depth estimation process.

pinhole_resolution

The pinhole_resolution sets the resolution for the pinhole views, with a default of 1024 pixels and a minimum of 64 pixels. Higher resolutions provide more detail but require more computational resources.

fisheye_resolution

This parameter determines the resolution of the fisheye image, with a default of 4096 pixels and a range from 64 to 8192 pixels. It affects the overall detail and quality of the fisheye depth map, with higher resolutions offering finer detail.

mode

The mode parameter allows you to select the method for merging the depth maps from different views. Options include "SRC", "DST", "AVERAGE", "SOFTMERGE", and "DISTANCE_AWARE", with "AVERAGE" as the default. Each mode offers a different approach to blending the depth information, impacting the final output's smoothness and accuracy.

softmerge_radius

This parameter specifies the Gaussian radius used for merging depth maps in "SOFTMERGE" mode. It has a default value of 25 and a minimum of 1, influencing the blending smoothness between overlapping depth areas.

median_blur_kernel

The median_blur_kernel sets the kernel size for applying a median blur to the depth map, with a default of 1 and a range from 1 to 31. This helps reduce noise and artifacts in the depth map, enhancing its clarity and quality.

FisheyeDepthEstimator Output Parameters:

TENSOR

The TENSOR output is the resulting depth map, represented as a tensor with dimensions [B,H,W,1]. This depth map provides a detailed representation of the scene's depth information, capturing the spatial relationships and distances within the fisheye view.

MASK

The MASK output is a circular mask of the fisheye region, represented as a tensor with dimensions [B,H,W]. This mask delineates the valid area of the fisheye depth map, ensuring that only relevant depth information is considered in subsequent processing or visualization.

FisheyeDepthEstimator Usage Tips:

• To achieve the best results, choose a model_name that aligns with your specific depth estimation needs, as different models may offer varying levels of detail and accuracy.
• Adjust the depth_scale parameter to fine-tune the depth perception in your output, especially if the default scale does not meet your artistic or technical requirements.
• Experiment with different mode settings to find the optimal blending method for your project, as each mode offers unique advantages in terms of smoothness and accuracy.

FisheyeDepthEstimator Common Errors and Solutions:

"Invalid model name"

• Explanation: The specified model_name does not match any available models.
• Solution: Ensure that the model_name is correctly specified and matches one of the available models in your environment.

"Resolution too low"

• Explanation: The specified pinhole_resolution or fisheye_resolution is below the minimum required value.
• Solution: Increase the resolution to meet the minimum requirement of 64 pixels for both pinhole_resolution and fisheye_resolution.

"Invalid mode selection"

• Explanation: The mode parameter is set to a value that is not recognized.
• Solution: Verify that the mode is set to one of the valid options: "SRC", "DST", "AVERAGE", "SOFTMERGE", or "DISTANCE_AWARE".