ComfyUI Node: Ollama LLM Prompt (Vision) Refined

Ollama LLM Prompt (Vision) Refined:

The Sage_OllamaLLMPromptVisionRefine node is designed to enhance your interaction with language models by allowing you to send a prompt and receive a refined response. This node is particularly useful for those who wish to leverage the capabilities of multimodal language models that can process both text and images. By using this node, you can input a text prompt and optionally include images to enrich the context, which the model will use to generate an initial response. Furthermore, the node provides a refinement feature that takes the initial response and rewrites it to be more vivid, detailed, and engaging, while preserving the original meaning. This dual-step process ensures that you receive a polished and enhanced output, making it ideal for creative projects where nuanced and expressive language is desired. The node requires the model to be installed via Ollama, ensuring compatibility and ease of use within the Sage Utils framework.

Ollama LLM Prompt (Vision) Refined Input Parameters:

prompt

The prompt parameter is a text input where you provide the initial text that you want the language model to process. This can be a question, a statement, or any text that you wish to explore further with the model's capabilities. The default value is set to a predefined vision prompt, and it supports multiline input, allowing for complex and detailed prompts.

model

The model parameter allows you to select the specific language model you wish to use from a list of available options. This selection determines how the prompt will be processed and the nature of the initial response. The models are sorted for easy selection, ensuring you can quickly find the one that best suits your needs.

image

The image parameter is an optional input where you can provide one or more images to the model. If the selected model supports multimodal input, these images will be used alongside the text prompt to generate a more contextually rich response. This feature is particularly useful for tasks that benefit from visual context.

seed

The seed parameter is an integer input that allows you to set a specific seed for the random number generator used by the model. This can be useful for ensuring reproducibility of results. The default value is 0, with a minimum of 0 and a maximum of 2^32

• 1, allowing for a wide range of possible seeds.

refine_prompt

The refine_prompt parameter is a text input where you provide instructions for refining the initial response. The default instruction is to rewrite the text to be more vivid, detailed, and engaging while preserving the original meaning. This parameter supports multiline input, allowing for detailed refinement instructions.

refine_model

The refine_model parameter allows you to select the specific model to be used for refining the initial response. Like the model parameter, this selection is made from a sorted list of available options, ensuring compatibility and ease of use.

refine_seed

The refine_seed parameter is an integer input that sets a specific seed for the random number generator used during the refinement process. This ensures that the refinement process can be reproduced if needed. The default value is 0, with a minimum of 0 and a maximum of 2^32

• 1.

Ollama LLM Prompt (Vision) Refined Output Parameters:

initial_response

The initial_response output parameter provides the text generated by the language model based on the initial prompt and any provided images. This response is the direct output from the model before any refinement is applied, offering a raw look at how the model interprets the input.

refined_response

The refined_response output parameter delivers the enhanced version of the initial response. This text has been processed according to the refine_prompt instructions, resulting in a more vivid, detailed, and engaging output. This refined response is ideal for applications requiring polished and expressive language.

Ollama LLM Prompt (Vision) Refined Usage Tips:

• To achieve the best results, ensure that your initial prompt is clear and provides enough context for the model to generate a meaningful response.
• Utilize the image parameter to provide additional context when using models that support multimodal input, as this can significantly enhance the quality of the response.
• Experiment with different refine_prompt instructions to tailor the refinement process to your specific needs, whether you require more descriptive language or a particular tone.

Ollama LLM Prompt (Vision) Refined Common Errors and Solutions:

Model not available

• Explanation: This error occurs when the selected model is not available or not installed via Ollama.
• Solution: Ensure that the model is correctly installed and available in the list of options. Check your installation of Ollama for any issues.

Image input not supported

• Explanation: This error arises when an image is provided, but the selected model does not support multimodal input.
• Solution: Verify that the chosen model supports image input. If not, either select a compatible model or remove the image input.

Refinement model not available

• Explanation: This error indicates that the selected refinement model is not available.
• Solution: Check the list of available refinement models and ensure that the selected model is correctly installed and accessible.