ComfyUI Node: Ollama Advanced Options

Ollama Advanced Options:

The Sage_OllamaAdvancedOptions node is designed to provide advanced configuration options for Ollama Language Learning Models (LLMs). This node is particularly useful for users who wish to fine-tune the behavior of their LLMs by adjusting various parameters that influence the model's output. By offering a range of customizable settings, this node allows you to control aspects such as the number of tokens to keep or predict, the randomness of the output, and penalties for repetition or frequency. These options enable you to tailor the model's responses to better suit specific tasks or creative projects, enhancing the flexibility and effectiveness of your AI-driven applications.

Ollama Advanced Options Input Parameters:

num_keep

The num_keep parameter specifies the number of tokens to retain from the input sequence. This can be useful for maintaining context in a conversation or text generation task. The value ranges from 0 to 100, with a default of 0, meaning no tokens are kept by default.

num_predict

The num_predict parameter determines the number of tokens the model should predict. A value of -1 allows the model to decide the number of tokens based on its internal logic, while you can specify a range from -1 to 2048 to control the length of the output. The default is -1.

top_k

The top_k parameter limits the sampling pool to the top k tokens, which can help in generating more coherent and focused outputs. It ranges from 1 to 1000, with a default value of 40, allowing you to balance between randomness and determinism.

top_p

The top_p parameter, also known as nucleus sampling, controls the cumulative probability threshold for token selection. It ranges from 0.0 to 1.0, with a default of 0.9, enabling you to adjust the diversity of the generated text by including only the most probable tokens until the threshold is met.

repeat_last_n

The repeat_last_n parameter sets the number of recent tokens to consider for repetition penalties. This helps in reducing redundancy in the output. The range is from 0 to 256, with a default of 64, allowing you to manage how much recent context affects repetition.

temperature

The temperature parameter influences the randomness of the model's predictions. A higher temperature results in more random outputs, while a lower temperature makes the output more deterministic. It ranges from 0.0 to 1.0, with a default of 0.8.

repeat_penalty

The repeat_penalty parameter applies a penalty to repeated tokens, discouraging the model from generating repetitive sequences. It ranges from 1.0 to 2.0, with a default of 1.1, providing control over the model's tendency to repeat itself.

presence_penalty

The presence_penalty parameter adjusts the likelihood of introducing new tokens that have not appeared in the text so far. It ranges from -2.0 to 2.0, with a default of 0.0, allowing you to encourage or discourage the introduction of new topics or ideas.

frequency_penalty

The frequency_penalty parameter penalizes tokens based on their frequency in the text, helping to reduce overuse of certain words or phrases. It ranges from -2.0 to 2.0, with a default of 0.0, enabling you to manage the balance between frequent and infrequent terms.

Ollama Advanced Options Output Parameters:

options

The options output parameter provides a dictionary of the configured settings for the Ollama LLM. This output is crucial as it encapsulates all the advanced options you have set, allowing for consistent application of these settings in subsequent nodes or processes. It ensures that the model operates with the desired configurations, enhancing the predictability and quality of the generated content.

Ollama Advanced Options Usage Tips:

• Experiment with the temperature and top_p parameters to find the right balance between creativity and coherence in your model's outputs.
• Use the repeat_penalty and frequency_penalty to manage repetition and ensure diverse language generation, especially in longer texts.
• Adjust num_predict based on the context and length of the desired output to optimize performance and relevance.

Ollama Advanced Options Common Errors and Solutions:

InvalidParameterValue

• Explanation: This error occurs when a parameter value is set outside its allowed range.
• Solution: Double-check the parameter values to ensure they fall within the specified minimum and maximum limits.

MissingOutputOptions

• Explanation: This error indicates that the options output was not generated, possibly due to misconfiguration.
• Solution: Review all input parameters to ensure they are correctly set and that the node is properly connected in the workflow.