Save 4 hours! We auto-setup your workflow! Free!

Drop your workflow.json — we handle every dependency, custom node, and model. Just open the link and run.

Auto-Setup Workflow Json (Free) Now!

ComfyUI Node: Local LLM

Class Name

LocalLLM

Category
🎨 DockerCPU API/Utilities
Author
trustypangolin (Account age: 2367days)
Extension
ComfyUI-API-DockerCPU
Latest Updated
2026-11-09
Github Stars
0.0K

How to Install ComfyUI-API-DockerCPU

Install this extension via the ComfyUI Manager by searching for ComfyUI-API-DockerCPU
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter ComfyUI-API-DockerCPU in the search bar
After installation, click the Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • 16GB VRAM to 80GB VRAM GPU machines
  • 400+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 200+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

Local LLM Description

Sends text or image prompts to local or OpenAI-compatible LLMs and returns generated answers.

Local LLM:

LocalLLM sends a request to a local or OpenAI-compatible language model service and returns the model’s answer for use elsewhere in your workflow. You can use it to draft image concepts, refine creative briefs, describe visual ideas, or interpret an image when the selected model supports image input. It supports two connection methods: a standard OpenAI-compatible chat API and LM Studio’s native API. The node can discover an available model, and the LM Studio method can manage loaded models before sending a request. The answer comes from the model service you connect to, so its quality and available capabilities depend on that service and model.

Local LLM Input Parameters:

endpoint_url

The address of the model service. Leave it empty, which is the default, to use the configured LOCAL_LLM_ENDPOINT environment value or, if none is available, http://localhost:1234. Enter an address when your service runs elsewhere. The node removes trailing slashes and sends the request to the API path appropriate for the selected connection method. An incorrect address prevents model discovery or the chat request from reaching the service. This string has no stated minimum or maximum length.

api_type

Chooses how the node communicates with the service. Options are openai_compatible (default), which sends a chat-completions request to an OpenAI-compatible API, and lmstudio_native, which uses LM Studio’s native API and can load or unload models. Choose the method your running service supports; a service that lacks the selected API may return an HTTP error. No other options are listed.

model

Selects the model that will answer. The default is auto: at execution time, the node asks the selected API for its models and uses the first one returned. Other choices are model names discovered from the default endpoint; the node’s tooltip says the list is refreshed in the UI from endpoint_url. A specific choice makes model selection more predictable, but it must be available to the service you contact. The list of model-name options varies by endpoint, so there is no fixed minimum or maximum value.

prompt

The main request you send to the model. Use it to describe the creative task and the form of answer you need, such as “Write three concise concepts for a moonlit garden illustration.” The default is an empty string; no length limit is stated. A clear prompt helps the model produce a more useful answer. The source does not specify how the connected service handles an empty prompt.

system_prompt

Optional background instructions sent separately from the main request, such as the role, tone, or response format you want the model to follow across a task. For example, you could ask for short, concrete descriptions suitable for image generation. The default is an empty string, and no length limit is stated. When it contains only whitespace, the node omits it from the request.

api_key

An optional credential override for a service that requires authorization. The default is an empty string, which allows the node’s credential resolution to use a configured environment value instead. When a key is available, the node sends it as a bearer authorization header. A key typed into this field is saved in the workflow and image metadata, so use an environment setting when you need to keep it out of saved workflow data. No minimum or maximum length is stated.

temperature

Controls the creativity or variability requested from the model. The default is 0.7; the allowed range is 0.0 to 2.0, in steps of 0.05. Lower values are useful when you want steadier, more constrained wording, while higher values may produce more varied ideas. The effect depends on the connected model and service, which receive this value in the request.

max_tokens

Sets the requested maximum size of the model’s generated answer. The default is 1024; the allowed range is 1 to 32768. Increase it for longer descriptions or multi-part creative briefs. If it is too low, the model may not have enough room to provide a complete answer; if it spends its available output on reasoning without producing answer text, the node raises a specific error. The service or model may impose its own limits.

timeout

Sets how many seconds the node waits for each network request before it times out. The default is 120 seconds; the allowed range is 1 to 3600 seconds. A longer timeout can help with slow local model loading or generation, but it also means waiting longer when a service does not respond. In LM Studio native mode, model-management requests can occur before the chat request.

unload_existing_models

Controls whether the node unloads existing LM Studio model instances before loading the selected model. The default is False; the options are False and True. It affects execution only with lmstudio_native. Turn it on when you deliberately need to clear loaded instances, bearing in mind that subsequent requests may need to load a model again. It is not used by openai_compatible.

force_rerun

Controls whether the node marks itself as changed on each evaluation. The default is False; the options are False and True. Turn it on when you need a fresh model request even if the other settings have not changed. Leave it off when you do not need repeated requests for unchanged settings.

image

An optional image supplied alongside the prompt. There is no default image and no stated size range. When connected, the node includes the image in the request; when absent, it sends a text-only request. Use it for tasks such as asking for a description or creative interpretation of an image. The selected model and service must support image input for the request to work.

reasoning

Controls the reasoning setting sent in lmstudio_native mode. Options are auto (default), off, low, medium, high, and on. With auto, the node omits the setting and lets the model use its default. The source notes that LM Studio can reject a reasoning setting a model does not support. This input has no effect on the OpenAI-compatible request.

Local LLM Output Parameters:

text

The answer text extracted from the service’s response. Connect this string to another node when you want to reuse a generated concept, description, or instruction. The node looks for answer text in supported response fields and raises an error if it finds none, rather than passing the raw response through as though it were an answer. What the text says depends on your prompt, the selected model, and the service.

model

The model identifier used for the request. If you chose auto, this identifies the first model returned by the selected API. In LM Studio native mode, the loading step may return an instance identifier, which becomes this output and is used in the chat request. This string helps you see which model or instance produced the accompanying text.

Local LLM Usage Tips:

  • Start with the connection method your service supports. Use openai_compatible for a compatible chat API; use lmstudio_native when you need the LM Studio native chat or model-loading behavior.
  • Select a specific model when consistency matters across runs. Use auto when choosing the first model listed by the service is acceptable.
  • Put the immediate creative task in prompt and reusable guidance, such as tone or answer structure, in system_prompt. State the desired length and format plainly.
  • Connect image only when the chosen model supports image requests. Describe what you want the model to do with the image in prompt.
  • Try a lower temperature for precise descriptions or revisions, and a higher value when exploring varied concepts. Increase max_tokens when the requested answer needs more space.
  • For a slow local service, increase timeout. Use unload_existing_models only when you intend to clear LM Studio’s loaded instances.
  • Leave api_key blank when an environment-configured credential is available, so a typed key is not stored in workflow or image metadata. Turn on force_rerun only when you need a new request from otherwise unchanged settings.

Local LLM Common Errors and Solutions:

Unsupported api_type: ...

  • Explanation: The connection method passed to the node is neither openai_compatible nor lmstudio_native.
  • Solution: Select one of the two listed api_type options and use the one your service supports.

No models returned for ... from ...

  • Explanation: model is set to auto, but the selected API returned no models, so the node cannot choose one.
  • Solution: Confirm that the service is running at endpoint_url and has a model available. Check api_type; alternatively, select a specific available model.

Connection error or connection refused

  • Explanation: The node could not reach the service, for example because it is stopped or the address is wrong.
  • Solution: Start the model service and check its host, port, and endpoint_url. If you left the field empty, confirm where its configured default points.

Request timed out

  • Explanation: A service request took longer than the timeout setting. Local model loading or generation may take time.
  • Solution: Confirm the service is responding, then increase timeout if the model genuinely needs longer. In LM Studio native mode, check whether model loading is taking time before chat begins.

HTTP 401 or 403

  • Explanation: The service rejected the request’s authorization. When the node receives an HTTP error, it includes the response body in the raised error if one is present.
  • Solution: Check whether the service requires a key and whether the configured or entered api_key is correct. Avoid putting a sensitive key in the field if an environment setting will work.

HTTP 404 or endpoint not found

  • Explanation: The service does not recognize the API path used for the selected api_type, or endpoint_url points to the wrong address.
  • Solution: Verify the service address and switch to the API method it supports. For LM Studio native mode, the service must provide its native API paths.

HTTP error indicating an unavailable model

  • Explanation: The service rejected the selected model name or could not make that model available.
  • Solution: Check the models available on the endpoint and select one it lists. Remember that the model choices initially come from the default endpoint and may differ from a custom endpoint_url.

HTTP error indicating unsupported image input

  • Explanation: An image was included, but the selected model or service may not accept images in the request format.
  • Solution: Try a model and service that support image input, or disconnect image and send a text-only request.

HTTP error indicating unsupported reasoning

  • Explanation: In lmstudio_native mode, a non-auto reasoning setting was sent, but the model may not support it.
  • Solution: Set reasoning to auto to use the model’s default, or choose a setting the model supports.

Model returned reasoning but no answer (... output tokens spent reasoning). Raise max_tokens or set reasoning to 'off'.

  • Explanation: The response reports reasoning output, but the node found no answer text to return.
  • Solution: Increase max_tokens or, in LM Studio native mode, set reasoning to off and try again.

No answer text in LLM response: ...

  • Explanation: The service returned a response, but the node could not extract answer text from it. The error includes the beginning of the response for diagnosis.
  • Solution: Inspect that response excerpt, confirm the chosen API method is compatible with the service, and try a straightforward text prompt. Check the service’s own logs if the response is still empty or unexpected.

Local LLM Related Nodes

Go back to the extension to check out more related nodes.
ComfyUI-API-DockerCPU
RunComfy
Copyright 2025 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals. RunComfy also provides AI Models, enabling artists to harness the latest AI tools to create incredible art.