comfyui-lmstudio Introduction
ComfyUI LM Studio Nodes overview
comfyui-lmstudio lets you use a model running in LM Studio (https://lmstudio.ai/) inside a ComfyUI workflow. You can ask the model to write text or, if it supports images, describe an image. The resulting text can then flow into other ComfyUI nodes.
For AI artists, a practical use is writing captions for images in a LoRA training dataset. You can also use it to draft descriptions or other text without sending requests to a cloud service or supplying a cloud API key. The extension provides an LM Studio Connection node and an LM Studio Generate Text node as an alternative to a CLIPLoader → TextGenerate pair.
How comfyui-lmstudio Works
Think of the two nodes as a connection and a request:
LM Studio Connectiontells ComfyUI where your running LM Studio server is and which loaded model to use.LM Studio Generate Textsends that model your prompt and, optionally, an image. It returns the model’s reply as ordinary text.
For example, you could connect an image from LoadImage, enter a prompt such as “Write a concise caption describing the visible subject, clothing, and setting,” and send the generated caption to a text preview or save node.
LoadImage ───────────────────────────────┐
▼
LM Studio Connection ──▶ LM Studio Generate Text ──▶ generated_text
├──▶ ShowText|pysssss
└──▶ SaveText|pysssss
ShowText|pysssss can display the reply in your workflow, while SaveText|pysssss can write it to a text file. Both are examples of downstream nodes: generated_text is a plain STRING output and can connect to any compatible string input.
LM Studio must have a model loaded and its server running. In LM Studio, the server is started from Developer → Start Server. The port shown there must match the address in the connection node; the default address uses port 1234.
comfyui-lmstudio Features
Connect to a local model
The LM Studio Connection node supplies these settings:
| Setting | What it means | When to change it |
|---|---|---|
base_url | The address of LM Studio’s compatible endpoint. The default is http://localhost:1234/v1. | Change it if LM Studio shows a different server address or port. |
model | The exact model ID shown in LM Studio’s Developer tab. | Enter an ID to request a particular model, or leave it blank to use the model currently loaded in LM Studio. |
api_key | A required, non-empty field that LM Studio ignores. | The default, lm-studio, is sufficient. |
timeout | How long the node waits for a request before giving up, in seconds. The default is 120. | Increase it if your local model regularly needs longer to respond. |
When the node connects, it checks LM Studio’s /v1/models endpoint. This helps reveal a stopped server or mismatched model name before a longer workflow reaches the generation step.
Generate text or describe an image
The LM Studio Generate Text node accepts:
| Input | What it does | Example |
|---|---|---|
connection | Uses the settings from LM Studio Connection. | Connect the connection node’s output here. |
prompt | Tells the model what you want it to produce. | “Describe this image in one or two factual sentences.” |
image (optional) | Sends an image along with your prompt. | Connect an IMAGE output from LoadImage for captioning; leave it unconnected for text-only requests. |
system_prompt (optional) | Provides an additional instruction about how the model should respond. | Ask for concise, objective captions across your workflow. |
max_tokens | Sets the maximum length available for the model’s response. | Allow more tokens for models that spend part of their response budget reasoning. |
temperature, top_p | Adjust how the model selects words. | Experiment with these when you want to change how varied its responses are. |
seed | Controls repeatability. | Use 0 for no fixed seed, or set a specific value when you want more reproducible results. |
Allow enough tokens for a clean caption
max_tokens is especially important for models with a “thinking” or reasoning mode. Such a model may use much of its allowance before writing the final caption. If it reaches the limit first, LM Studio can return an empty final answer. The node may then output the raw reasoning text instead and print a warning in the ComfyUI console. That output is unlikely to be suitable as a finished caption.
As a starting point:
- For a non-reasoning model,
500–1000tokens is typically enough for a one- or two-sentence caption. - For a reasoning or thinking model, try
4000–8000tokens. The amount it uses can vary from image to image. - If captions are slow and your model offers a way to turn thinking off in LM Studio’s settings or chat template, that may make response length and speed more predictable.
These are guidelines, not guarantees. Check the saved or displayed text before relying on captions for a dataset.
Understand the image limits
Only the first image in a batch is sent in each call. A batch of images therefore does not automatically produce one caption per image.
Each request also includes the full image encoded as a PNG. Sending and processing those images can take time, particularly on a modest local GPU.
comfyui-lmstudio Models
The extension does not supply its own models. It uses a model you run in LM Studio. You can enter that model’s exact ID in the connection node or leave model blank to use the model currently loaded.
Choose a model based on the task:
- For image captions: Use a vision-capable model. A text-only model may produce an error or ignore the image.
- For text generation without an image: Use a model suited to following written instructions.
- For quicker, short captions: A non-reasoning vision-instruct model may need fewer output tokens.
- For a model with thinking mode: Allow a larger
max_tokensvalue, or check whether thinking can be disabled if you do not need it.
The original documentation mentions plain Gemma 3/4 instruction-tuned models, vision-instruct models, reasoning-tuned Gemma models, Qwen thinking variants, and DeepSeek-R1 distills as examples of different response behaviors. These are not a list of included or guaranteed-compatible models. In particular, confirm that your chosen model supports images before using the image input.
Troubleshooting comfyui-lmstudio
The connection node says it cannot reach LM Studio
- Check that a model is loaded in LM Studio.
- In LM Studio, open Developer and start the server.
- Compare the server’s displayed port with
base_urlinLM Studio Connection. The default URL ishttp://localhost:1234/v1. - Run the workflow again. The connection node checks the server as soon as it connects, so a remaining connection error should appear promptly.
The model name does not match
- Find the model’s exact ID in LM Studio’s Developer tab.
- Enter that ID in the connection node’s
modelfield. - Alternatively, leave the field blank to use whichever model is currently loaded.
- Retry the connection.
The result is empty or contains reasoning instead of a caption
- Check the ComfyUI console for a warning.
- Raise
max_tokens; start at4000–8000for a thinking model. - If the model offers a thinking toggle or chat-template setting, consider disabling thinking for straightforward captioning.
- Generate again and inspect the
generated_textoutput before saving a large set of captions.
The model does not describe the image
- Confirm that an
IMAGEoutput is connected to the generate node’s optionalimageinput. - Confirm that the model loaded in LM Studio is vision-capable.
- Make the
promptexplicit about the description you want, such as a short caption of the visible contents.
A batch produces only one caption
This is the extension’s current behavior: one call uses only the first image in a batch. Passing a batch does not make the node process every image or create a separate text file for each one.
Responses take too long or time out
- Allow the local model time to process the image; each call sends a full PNG.
- Increase
timeoutfrom its default of120seconds if requests regularly need longer. - For short captions, check whether your model can run without thinking mode.
Frequently asked questions
Do I need a cloud API key?
No. The extension sends requests to your running LM Studio server. The connection node still requires a non-empty api_key value, but LM Studio ignores it; the default lm-studio is fine.
Can I use the node without an image?
Yes. Leave image unconnected and use prompt to request text.
Can I see a caption before saving it?
Yes. Connect generated_text to a compatible preview node such as ShowText|pysssss.
Will it automatically caption every image in a ComfyUI batch?
No. Each call sends only the batch’s first image.
Learn More about comfyui-lmstudio
- LM Studio (https://lmstudio.ai/) is the application that runs the model and provides the local server this extension uses.
- The overview image shows the extension’s nodes.
- The sample workflow image shows how to connect an image, generate a caption, and route the text to preview and save nodes.
