Save 4 hours! We auto-setup your workflow! Free!

Drop your workflow.json — we handle every dependency, custom node, and model. Just open the link and run.

Auto-Setup Workflow Json (Free) Now!
ComfyUI > Nodes > kandinsky-s2v > Kandinsky S2V Encode (speech + prompt)

ComfyUI Node: Kandinsky S2V Encode (speech + prompt)

Class Name

KandinskyS2VEncode

Category
Kandinsky S2V
Author
kandinskylab (Account age: 362days)
Extension
kandinsky-s2v
Latest Updated
2026-11-05
Github Stars
0.04K

How to Install kandinsky-s2v

Install this extension via the ComfyUI Manager by searching for kandinsky-s2v
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter kandinsky-s2v in the search bar
After installation, click the Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • 16GB VRAM to 80GB VRAM GPU machines
  • 400+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 200+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

Kandinsky S2V Encode (speech + prompt) Description

Encodes speech and text into conditioning for Kandinsky S2V speech-driven video generation.

Kandinsky S2V Encode (speech + prompt):

KandinskyS2VEncode prepares speech and descriptive text for video generation with Kandinsky S2V. It combines Qwen2.5-Omni’s speech and text representations with a pooled CLIP-L text representation, giving the video model complementary information about the spoken performance and the scene you describe. Rather than generating images itself, it translates this material into numerical guidance that the Kandinsky S2V sampling process can use. It also organizes the speech into fixed-duration sections, padding the final section with silence when necessary, and preserves duration information so that the extra video frames can be removed later. This preparation helps you build a speech-driven avatar workflow while keeping text guidance, speech processing, and clip length coordinated.

Kandinsky S2V Encode (speech + prompt) Input Parameters:

encoder

A required connection of type KANDINSKY_S2V_ENCODER, supplied by Kandinsky S2V Encoder Loader. This provides the Qwen2.5-Omni conditioning encoder that converts your prompt and speech into representations the video model can understand. The loader uses the Qwen2.5-Omni Thinker component; its speech-generating Talker component is not needed for this workflow. The encoder also determines the maximum audio duration its feature extractor can process, which this node checks before encoding. There is no numeric minimum, maximum, default value, or selectable option for this connection; connect the compatible encoder rather than a standard CLIP encoder or diffusion model.

clip

A required CLIP connection providing CLIP-L alone. Use the stock Load CLIP node with clip_l.safetensors and its type set to stable_diffusion, as specified by the node’s tooltip and validation. This encoder creates an additional compact text representation that complements Qwen2.5-Omni’s text features. The implementation uses CLIP-L’s tokenizer directly, truncates text to the configured CLIP token limit, and encodes tokens with uniform weights; it does not use ComfyUI’s usual long-prompt window splitting or emphasis parsing for this CLIP-L path. There is no numeric range or default connection. A different CLIP architecture or a combined encoder setup can fail the compatibility check even if its socket type is also CLIP.

audio

A required AUDIO connection containing the speech that drives the avatar, typically supplied by LoadAudio. The node reads the waveform and sample rate, calculates the original duration, and prepares the speech for the requested number of generation chunks. It accepts exactly one audio batch entry and rejects an empty track; this batch restriction does not itself mean that your file must have only one audio channel. Speech preparation resamples the audio and fits it to the selected processing duration, with silence padding when the selected duration extends beyond the recording. There is no default recording or fixed numeric duration range declared on this input. The effective duration limit depends on the encoder’s maximum supported audio length and the selected number of chunks.

prompt

A required multiline STRING describing the intended video content. The same text is processed through Qwen2.5-Omni and CLIP-L, so it provides descriptive guidance alongside the speech-driven information. You can use it to describe the subject, setting, appearance, and visible behavior you want the model to depict, without treating the text as a guarantee of exact results. Its default is contract.PROMPT; the supplied source does not show that constant’s actual text. No character-based minimum or maximum is declared, but the resulting Qwen2.5-Omni text representation must fit the model’s configured text-position limit or the node raises an error. The CLIP-L path separately truncates to its configured token limit, so a very long prompt may not be represented in full by both encoders.

chunks

A required INT controlling how many fixed-duration sections of speech are prepared for generation. The default is 0, the minimum is 0, and the maximum exposed by the interface is 60. Setting it to 0 automatically chooses enough chunks to cover the entire speech track, rounding up to a whole chunk. Setting a positive value requests exactly that number: fewer chunks than needed processes only the beginning of the recording, while more chunks than needed extends the prepared duration with silence. Each chunk lasts contract.CHUNK_SECONDS seconds; its numerical value is not included in the supplied context. The total requested duration must also fit the encoder’s maximum audio duration, so a value accepted by the interface can still be rejected during execution.

Kandinsky S2V Encode (speech + prompt) Output Parameters:

conditioning

A single output of type KANDINSKY_S2V_CONDITIONING containing the prepared guidance and timing information for the Kandinsky S2V workflow. Internally, it includes Qwen2.5-Omni text features as text_embeds, a pooled CLIP-L text representation as pooled_embed, speech features as audio_embeds, the resolved generation count as chunks, and the usable recording duration as track_seconds. The feature tensors are moved to ComfyUI’s intermediate device and stored in bfloat16 format. The duration is the shorter of the original recording and the requested processing length, which lets the trimming stage distinguish useful footage from silence-padded footage. Connect this output to Kandinsky S2V Sampler to guide generation and to Kandinsky S2V Trim To Audio to remove the padded tail after decoding; it is a custom conditioning package, not a standard ComfyUI CONDITIONING output.

Kandinsky S2V Encode (speech + prompt) Usage Tips:

  • Set chunks to 0 when you want to cover the complete recording. Automatic coverage still has to fit the encoder’s audio-duration limit.
  • For an initial test, set chunks to 1 to prepare only the first chunk. This reduces the amount of video you need to sample while checking your prompt, reference image, and overall workflow.
  • Keep Load CLIP configured with clip_l.safetensors and type stable_diffusion. Connect Qwen2.5-Omni through Kandinsky S2V Encoder Loader, not through the clip socket.
  • Keep prompts focused and put important visual details early. The CLIP-L representation is truncated at its configured token limit, and excessive Qwen2.5-Omni prompt length causes an explicit error.
  • Do not rely on ComfyUI-style emphasis syntax such as (expression:1.2) to apply CLIP-L token weights here. This node bypasses that emphasis parser and assigns uniform token weights.
  • Send decoded frames and the same conditioning output to Kandinsky S2V Trim To Audio before assembling the final video. Padding is added during preparation, but this encoding node does not remove generated frames itself.
  • If you intentionally generate only the beginning of a recording, make sure the audio used in your final video assembly matches that shorter result rather than assuming the entire original track was processed.
  • If the requested duration exceeds the encoder limit, trim or split the recording into shorter tracks, or choose fewer chunks. Increasing the chunk count does not bypass the feature extractor’s duration limit.

Kandinsky S2V Encode (speech + prompt) Common Errors and Solutions:

Kandinsky S2V takes one speech track, got a batch of N

  • Explanation: The audio waveform contains more than one batch entry. The node checks the first waveform dimension and accepts exactly one track at a time; this error is about batching, not simply stereo versus mono audio.
  • Solution: Supply a single recording through LoadAudio or process each batch entry separately. If another node creates audio batches, select one entry before connecting it here.

The speech track is empty

  • Explanation: The waveform has no audio samples, so its calculated duration is zero or less. There is no speech material for the node to encode.
  • Solution: Check that the audio file loads and plays correctly. If you trimmed or cropped it upstream, ensure the selected interval contains audio samples.

Kandinsky S2V needs CLIP-L alone: load clip_l.safetensors with Load CLIP, type stable_diffusion

  • Explanation: The connected CLIP object does not have the standalone CLIP-L tokenizer and model structure expected by this node.
  • Solution: Load clip_l.safetensors through Load CLIP, set its type to stable_diffusion, and connect it directly to clip. Keep the Qwen2.5-Omni encoder on the separate encoder connection.

N chunks need T s of speech, but the Qwen2.5-Omni feature extractor truncates at M s; trim the track or set fewer chunks

  • Explanation: The selected chunk count requires more audio-processing time than the encoder supports. This check uses the complete chunk-aligned duration, including silence padding, rather than only the original recording length. The message substitutes the actual chunk count and durations for N, T, and M.
  • Solution: Reduce chunks or shorten the recording. For long recordings, process shorter portions separately. If chunks is 0, automatic full-track coverage can still exceed the limit.

The prompt is N Qwen2.5-Omni tokens long; Kandinsky S2V takes at most M. Shorten the prompt.

  • Explanation: The encoded prompt exceeds the video model’s supported text-position limit. Tokens are pieces of text rather than a direct count of words or characters, so the limit cannot be reliably managed by word count alone.
  • Solution: Shorten the prompt by removing repeated descriptions and less important details. Keep the essential subject, setting, and appearance, then run the node again.

CUDA out of memory

  • Explanation: This is a possible underlying PyTorch or device error rather than a custom validation message from this node. The encoding step requests a full load of the Qwen2.5-Omni encoder, which can fail if available GPU memory is insufficient.
  • Solution: Close other GPU-heavy applications and use ComfyUI’s memory-management or offloading settings where appropriate. Try a shorter speech segment and a smaller chunk count to reduce duration-dependent processing demands, but remember that these changes do not reduce the encoder’s model-weight memory requirements.

Kandinsky S2V Encode (speech + prompt) Related Nodes

Go back to the extension to check out more related nodes.
kandinsky-s2v
RunComfy
Copyright 2025 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals. RunComfy also provides AI Models, enabling artists to harness the latest AI tools to create incredible art.

Kandinsky S2V Encode (speech + prompt)