LTX 2.3 Character LoRA Training: How to Get Consistent Characters Without Relying on Old LoRAs

If you want LTX 2.3 character LoRA training to produce a character that stays recognizable across scenes, you need to start from the right assumptions.

The most common mistake right now is treating older LTX 2.0 character LoRAs as a starting point. They load, they produce something, but the quality is degraded enough that you will waste time debugging output that was broken from the start.

This guide covers what it actually takes to do LTX 2.3 character LoRA training from scratch: when to retrain instead of reuse, how to judge if a character LoRA is working, how to handle multi-character setups, and what dataset decisions matter most.

By the end, you will know:

• why old character LoRAs do not transfer reliably to LTX 2.3
• how to build a dataset for LTX 2.3 character LoRA training
• what training recipe works for single vs multi-character LoRAs
• how to tell if your character LoRA is actually working on 2.3
• how to combine character LoRAs with style on the same model

Start with the LTX 2.3 LoRA training guide if you need help with the basic checkpoint and environment setup first.

---

Table of contents

• 1. Why old character LoRAs fail on LTX 2.3
• 2. When to retrain vs when to reuse
• 3. Building a dataset for LTX 2.3 character LoRA training
• 4. Single character training recipe
• 5. Multi-character training recipe
• 6. How to judge if your character LoRA works on 2.3
• 7. Character plus style: what works and what breaks
• 8. When to use RunComfy Cloud for LTX 2.3 character LoRA training
• 9. Bottom line

---

1. Why old character LoRAs fail on LTX 2.3

LTX 2.3 is a 22B parameter model. LTX 2.0 was 19B. The architecture changed enough that LoRA weights trained on the old model do not map cleanly onto the new one.

Community testing confirms this clearly. Users who continued using old LoRAs on LTX 2.3 report that the old weights "completely ruined the results" compared to LoRAs retrained on the new model.

What happens in practice:

• Old LoRAs load without errors. The failure is silent.
• Output looks plausible at first glance, but character consistency, motion quality, and detail are all worse than what the base model can do without any LoRA.
• The degradation is subtle enough that you might spend hours adjusting settings before realizing the LoRA itself is the problem.

This is why training from scratch on 2.3 is not optional if you care about quality. Old weights are not a reliable shortcut.

---

2. When to retrain vs when to reuse

Not every old LoRA needs immediate retraining. Here is the practical split.

2.1 Retrain when

• The LoRA is a character LoRA where identity consistency matters.
• The LoRA was trained on the 19B architecture.
• You need the LoRA to interact well with 2.3's improved motion and audio capabilities.
• The output quality from your old LoRA on 2.3 is noticeably worse than the bare model.

2.2 Possibly reuse when

• The LoRA is a simple helper LoRA (e.g., NSFW adjustments) used at very low weight (0.2–0.4).
• You are testing quickly and do not need production quality.
• The LoRA's contribution is subtle enough that quality loss is acceptable.

2.3 The default recommendation

If in doubt, retrain. The cost of a LTX 2.3 character LoRA training run is lower than the cost of debugging subtle quality problems that turn out to be caused by incompatible old weights.

---

3. Building a dataset for LTX 2.3 character LoRA training

The dataset is where most character LoRA quality is won or lost.

3.1 Use video clips, not images

This is specific to LTX. Images are not good training data for this model. They make training slower, consume more resources, and produce worse results than equivalent video data. If you only have still images, convert them to short clips, but prefer native video whenever possible.

Clips of 9–25 frames work well for most use cases. Longer clips (up to 49 or 121 frames) are fine but require more VRAM.

3.2 Cover the failure cases

A character LoRA's job is keeping identity consistent under change. Build your dataset around the variations the model needs to survive:

• Different expressions (neutral, smiling, talking, surprised)
• Different head angles (front, three-quarter, profile)
• Different lighting conditions
• Different body framing (close-up, medium shot, full body)

If all your training data shows the same angle and expression, the LoRA will break the moment you prompt for anything different.

3.3 Keep the identity signal clean

Use footage of one consistent character per LoRA. Avoid:

• Mixing photo studio shots with video frames — the LoRA will learn something in between that matches neither.
• Low-quality compressed video where the face is unreadable.
• Clips where motion blur destroys facial detail.
• Clips where the character's face is too small in the frame.

3.4 Caption with a unique trigger word

Every clip should include a unique trigger word for the character in its caption. Keep captions simple and consistent. Describe what varies (scene, action, expression), not every facial feature. The LoRA should learn the identity from the visual data, not from the caption.

3.5 Audio considerations

If your clips include the character speaking, this is valuable — LTX 2.3 can learn voice characteristics alongside visual identity. But the audio must be clean: no background music, no overlapping speakers, no heavy noise. Dirty audio teaches the LoRA to associate noise with the character.

---

4. Single character training recipe

For a single character, the setup is straightforward.

4.1 Recommended settings

4.2 Multi-stage approach

Some community members use a staged training approach for stronger character lock:

The staged approach is more work but produces stronger results when facial consistency is the primary goal.

4.3 What to watch for

• Audio trains faster than video. If you include audio data, check audio convergence separately.
• Lower learning rates help prevent aggressive overwriting of the base model's motion quality.
• Checkpoint every 200–500 steps and evaluate consistently.

---

5. Multi-character training recipe

Training multiple characters in a single LoRA is possible but requires more careful dataset design.

5.1 Proven approach

The most successful community example used ~440 video clips from a single visual source, covering 6 distinct characters with a shared art style. The key design decisions:

• Rank 64 (higher than single-character, to accommodate more concepts)
• Per-character trigger words (e.g., char_bb, char_rr) — every clip caption includes the specific character's trigger
• Weighted dataset balancing — more clips for priority characters, fewer for secondary ones
• No still images — video clips only
• Consistent captioning with a detailed system prompt for the captioning model
• Checkpoint every 500 steps — with the same test prompts and seeds for comparison

5.2 Dataset size for multi-character

Expect to need significantly more data than single-character training. The 440-clip, 6-character example trained for ~31K steps over ~48 hours on an RTX 5090. This is a large run, but the results showed minimal character bleed between trigger words.

5.3 Character bleed

The main risk of multi-character LoRAs is characters leaking into each other. Prevention comes from:

• Strong per-character trigger words used consistently in captions
• Balanced dataset so one character does not dominate
• Descriptive captions that clearly identify which character appears

If you see character bleed, the fix is usually in the captioning, not the training settings.

---

6. How to judge if your character LoRA works on 2.3

A character LoRA "working" means more than producing a video. It means the character is recognizably consistent across different prompts, expressions, and scenes.

6.1 Test protocol

For every checkpoint you want to evaluate:

1. Use 3–4 fixed prompts that cover different scenarios (talking, moving, different scene).
2. Use the same seed and inference settings for every test.
3. Compare to the base model without the LoRA — the LoRA should improve consistency, not degrade quality.
4. Compare across checkpoints — later is not always better.

6.2 What "working" looks like

• The character's face is recognizable across prompts.
• Expression changes (smiling, talking, turning) do not break identity.
• The base model's motion quality is preserved — not stiffer or more artificial.
• If audio is trained, the character's voice is consistent and not garbled.

6.3 What "not working" looks like

• The character looks different in every output.
• The face morphs when the head turns or the expression changes.
• Video quality is worse than the base model without the LoRA.
• Motion is stiffer or more repetitive than the base model.

If the LoRA degrades base model quality, the issue is usually: (a) old weights from a 2.0 LoRA, (b) too-high learning rate, or (c) dataset problems (too few angles, too few expressions, or mixed source types).

---

7. Character plus style: what works and what breaks

7.1 A combined LoRA can work

The community has shown that a single LoRA can carry both character identity and visual style. This works best when all training data shares a consistent art direction — for example, all clips come from the same game or animation.

In that case, the style is implicit in the data and does not need a separate training objective.

7.2 When it breaks

Problems appear when:

• You mix photorealistic footage with stylized animation in the same dataset.
• You expect the LoRA to generalize a style to new characters it has not seen.
• The style conflicts with the base model's motion priors — for example, 2D animation with hard lines on a model trained primarily on live-action footage.

7.3 Fast motion is a base model issue

If your character LoRA produces artifacts during fast motion, this is usually a LTX 2.3 base model limitation, not a LoRA problem. The model's motion quality under fast movement is still a known weakness. Do not try to fix this by adjusting training settings — it is not caused by the LoRA.

---

8. When to use RunComfy Cloud for LTX 2.3 character LoRA training

Character LoRA training requires iterating on datasets and checkpoints. Each experiment means downloading large models, preprocessing video, and running training jobs that need 32GB+ VRAM.

Use RunComfy Cloud AI Toolkit when:

• You want to test different dataset designs without managing local model files.
• Your local GPU is under 32GB and low-VRAM optimizations are too fragile for reliable iteration.
• You need to compare multiple checkpoints quickly and do not want to rebuild the environment each time.
• Your goal is a working character LoRA, not proving that local training can be done.

The most expensive part of LTX 2.3 character LoRA training is usually the iteration cycle — not the raw GPU cost. Moving to cloud shortens that cycle.

RunComfy is also adding LTX 2.3 LoRA training support to the Cloud AI Toolkit soon — including character LoRA workflows. Once available, you will be able to run multi-stage character training, compare checkpoints, and iterate on datasets without managing the local LTX Trainer environment.

Open it here: RunComfy Cloud AI Toolkit

---

9. Bottom line

To get a character LoRA that actually works on LTX 2.3:

1. Retrain from scratch instead of reusing old 2.0 weights.
2. Use video clips, not images — LTX trains better on video.
3. Cover the failure cases in your dataset: different angles, expressions, lighting.
4. Use per-character trigger words and keep captions consistent.
5. Evaluate with fixed prompts and seeds across checkpoints.

The current state of LTX 2.3 character LoRA training is that the tooling works, the results can be strong, but the documentation is still catching up. Community supply of trained LoRAs is still low — if you train a good one, you are likely ahead of most people working with this model.