LTX 2.3 Trainer — Keyframe Interpolation ()
Overview
The endpoint trains a LoRA for the LTX 2.3 model that generates the video between keyframes. During training, the first and last frames (and optionally a middle frame) of each clip are kept clean and the model learns to generate the in-between motion. At inference you supply a start image and an end image (and optionally a middle image), and the model produces a smooth video connecting them.
Key features:
- Learns first + last (optionally + middle) keyframe → video interpolation.
- Optional middle keyframe for first+middle+last interpolation.
- Trains on plain video clips (plus optional captions) — keyframes are taken from each clip automatically.
- Video-only (no audio is learned).
Provide a single archive (linked via ) of plain videos:
- Videos: , , , ,
- Captions: a with the same base name as each video (optional but recommended).
Images are rejected (there is nothing to interpolate over time on a still). The first and last frames of each clip (and the middle, if enabled) become the kept keyframes; the model learns to generate the rest. Aim for at least 10 clips. File names must be unique across the archive.
Minimum clip length: with off (the default), each video should have at least frames (default 89, ≈ 3.7 s at 24 fps). Shorter clips are dropped during preprocessing; if no clip is long enough, the run fails with no usable training data. Turn on to resample shorter clips to the target frame count instead.
Dataset
(required)
Type:
URL to the archive of videos.
Type:
Default:
Phrase prepended to captions during training; include it at inference.
Training Parameters
Type: (, , , , )
Default:
LoRA capacity.
Type:
Default: (range –)
Number of optimization steps.
Type:
Default:
Optimization step size.
Type:
Default:
When true, a middle keyframe is also kept (first + middle + last → video) instead of just first + last. When enabled, every validation sample must also provide a middle keyframe image.
Video Configuration
Type:
Default: (range –)
Frames per training clip. Must satisfy ; other values are snapped down to the nearest valid count.
Type:
Default: (range –)
Target frames per second.
Type: (, , )
Default:
| Resolution | 16:9 | 1:1 | 9:16 |
|---|
| low | 512×288 | 512×512 | 288×512 |
| medium | 768×448 | 768×768 | 448×768 |
| high | 960×544 | 960×960 | 544×960 |
Type: (, , )
Default:
Type:
Default:
Fit videos to the target frame count and frame rate.
Type:
Default:
Split long clips into scenes before training.
Type:
Default: (range –)
Duration above which a clip is eligible for scene splitting.
Validation
Type:
Default: (max 2 entries)
Validation samples, each an object with:
- () — the text prompt.
- (, required) — the first-frame keyframe image.
- (, required) — the last-frame keyframe image.
- (, optional) — a middle keyframe image; required when is true.
Type:
Default: a built-in quality negative prompt.
Type:
Default: (range –)
Type:
Default: (range –)
Type:
Default:
Type:
Default:
Type:
Default: (range –)
Type:
Default:
Return an archive of the preprocessed data for inspection.
Outputs
- — the trained LoRA weights ().
- — JSON describing the trigger phrase and training type.
- — combined validation reel (when validation samples were provided).
- — preprocessed-data archive, only when is enabled.
Billing
A successful run is billed billable units. Requests that fail before training completes (input-validation errors / HTTP 422, or dataset-download failures) are billed 0 units.
How the Training Works
Pipeline Overview
- Preprocessing — the archive is extracted, clips fit to the resolution bucket (optionally scene-split), and a temporal pattern is built per clip that keeps the first and last (and optionally middle) frames clean.
- Training — the kept keyframes are held fixed and the model learns to generate the in-between frames. Validation previews run at intervals.
- Output — the LoRA, config, and validation reel are uploaded.
What Happens to Your Data
- Archive extraction: the is unpacked; macOS metadata and hidden files are ignored.
- Video fitting: clips are resized to fill the resolution bucket and center-cropped; with they are resampled to the target frame rate/count.
- Keyframe selection: the first and last frames (and the middle, if enabled) of each clip are kept clean as the keyframes; the rest is the generation target. This happens automatically.
- Captions: the trigger phrase (if set) is prepended.
How Interpolation Training Works
The model is trained to fill in the motion between fixed keyframes. The trained LoRA specializes the base model at producing smooth, plausible transitions consistent with your footage. At inference you supply the start and end images (and optionally a middle image) plus a prompt.
Tips for Getting Good Results
Dataset Quality
- Use at least 10 clips whose start→end motion is representative of the transitions you want learned.
- Clips should contain clear, coherent motion between their first and last frames.
- Keyframes (first/last frames) should be clean and sharp, since they anchor generation.
Caption Best Practices
- Describe the motion/transition plainly, optionally with a trigger phrase.
- Keep captions consistent across clips.
Good caption:
Weak caption:
Trigger Phrases
- A distinctive trigger phrase helps invoke a particular interpolation style; include it in every caption and at inference.
At inference, supply start and end images (and a middle image if you trained with ), and use the same caption style and trigger phrase.
Recommended Starting Configuration
Diagnosing Issues
- Transitions are jumpy or implausible: add more clips with smooth motion; ensure keyframes are clean.
- Middle keyframe missing error: when is true, every validation sample needs .
- Overfitting: fewer steps, lower , more clips.
Validation Prompt Tips
- Use fresh keyframe images to gauge generalization.
- Keep the prompt aligned with the transition you expect between the keyframes.
Common Pitfalls
- Uploading still images instead of clips (rejected).
- Enabling but omitting in validation.
- Keyframes that are blurry or unrepresentative.
fal-ai/ltx23-trainer-v2/interpolate