LTX 2.3 Trainer — Image-to-Video (/i2v)
Overview
The /i2v endpoint trains a LoRA adapter for the LTX 2.3 video model that learns to animate a starting image. The first frame is held as a condition during training, so at inference you supply a still image plus a prompt and the model generates a video that begins from that image.
Key features:
- Learns to bring a subject or style to life starting from a first-frame image.
- Adjustable first-frame conditioning: at the default
0.5, the LoRA is versatile and works for both image-to-video and plain text-to-video inference.
- Optional joint audio training (auto-detected — audio is used only if every clip has a track).
- Trains on videos or images, with an optional trigger phrase.
- Validation previews (each with a first-frame image) generated during training.
Provide a single .zip archive (linked via training_data_url) containing your clips and optional captions:
- Videos:
.mp4, .mov, .avi, .mkv
- Images:
.png, .jpg, .jpeg
- Captions: a
.txt file with the same base name as each media file (e.g. clip01.mp4 + clip01.txt). Optional but recommended.
The archive must contain only videos OR only images. Aim for at least 10 files. Files in subfolders are fine — clips with the same name in different subfolders are kept distinct automatically. The first frame of each clip is what the model learns to condition on, so make sure clips open on a representative, clean frame.
Minimum clip length: with auto_scale_input off (the default), each video must already have at least number_of_frames frames (default 89, ≈ 3.7 s at 24 fps). Shorter clips are silently skipped, and if every clip is too short the request fails (422, "All training videos are too short to be trainable"). Turn on auto_scale_input to resample shorter clips to the target frame count instead. This applies to video datasets only; images are single-frame and always usable.
Dataset
training_data_url (required)
Type: string
URL to the .zip archive of training clips (and optional captions).
trigger_phrase
Type: string
Default: ""
A phrase prepended to every caption during training and used at inference to activate the learned concept.
Training Parameters
rank
Type: integer (one of 8, 16, 32, 64, 128)
Default: 32
LoRA capacity. Higher values capture more detail but use more memory and overfit more easily.
| Value | Use Case |
|---|
| 8–16 | Small datasets, subtle changes |
| 32 | Balanced default |
| 64–128 | Larger datasets or complex subjects |
number_of_steps
Type: integer
Default: 2000 (range 100–20000)
Number of optimization steps.
learning_rate
Type: number
Default: 0.0002
Optimization step size. Start with the default; lower it if results are unstable.
first_frame_conditioning_p
Type: number
Default: 0.5 (range 0.0–1.0)
Probability that the first frame is held as a condition during training.
| Value | Behavior |
|---|
| 0.0 | Never conditions on the first frame (behaves like text-to-video) |
| 0.5 | Versatile — the resulting LoRA works for both image-to-video and text-to-video inference |
| 1.0 | Always conditions on the first frame (most specialized to image-to-video) |
Note: for single-image datasets the first frame is the entire sample, so first-frame conditioning is skipped automatically and training behaves like text-to-video on stills.
Video Configuration
number_of_frames
Type: integer
Default: 89 (range 9–121)
Frames per training clip. Must satisfy frames % 8 == 1; other values snap down to the nearest valid count.
frame_rate
Type: integer
Default: 24 (range 8–60)
Target frames per second. LTX 2.3's native rate is 24.
resolution
Type: string (low, medium, high)
Default: medium
| Resolution | 16:9 | 1:1 | 9:16 |
|---|
| low | 512×288 | 512×512 | 288×512 |
| medium | 768×448 | 768×768 | 448×768 |
| high | 960×544 | 960×960 | 544×960 |
aspect_ratio
Type: string (16:9, 1:1, 9:16)
Default: 1:1
Type: boolean
Default: false
Fit videos to the target frame count and frame rate. No effect on images.
Type: boolean
Default: true
Split long videos into scenes before training.
Type: number
Default: 30.0 (range 1.0–60.0)
Duration above which a video is eligible for scene splitting.
Audio Configuration
with_audio
Type: boolean or null
Default: null (auto-detect)
Controls joint audio-video training, all-or-nothing across the dataset. With null (default, auto-detect), audio is enabled only when every clip has an audio track — if even one clip is silent, the whole run trains video-only. Set true to force audio training (the request fails with a 422 if any clip lacks an audio track), or false to ignore audio even when present.
audio_normalize
Type: boolean
Default: true
Peak-normalize audio for consistent loudness.
audio_preserve_pitch
Type: boolean
Default: true
Preserve pitch when fitting audio to video duration.
Validation
validation
Type: array
Default: [] (max 2 entries)
Validation samples, each an object with:
prompt (string) — the text prompt.image_url (string, required) — the first-frame image to animate.
validation_negative_prompt
Type: string
Default: a built-in quality negative prompt.
validation_number_of_frames
Type: integer
Default: 89 (range 9–121)
validation_frame_rate
Type: integer
Default: 24 (range 8–60)
validation_resolution
Type: string
Default: high
validation_aspect_ratio
Type: string
Default: 1:1
stg_scale
Type: number
Default: 1.0 (range 0.0–3.0)
Spatio-Temporal Guidance scale for previews; 1.0 recommended, 0.0 disables.
debug_dataset
Type: boolean
Default: false
Return an archive of the preprocessed data for inspection.
Outputs
lora_file — the trained LoRA weights (.safetensors).config_file — JSON describing the trigger phrase and training type.video — a combined validation reel (when validation samples were provided).debug_dataset — preprocessed-data archive, only when debug_dataset is enabled.
Billing
A successful run is billed max(100, number_of_steps) billable units. Requests that fail before training completes (input-validation errors / HTTP 422, or dataset-download failures) are billed 0 units.
How the Training Works
Pipeline Overview
- Preprocessing — the archive is extracted, media matched to captions, clips resized/cropped to the resolution bucket, optionally scene-split, and audio prepared.
- Training — the LoRA trains for
number_of_steps, conditioning on the first frame with probability first_frame_conditioning_p, running validation previews at intervals.
- Output — the LoRA, config, and validation reel are uploaded.
What Happens to Your Data
- Archive extraction: the
.zip is unpacked; macOS __MACOSX metadata folders are ignored.
- File matching: each media file is paired with the
.txt of the same base name.
- Video fitting: clips are resized to fill the target resolution (keeping aspect ratio), then center-cropped to the exact bucket size. With
auto_scale_input, clips are resampled to the target frame rate and frame count.
- First frame: during training the first frame of each clip is sometimes held as a clean condition (per
first_frame_conditioning_p) and the model learns to generate the rest.
- Scene splitting: long clips are cut into shots when enabled.
- Captions: the trigger phrase (if set) is prepended.
- Audio: when enabled, each clip's soundtrack is prepared and learned jointly.
LoRA Training
A LoRA is a compact set of adapter weights trained on top of the frozen base model. You load it alongside the base LTX 2.3 model at inference. Here it specializes the model at animating a supplied first frame.
Audio Support
If clips have sound and audio is enabled, the LoRA can produce a matching soundtrack. Audio is all-or-nothing across the dataset: if any clip is silent, the whole run trains video-only.
Tips for Getting Good Results
Dataset Quality
- At least 10 clips; more variety helps.
- Clips should open on a clean, representative first frame — that frame drives generation.
- Show the subject from multiple angles and contexts for a robust concept.
Caption Best Practices
- Describe the content of each clip plainly and specifically.
- Subject/object: combine your trigger phrase with a description, e.g.
tronlog0 a glowing logo rotating slowly.
- Style: describe the content and let the look be learned.
Good caption: a golden retriever runs across a grassy field
Weak caption: dog
Trigger Phrases
- Use a rare, distinctive trigger token for a specific subject and include it in every caption.
- Skip the trigger phrase for an always-on style.
Scene Splitting and Captions
With scene splitting on, one long video produces several clips sharing one caption — which may not describe each split accurately. Pre-split and disable scene splitting for precise captions.
Prompt the LoRA the way you captioned it, including the trigger phrase. Supply a first-frame image consistent with your training footage. At first_frame_conditioning_p = 0.5 the LoRA also works without an image (text-to-video).
Recommended Starting Configuration
{
"training_data_url": "https://example.com/my_dataset.zip",
"trigger_phrase": "tronlog0",
"rank": 32,
"number_of_steps": 2000,
"learning_rate": 0.0002,
"first_frame_conditioning_p": 0.5,
"number_of_frames": 89,
"frame_rate": 24,
"resolution": "medium",
"aspect_ratio": "1:1",
"validation": [
{ "prompt": "tronlog0 rotating slowly", "image_url": "https://example.com/start.png" }
]
}
Diagnosing Issues
- Overfitting (previews copy training clips or ignore the prompt): fewer steps, lower
rank, more varied data.
- Underfitting (concept barely shows): more steps, higher
rank, better captions.
- Animation ignores the supplied image: raise
first_frame_conditioning_p toward 1.0.
- Training fails: verify the archive is all-video or all-image, and that captions match base names.
Validation Prompt Tips
- Pair each prompt with a representative first-frame image.
- Use prompts in the same style as your captions, including the trigger phrase.
Common Pitfalls
- Mixing images and videos in one archive.
- Clips that open on blurry or unrepresentative first frames.
- Forgetting the trigger phrase or a first-frame image at inference.
fal-ai/ltx23-trainer-v2/i2v