How to Lip Sync AI Videos with Audio Using an API In 2026

One fal API key covers every model in this guide, with H3 Max Lip Sync animating a single portrait to any read of 5 seconds or more. Output costs $0.16 per second at 1080p and $0.08 at 768p, and each video ends when its audio does. I reuse one presenter portrait for the English read and both dubs, and give a queued batch of customer greetings a portrait of its own.

I'll show you how to lip-sync AI videos with audio using an API, and the request itself only needs an image URL and an audio URL, so I spend most of my time getting those two inputs right.

I build the chain on fal, with H3 Max rendering the portrait and ElevenLabs Eleven v3 the voice before H3 Max Lip Sync animates one to the other.

TL;DR

One fal API key reaches all the models in this guide, from ElevenLabs Eleven v3 and H3 Max to H3 Max Lip Sync.
At $0.16 per second of 1080p output, a 10-second presenter sync on H3 Max Lip Sync costs about $1.58.
There's no duration setting to manage, since each video ends with its read, from a 5-second greeting to an 11-second German dub.
Reads must be at least 5 seconds long, and I pad shorter ones with silence using ffmpeg.
I'd recommend checking that the portrait's eyes are open before syncing, because closed eyes in the still stay closed for the entire clip.
For personalized greetings, I usually queue the jobs with a webhook and pay $0.40 per 5-second clip at 768p.

Where can you lip sync AI videos through an API?

fal is the best place to lip sync AI videos through an API, as the models that make the inputs run behind the same key as the lip sync endpoint that consumes them.

From the more than 1,000 hosted models in our catalog, I use H3 Max Lip Sync for every format in this guide, from a single presenter to queued batches.

H3 Max Lip Sync takes an image_url and an audio_url and hands back the synced video under video.url, along with its duration in seconds, once the job finishes.

H3 Max and Eleven v3 also return fal CDN URLs, and I pass those directly into the next request without downloading anything.

When does a portrait make a better lip sync source than footage?

A portrait makes a good lip sync source whenever the words change more often than the picture, as with translated reads or a greeting for every new customer.

H3 Max Lip Sync generates the head movement and blinking from that one still, and with no source footage involved, nothing has to line up with the new read.

Every read synced to a portrait inherits its framing and lighting.

In my experience the bigger win is timing, because a long translation and a short greeting both come back at their own length with no trimming on my side.

How does an API lip sync pipeline fit together?

An API lip sync pipeline runs in a fixed order, with the voice track and the portrait made first and both URLs sent to H3 Max Lip Sync last.

Eleven v3 voices the script first, and I measure the MP3 with ffprobe before anything else happens.

For the portrait, H3 Max text-to-video renders a short clip of the speaker and I save one frame of it with ffmpeg.

Both URLs then go to H3 Max Lip Sync in a single request that returns the finished MP4.

Audio shorter than 5 seconds gets rejected, and that minimum is the one length check I run on every read.

It's worth keeping the portrait URL around, because every later read on the same face skips the H3 Max render.

Setting up the clients and FFmpeg

To run the scripts in this guide, you'll want a fal API key exported as FAL_KEY and FFmpeg installed on the same machine.

Keys are created in the fal dashboard, though requests only run once the account holds credits, starting from a $1 top-up.

bash

export FAL_KEY="your-api-key-here"

pip install fal-client
npm install --save @fal-ai/client

ffprobe can measure a remote MP3 without downloading it, and the ffmpeg binary handles the two small edits in this guide: grabbing a portrait frame and padding short audio.

For apps that call fal from a browser, our proxy setup guide covers keeping FAL_KEY on the server.

What makes a portrait easy to lip sync?

It's ideal to start from a front-facing, evenly lit portrait with the whole mouth in view.

Check the eyes before you sync, since H3 Max Lip Sync keeps them in whatever state the still shows.

My first presenter portrait caught a blink, and every frame of the 10-second sync came back with her eyes closed.

I'd normally write hands and props below the chin in the prompt to keep the mouth clear.

Any aspect ratio from 0.4 to 2.5 is accepted, and the output uses the supported ratio closest to your image.

I match the portrait's resolution to the sync resolution, with 1080p for the presenter and dubs and 768p for the greetings.

Based on my experience, it's cleaner to set prompt_expansion_mode to disabled on H3 Max text-to-video and have the model work from my own wording.

How do you lip sync an AI video in Python?

In Python, you lip sync an AI video with three fal_client.subscribe() calls in a row.

The example is a 16:9 presenter announcing a feature in a budgeting app.

A talking-head explainer is a good first build, because a steady, front-facing shot makes sync errors easy to spot.

Step 1: Voice the script with Eleven v3

The first call sends the script to fal-ai/elevenlabs/tts/eleven-v3 and times the MP3 it returns with ffprobe.

Eleven v3 runs at $0.10 per 1,000 characters on fal, putting this 149-character script at about $0.015.

I went with the Charlotte voice from the schema's example list and left stability at its 0.5 default.

This read came back at 9.87 seconds, well clear of the 5-second floor.

python

import subprocess
import urllib.request

import fal_client

SCRIPT = (
    "Our new budgeting dashboard pulls your cards and accounts into one weekly view. "
    "Open the app this morning and tap Insights to see your first summary."
)


def seconds(url):
    probe = subprocess.run(
        ["ffprobe", "-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", url],
        capture_output=True, text=True, check=True,
    )
    return float(probe.stdout.strip())


voice = fal_client.subscribe(
    "fal-ai/elevenlabs/tts/eleven-v3",
    arguments={"text": SCRIPT, "voice": "Charlotte", "stability": 0.5},
)
audio_url = voice["audio"]["url"]
read_length = seconds(audio_url)
print(audio_url, read_length)

Generated using ElevenLabs Eleven v3 on fal, an AI model from ElevenLabs.

Step 2: Render the presenter portrait with H3 Max

Next, H3 Max text-to-video renders the presenter at 1080p in 16:9, and the script saves the first frame as the portrait.

Although the model accepts clips as short as 0.92 seconds, I ask for 5 to have spare frames on hand if the first one catches a blink.

To take a frame one second in, add "-ss", "1" ahead of "-i" in the ffmpeg call.

This portrait cost $0.52 at the promotional 1080p rate of $0.104 per second, and from November 1 the rate becomes $0.16, or $0.80 per portrait.

The presenter prompt below is the exact text that went into this request:

Prompt: A woman in her early thirties with shoulder-length dark hair and a rust-colored knit sweater talks directly to the camera in a bright home office, with relaxed head movement and an occasional open-hand gesture below her chin. A large window to her left lights her face evenly, and her mouth stays clear of her hands for the whole shot. Static medium close-up at eye level on a 50mm lens, with a softly blurred bookshelf behind her.

python

PRESENTER = (
    "A woman in her early thirties with shoulder-length dark hair and a rust-colored knit sweater "
    "talks directly to the camera in a bright home office, with relaxed head movement and an "
    "occasional open-hand gesture below her chin. A large window to her left lights her face evenly, "
    "and her mouth stays clear of her hands for the whole shot. Static medium close-up at eye level "
    "on a 50mm lens, with a softly blurred bookshelf behind her."
)

clip = fal_client.subscribe(
    "minimax/h3-max/text-to-video",
    arguments={
        "prompt": PRESENTER,
        "prompt_expansion_mode": "disabled",
        "aspect_ratio": "16:9",
        "resolution": "1080P",
        "duration": 5,
    },
)
subprocess.run(
    ["ffmpeg", "-v", "error", "-y", "-i", clip["video"]["url"], "-frames:v", "1", "presenter.png"],
    check=True,
)
portrait_url = fal_client.upload_file("presenter.png")
print(clip["video"]["url"], portrait_url)

Generated using MiniMax H3 Max on fal, an AI model from MiniMax.

The clip above is my second render, after the first one opened on the blink I mentioned earlier.

Step 3: Animate the portrait with H3 Max Lip Sync

The last call passes both URLs to minimax/h3-max/lip-sync/image-to-video at 1080p, then saves the synced MP4 to disk.

With enable_transcription on, the model transcribes the read first and uses the text to shape the mouth.

The 9.87-second result adds $1.58 to the run at $0.16 per second of 1080p output.

python

synced = fal_client.subscribe(
    "minimax/h3-max/lip-sync/image-to-video",
    arguments={
        "image_url": portrait_url,
        "audio_url": audio_url,
        "resolution": "1080P",
        "enable_transcription": True,
    },
)
urllib.request.urlretrieve(synced["video"]["url"], "presenter-en.mp4")
print(synced["video"]["url"], synced["duration"])

Generated using H3 Max Lip Sync on fal, an AI model from MiniMax.

When a single sync looks off, I paste the same two URLs into the H3 Max Lip Sync playground and compare runs there.

fal^{MODEL APIs}

The fastest, cheapest and most reliable way to run genAI models. 1 API, 100s of models

Build

fal^SERVERLESS

Scale custom models and apps to thousands of GPUs instantly

Deploy

fal^COMPUTE

A fully controlled GPU cloud for enterprise AI training + research

Train

What changes when you dub a clip into another language?

When you dub with H3 Max Lip Sync, the translated read can be any length, since each video is generated to fit its own audio.

Translations often run longer than the source line, and with a portrait as the source that overrun needs no handling.

For this format I reuse the presenter portrait from Step 2 and voice Spanish and German reads with the same Charlotte voice, setting language_code on each Eleven v3 request.

Against the 9.87-second English read, the Spanish version ran 10.45 seconds and the German 11.31, and both videos matched those lengths.

python

import fal_client

PORTRAIT_URL = "https://v3b.fal.media/files/<path>.png"  # the presenter portrait from Step 2

DUBS = {
    "es": (
        "Nuestro nuevo panel de presupuesto reúne tus tarjetas y cuentas en una sola vista semanal. "
        "Abre la app esta mañana y toca Insights para ver tu primer resumen."
    ),
    "de": (
        "Unser neues Budget-Dashboard bündelt deine Karten und Konten in einer Wochenansicht. "
        "Öffne heute Morgen die App und tippe auf Insights, um deine erste Zusammenfassung zu sehen."
    ),
}

for lang, text in DUBS.items():
    voice = fal_client.subscribe(
        "fal-ai/elevenlabs/tts/eleven-v3",
        arguments={"text": text, "voice": "Charlotte", "language_code": lang},
    )
    dubbed = fal_client.subscribe(
        "minimax/h3-max/lip-sync/image-to-video",
        arguments={
            "image_url": PORTRAIT_URL,
            "audio_url": voice["audio"]["url"],
            "resolution": "1080P",
            "enable_transcription": True,
        },
    )
    print(lang, dubbed["duration"], dubbed["video"]["url"])

Generated using H3 Max Lip Sync on fal, an AI model from MiniMax.

The pair of dubs costs about $3.48 on H3 Max Lip Sync at 1080p, or half that at 768p.

For translations you don't want to write yourself, ElevenLabs Dubbing on fal translates an existing audio track for $0.60 per minute, and its output works as the audio_url here.

Batching personalized greetings with H3 Max Lip Sync and webhooks

For batches of short personalized clips, I tend to queue the lip sync jobs with a webhook URL and let a small Express handler record each video when it lands.

The format here is a 5-second square welcome clip for a coffee subscription, in which a roaster greets new members by first name.

One portrait of the roaster covers every member in the batch.

Output goes from 480p up to 2K, and for greetings I drop to 768p, priced at $0.08 per second on our H3 Max Lip Sync page, or $0.40 for a 5-second clip.

A dozen-word greeting usually reads in four to five seconds, short of the minimum, and the submit script pads it with trailing silence.

A longer read passes through unchanged and is billed for its extra seconds.

I render the roaster once with H3 Max text-to-video and keep the first frame, as I did for the presenter.

I sent the prompt below as a 5-second, 768p square clip with prompt expansion off:

Prompt: Square video of a man in his forties with a trimmed grey beard and a navy canvas apron, standing behind the counter of a small coffee roastery and talking warmly to the camera. He keeps facing the lens while he speaks, with one small nod partway through and shelves of coffee bags softly blurred behind him. Static camera at chest height under warm light from pendant lamps.

At 768p the clip costs about $0.26 before October 31 and $0.40 after it.

This script saves the frame and uploads it to fal storage, giving the whole batch one portrait URL:

javascript

// make-roaster.mjs
import { execFileSync } from "node:child_process";
import { readFile } from "node:fs/promises";
import { fal } from "@fal-ai/client";

const ROASTER = [
  "Square video of a man in his forties with a trimmed grey beard and a navy canvas apron,",
  "standing behind the counter of a small coffee roastery and talking warmly to the camera.",
  "He keeps facing the lens while he speaks, with one small nod partway through",
  "and shelves of coffee bags softly blurred behind him.",
  "Static camera at chest height under warm light from pendant lamps.",
].join(" ");

const { data: clip } = await fal.subscribe("minimax/h3-max/text-to-video", {
  input: {
    prompt: ROASTER,
    prompt_expansion_mode: "disabled",
    aspect_ratio: "1:1",
    resolution: "768P",
    duration: 5,
  },
});

// H3 Max Lip Sync animates a still image, so the clip's first frame becomes the portrait.
execFileSync("ffmpeg", ["-v", "error", "-y", "-i", clip.video.url, "-frames:v", "1", "roaster.png"]);
const portraitUrl = await fal.storage.upload(new Blob([await readFile("roaster.png")], { type: "image/png" }));

console.log(clip.video.url);
console.log(portraitUrl);

Generated using MiniMax H3 Max on fal, an AI model from MiniMax.

For each member, the submit script voices and pads a greeting before queuing its sync, then stores the returned request_id on that member's record:

javascript

// submit-greetings.mjs
import { execFileSync } from "node:child_process";
import { readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fal } from "@fal-ai/client";
import { db } from "./db.js"; // your own data layer

const PORTRAIT_URL = "https://v3b.fal.media/files/<path>.png"; // the roaster frame from make-roaster.mjs
const WEBHOOK_URL = "https://roastery.example.com/api/fal/webhook";

const members = [
  { id: "m_1042", firstName: "Priya" },
  { id: "m_1043", firstName: "Mateo" },
  { id: "m_1044", firstName: "Hana" },
  { id: "m_1045", firstName: "Jonas" },
];

// H3 Max Lip Sync needs at least 5 seconds of audio, so shorter reads get trailing silence.
async function padToFiveSeconds(audioUrl, name) {
  const file = join(tmpdir(), `${name}-5s.mp3`);
  execFileSync("ffmpeg", ["-v", "error", "-y", "-i", audioUrl, "-af", "apad=whole_dur=5", file]);
  const url = await fal.storage.upload(new Blob([await readFile(file)], { type: "audio/mpeg" }));
  await rm(file);
  return url;
}

for (const member of members) {
  const { data: voice } = await fal.subscribe("fal-ai/elevenlabs/tts/eleven-v3", {
    input: {
      text: `Hi ${member.firstName}, welcome to the coffee club. Your first bag ships tomorrow.`,
      voice: "Daniel",
    },
  });
  const audioUrl = await padToFiveSeconds(voice.audio.url, member.id);

  const { request_id } = await fal.queue.submit("minimax/h3-max/lip-sync/image-to-video", {
    input: { image_url: PORTRAIT_URL, audio_url: audioUrl, resolution: "768P", enable_transcription: true },
    webhookUrl: WEBHOOK_URL,
  });

  await db.greetings.insert({ requestId: request_id, memberId: member.id, status: "queued" });
}

Transcription guidance stays on for the greetings as well.

The handler below checks the signature on each delivery before writing the result onto the matching row:

javascript

// server.mjs
import express from "express";
import { db } from "./db.js";
import { verifyWebhookSignature } from "./verify-webhook.js";

const app = express();

app.post("/api/fal/webhook", express.raw({ type: "application/json" }), async (req, res) => {
  const valid = await verifyWebhookSignature(
    req.get("x-fal-webhook-request-id"),
    req.get("x-fal-webhook-user-id"),
    req.get("x-fal-webhook-timestamp"),
    req.get("x-fal-webhook-signature"),
    req.body,
  );
  if (!valid) return res.sendStatus(401);

  const event = JSON.parse(req.body.toString("utf8"));
  const update =
    event.status === "OK" && event.payload
      ? { status: "ready", videoUrl: event.payload.video.url }
      : { status: "failed", error: event.error ?? event.payload_error };

  await db.greetings.update({ requestId: event.request_id }, update);
  res.sendStatus(200);
});

app.listen(3000);

Generated using H3 Max Lip Sync on fal, an AI model from MiniMax.

verifyWebhookSignature implements the steps from our webhooks documentation with Node's built-in Ed25519 support, and the handler needs no extra packages:

javascript

// verify-webhook.js
import { createHash, createPublicKey, verify } from "node:crypto";

// Implements the verification steps in fal's webhooks documentation with Node's built-in Ed25519 support.
const JWKS_URL = "https://rest.fal.ai/.well-known/jwks.json";
const JWKS_TTL_MS = 24 * 60 * 60 * 1000; // fal's keys can change, so never cache them longer than 24 hours

let cachedKeys = null;
let cachedAt = 0;

async function falPublicKeys() {
  if (!cachedKeys || Date.now() - cachedAt > JWKS_TTL_MS) {
    const res = await fetch(JWKS_URL, { signal: AbortSignal.timeout(10_000) });
    if (!res.ok) throw new Error(`JWKS fetch failed: ${res.status}`);
    const { keys = [] } = await res.json();
    cachedKeys = keys
      .filter((key) => typeof key.x === "string")
      // fal pads x; Node's JWK import expects unpadded base64url, so re-encode it
      .map((key) => Buffer.from(key.x, "base64url").toString("base64url"))
      .map((x) => createPublicKey({ key: { kty: "OKP", crv: "Ed25519", x }, format: "jwk" }));
    cachedAt = Date.now();
  }
  return cachedKeys;
}

export async function verifyWebhookSignature(requestId, userId, timestamp, signatureHex, body) {
  if (!requestId || !userId || !timestamp || !signatureHex) return false;

  // Reject deliveries more than 5 minutes from now to block replays.
  const sentAt = Number(timestamp);
  if (!Number.isFinite(sentAt) || Math.abs(Date.now() / 1000 - sentAt) > 300) return false;

  const bodyHash = createHash("sha256").update(body).digest("hex");
  const message = Buffer.from([requestId, userId, timestamp, bodyHash].join("\n"), "utf8");
  const signature = Buffer.from(signatureHex, "hex");

  const keys = await falPublicKeys();
  return keys.some((key) => {
    try {
      return verify(null, message, key, signature);
    } catch {
      return false;
    }
  });
}

Because fal signs the raw request body, the route reads it with express.raw and parses the JSON only after the signature checks out.

fal retries a delivery that fails or returns an error with growing backoff, for up to about an hour after the job completes, or about six minutes for results of 10 KB or more.

With the update keyed on request_id, a repeat delivery writes the same values to the same row.

Point the webhook at the final public HTTPS address, since fal doesn't follow redirects.

When a delivery never arrives, the result stays readable from the queue for a while, and this fallback collects it for any row still marked queued:

javascript

// recover-greetings.mjs
import { readFile } from "node:fs/promises";
import { fal } from "@fal-ai/client";
import { db } from "./db.js";

// Fallback for deliveries that never arrived: read finished jobs straight from the queue.
const rows = JSON.parse(await readFile(new URL("./greetings.json", import.meta.url), "utf8"));

for (const row of rows.filter((r) => r.status === "queued")) {
  const endpoint = "minimax/h3-max/lip-sync/image-to-video";
  const { status } = await fal.queue.status(endpoint, { requestId: row.requestId });
  if (status !== "COMPLETED") continue;

  const { data } = await fal.queue.result(endpoint, { requestId: row.requestId });
  await db.greetings.update({ requestId: row.requestId }, { status: "ready", videoUrl: data.video.url });
}

All four greetings cost $1.60 to sync on H3 Max Lip Sync at 5 seconds apiece, and the four Eleven v3 reads add about $0.03.

Adding the $0.26 portrait clip brings the batch to about $1.89.

How many of those jobs run at once depends on the concurrency limits on your account, and the rest wait in the queue.

How much does an API lip sync pipeline cost on fal?

The 10-second presenter build costs about $2.12 on fal as of October 21, 2026, at the rates on our model pages and pricing page.

Most of it is the $1.58 sync, with the $0.52 portrait making up nearly all of the rest.

H3 Max Lip Sync bills per second of output at $0.05 for 480p, $0.08 for 768p, $0.16 for 1080p and $0.32 for 2K, with a 1.2× multiplier on videos over 15 seconds.

H3 Max text-to-video bills $0.052 per second at 768p and $0.104 at 1080p until October 31, then $0.08 and $0.16.

A portrait is a one-time cost, and the dub row in the table below lists only reads and syncs for that reason.

My total for the three formats in this guide is about $7.53 at the measured lengths, or about $7.95 once the H3 Max promotion ends:

Format	Endpoints	Settings	Cost (to Oct 31)	Cost (after Oct 31)
Presenter clip (Python)	Eleven v3, H3 Max, H3 Max Lip Sync	16:9, 1080p, 9.87 s	$2.12	$2.40
Spanish and German dubs (Python)	Eleven v3 ×2, H3 Max Lip Sync ×2	Reuses the presenter portrait, 1080p	$3.52	$3.52
Four member greetings (JavaScript)	H3 Max, Eleven v3 ×4, H3 Max Lip Sync ×4	1:1, 768p, 5 s each	$1.89	$2.03

What breaks a lip sync pipeline on fal?

Most of what breaks a lip sync pipeline on fal is an input outside a limit on the model page, along with a few portrait choices that weaken the sync without raising an error.

I've grouped those checks below by the pipeline stage where each input is made.

Stage	What you see	Likely cause	Fix
Voice track	H3 Max Lip Sync rejects the audio	The read runs under 5 seconds	Trailing silence up to 5 seconds, as the submit script adds
Portrait	The eyes stay closed for the whole video	The kept frame caught a blink	A later frame with `-ss`, or a new render, checked before the sync call
Portrait	Mouth movement looks off in part of the video	A hand or prop covers the mouth in the portrait	A portrait with hands and props below the chin
Portrait	H3 Max Lip Sync rejects the image	The image's aspect ratio falls outside 0.4 to 2.5	A portrait cropped to a ratio inside that range
Lip sync	A long read costs more per second than expected	Output over 15 seconds carries a 1.2× multiplier	Budget for it, or split the script at a sentence break
Lip sync	A model can't fetch an input file	The host blocks automated requests or the link needs a login	A fal CDN URL for the same file
Delivery	Webhook deliveries never arrive	The `webhookUrl` redirects or resolves to a private IP	The final public HTTPS address, with the recover script as a fallback

Validation and content errors that models return are documented in our model errors reference.

How to Lip Sync AI Videos with Audio Using an API In 2026

TL;DR

Where can you lip sync AI videos through an API?

When does a portrait make a better lip sync source than footage?

How does an API lip sync pipeline fit together?

Setting up the clients and FFmpeg

What makes a portrait easy to lip sync?

How do you lip sync an AI video in Python?

Step 1: Voice the script with Eleven v3

Step 2: Render the presenter portrait with H3 Max

Step 3: Animate the portrait with H3 Max Lip Sync

falMODEL APIs

falSERVERLESS

falCOMPUTE

What changes when you dub a clip into another language?

Batching personalized greetings with H3 Max Lip Sync and webhooks

How much does an API lip sync pipeline cost on fal?

What breaks a lip sync pipeline on fal?

Recently Added

Build your lip sync pipeline on fal

Lip sync API FAQ

Can H3 Max Lip Sync use a recorded voiceover?

How is lip syncing a portrait different from lip syncing existing footage?

Does lip sync work on animated or illustrated characters?

Can you chain a lip sync pipeline into one fal endpoint?

How long can a lip-synced clip be?

Related articles

How To Make a Character Replacement Video [2026]

How To Use Grok Imagine Image 2.0: Prompts & Workflows [2026]

Kling 3.0 Pro vs. Sora 2 Pro: What's The Difference?

fal^{MODEL APIs}

fal^SERVERLESS

fal^COMPUTE