How to Deploy a Video Generation Model in Production

Open video models such as Wan are easy to run once on a GPU and hard to run as a product. Each request can take minutes and the weights run to tens of gigabytes. Each output is a file, not a JSON value, and traffic tends to arrive in bursts. A setup that works in a notebook breaks on a cold start, a timeout or a GPU that sits idle overnight.

fal Serverless takes care of the operations side. You write the model as a Python class, and fal builds the container and puts the app behind a queue. It adds GPU runners as requests arrive and scales back to zero between bursts, billed per second of runner time. This guide follows fal's Wan text-to-video example. It covers the choices that matter for video in particular: the GPU, where the weights live, how callers wait, and how to keep cold starts short.

Why video models need different settings

A few traits set video models apart from image or text models:

• Long requests. A clip can take far longer than an HTTP connection should stay open. Callers need a queue, not a blocking request.
• Large weights. Downloading and loading the model is often the slowest part of starting a runner.
• File outputs. The result is an MP4 that has to be encoded and hosted somewhere.
• Bursty traffic. Demand comes in waves, and an idle GPU is expensive.

Each section below deals with one of these.

Pick a GPU for video

Start from the smallest GPU your model fits on, then look at the video encoders. fal's machine types list the hardware NVENC and NVDEC engines for each GPU. These are dedicated units that encode and decode video without using the GPU's compute cores.

The docs recommend the RTX PRO 6000 or L40 for video generation. Both have hardware encoders that output frames without using GPU compute time, and GPUs without a hardware encoder have to encode on the CPU. The Wan example uses GPU-H100, which has more memory bandwidth (3.4 TB/s, against 1.8 TB/s on the RTX PRO 6000) but no hardware encoder.

Structure the app

A video app is a fal.App subclass. The class declares the hardware and the dependencies, setup() loads the model once per runner, and each endpoint handles one kind of request:

class Wan(fal.App):
    app_name = "wan"
    min_concurrency = 0
    max_concurrency = 1
    keep_alive = 300
    machine_type = "GPU-H100"
    requirements = ["torch==2.6.0", "diffusers==0.32.2", "transformers==4.49.0", ...]

    def setup(self):
        checkpoint_dir = Path(
            snapshot_download(
                "Wan-AI/Wan2.1-T2V-1.3B",
                local_dir=FAL_MODEL_WEIGHTS_DIR / "wan_t2v_1.3B_480p",
            )
        )
        ...

    @fal.endpoint("/v2.1/1.3b/text-to-video")
    def generate_image_to_video(self, request: WanT2VRequest) -> WanT2VResponse:
        ...
        return WanT2VResponse(video=File.from_path(save_path), seed=seed)

An excerpt from fal's Wan example. The full example also clones the Wan repo, expands prompts and runs safety checks.

A few details are worth copying:

• Pin your dependencies. The example pins exact versions of torch, diffusers and transformers. fal builds the container remotely from this list, so you don't need Docker installed locally.
• Download into FAL_MODEL_WEIGHTS_DIR. This directory is on the persistent /data volume, so the first runner downloads the weights and later runners reuse them. fal also points HF_HOME at /data, which means Hugging Face downloads persist without any extra setup.
• Return a File. File.from_path() uploads the rendered MP4 to fal's CDN and returns a URL. Callers get a link, not megabytes of video in the response body.
• Put the version in the path. /v2.1/1.3b/text-to-video names the model version and size, which leaves room to add other variants to the same app as new endpoints.

An endpoint can also call other models on fal through fal_client. The Wan example uses this to expand short prompts and to check inputs before rendering, so a whole pipeline sits behind one endpoint.

Let callers wait in the queue

Deployed apps sit behind fal's queue, the same queue the hosted Model APIs use. Callers submit a request, get a request ID straight away, and collect the result when the render finishes:

import fal_client

result = fal_client.subscribe(
    "your-username/wan/v2.1/1.3b/text-to-video",
    arguments={"prompt": "A cat walking through a futuristic city", "seed": 42},
)
print(result["video"]["url"])

subscribe() waits for the result. You can also submit and poll the status URL, or register a webhook. Over plain HTTP, submit to https://queue.fal.run/your-username/wan/.... fal's cold start example advises always calling video endpoints through the queue rather than raw synchronous HTTP, because renders run for tens of seconds to minutes.

Keep cold starts short

A runner that starts cold has three jobs before it can serve a request. It has to get the weights, load them onto the GPU and compile kernels for the shapes the model will run. For Wan 2.1, compiling kernels takes the longest, more than the download. fal's cold start example for Wan 2.1 applies one optimization to each job, and all three build on /data:

1. Pack the weights onto /data once. The first runner downloads the weights, converts them to the serving data types and writes a pack to /data. Later runners skip this step. Build large artifacts on local disk first, then copy them to /data in one go and rename them into place. Writing in small chunks straight to /data is slow, and the rename stops other runners from reading a half-written file.
2. Load with FlashPack. FlashPack streams the packed weights from disk to the GPU without a CPU deserialization step.
3. Share the compiled kernels. synchronized_inductor_cache from the fal toolkit compiles torch.compile kernels once per GPU type and syncs them to /data. Later runners on the same GPU type unpack the cached kernels instead of compiling again.

Kernel caches are per GPU type, so a cache built on an H100 does nothing for an L40. If you list several machine types, each type pays its own one-time compile.

These optimizations aren't always worth the effort. The example says to skip them when the model has little or no compile step, when the app rarely scales from zero, or when a cold start is already only a few seconds. In those cases a longer keep_alive or a small min_concurrency is simpler.

Large first boots can also run past the default setup time limit. Raise startup_timeout on the class to give setup() enough time on a runner's first boot.

Use more than one GPU

When a model doesn't fit on one GPU, or you want to split work across several, set num_gpus on the app:

class MyApp(fal.App):
    machine_type = "GPU-H100"
    num_gpus = 2

The fal.distributed module provides DistributedRunner and DistributedWorker to coordinate one worker per GPU. fal's Wan 2.2 multi-GPU example serves a model too big for one GPU across four H100s, splitting every denoising step across the workers. For running independent generations side by side, the SDXL data-parallel example shows the simpler pattern. A multi-GPU runner is billed as GPU count × duration, so a two-GPU runner alive for 60 seconds costs 120 GPU-seconds.

Scale between bursts

The Wan example scales to zero (min_concurrency = 0) and keeps an idle runner warm for five minutes (keep_alive = 300). Short gaps between requests therefore don't each cause a cold start. max_concurrency = 1 caps it at one runner. Raise that limit when you're ready to serve more than one render at a time. Each runner handles one request at a time unless you raise max_multiplexing. Only raise it when your handlers are async and the model has the GPU memory to render several clips at once.

You pay per second of runner lifetime at your machine type's rate, not per video. Runner time spent in setup(), idle within keep_alive, serving requests or shutting down is billed. Time waiting for hardware or pulling the image is not. For a closer look at these settings, see the serverless inference guide and what a serverless GPU cold start is.

FAQ

What is the best GPU for serving a video generation model?

The machine types guide recommends RTX PRO 6000 (96 GB, 4 NVENC and 4 NVDEC, AV1 encode) or L40 (48 GB, 3 NVENC and 3 NVDEC) for video generation. Their hardware encoders output video without using GPU compute time. H100, H200 and B200 have more memory bandwidth but no hardware encoder. Start from the smallest GPU your model fits on.

How do I keep cold starts short for a large video model?

Keep the weights on the persistent /data volume instead of in the image, cache compiled kernels, and hold warm capacity with keep_alive, min_concurrency or concurrency_buffer. fal's cold start example packs the Wan 2.1 weights onto /data, loads them with FlashPack and shares a torch.compile cache across runners.

Can a video model use more than one GPU?

Yes. Set num_gpus on the app, for example num_gpus = 2 with machine_type = "GPU-H100". Multi-GPU runners are billed as GPU count × duration. fal's Wan 2.2 example splits one model across four H100s with fal.distributed.

How do callers wait for a long video generation?

Deployed apps sit behind fal's queue. Callers submit to queue.fal.run/your-username/app-name, or use fal_client.subscribe(), which waits for the result. In the Wan example, the video comes back as a fal File with a CDN URL.

How much does it cost to serve a video model on fal?

You pay per second of runner lifetime at your machine type's rate, not per video. Setup, idle, running, draining and terminating time are billed. Time waiting for hardware or pulling the image is not. Current GPU rates are on fal.ai/pricing.

Do I need approval to deploy a video model on fal?

Yes. Deploying your own apps on fal Serverless needs access that the fal team approves for each account. Request it on the Serverless Get Started page. Creating a fal account is free, and calling hosted Model APIs needs no approval.