How to Deploy a LoRA as an API

A LoRA is a small set of weights that adapts a large base model to a style, a product or a person. Training one is the easy part. To turn it into something other people can call, you need a few more pieces: an endpoint, a place to keep the weights, a GPU that has the base model loaded, and a way to scale with demand.

On fal there are two ways to do this, and which one fits depends on how much of the stack you want to own. You can train with a hosted trainer and pass the weights to a hosted LoRA inference API, with no servers involved. Or you can deploy your own trainer and serving code on fal Serverless, which gives you a permanent endpoint for your model. This guide covers both, then the storage, routing and access details that matter once more than one person calls your model.

Two ways to serve a LoRA

Start with the hosted path if fal already hosts a trainer for your base model. Move to your own app when you need a base model fal doesn't host, custom pre- or post-processing, or one endpoint that serves many adapters.

Option 1: hosted trainer, hosted inference

FLUX LoRA fast training takes a ZIP of images and returns LoRA weights. It returns a weights file, not an endpoint. You then pass that file to a hosted LoRA inference API such as fal-ai/flux-lora, which takes a list of LoRAs on each request:

import fal_client

training = fal_client.subscribe(
    "fal-ai/flux-lora-fast-training",
    arguments={
        "images_data_url": "https://your-bucket/training-images.zip",
        "trigger_word": "your-style",
        "is_style": True,
    },
)
lora_url = training["diffusers_lora_file"]["url"]

image = fal_client.subscribe(
    "fal-ai/flux-lora",
    arguments={
        "prompt": "a lighthouse at dusk in your-style",
        "loras": [{"path": lora_url, "scale": 1.0}],
    },
)
print(image["images"][0]["url"])

Field names from the trainer and inference API schemas.

The trainer is set up for subjects (people, products) by default. is_style switches it to style training, which turns off auto-captioning and segmentation. Both models are hosted Model APIs, so this path needs a fal account, an API key and credits, but no Serverless approval. Keep lora_url, or copy the file somewhere you control, because every inference call references it.

Option 2: deploy your own trainer

When fal doesn't host a trainer for your base model, or you want to own the training loop, deploy a trainer as an app. fal's WAN LoRA training example trains LoRAs for Wan 2.1 T2V 1.3B, and deploys as a hosted API with fal deploy:

pip install fal
fal auth login
git clone https://github.com/fal-ai-community/fal-demos.git
cd fal-demos
pip install -e .
fal deploy fal_demos/video/wan_trainer.py::WanLoRATrainerDemo

The trainer downloads the WAN weights and a pinned training repo. It builds a dataset from your ZIP, trains with DeepSpeed and returns the adapter. Call it like any other fal endpoint:

import fal_client

result = fal_client.submit(
    "your-username/wan-lora-trainer-demo",
    arguments={
        "training_data_url": "https://your-bucket/train.zip",
        "rank": 16,
        "number_of_steps": 400,
        "learning_rate": 2e-4,
        "trigger_phrase": "your-style",
        "auto_scale_input": True,
        "video_clip_mode": "single_beginning",
    },
).get()

print(result["lora_file"]["url"])

The ZIP can contain .png, .jpg, .jpeg, .gif or .mp4 files, each with an optional caption file next to it (my_clip.txt beside my_clip.mp4). With auto_scale_input on, the trainer fits videos to 81 frames at 16 fps, which is what WAN trains on.

submit() returns a handle straight away, and .get() waits for the result. Training runs can be long, so submitting through the queue and collecting the result later fits better than holding a connection open.

Option 2, continued: serve the adapter from your own endpoint

The serving app follows the usual fal.App shape. setup() loads the base model once per runner, and the endpoint applies the adapter and generates. Where the adapter lives decides how fast a new runner can start serving.

Keep weights on /data

Every runner in your account mounts /data, a persistent, distributed volume. Files there survive runner restarts and new deployments. Reads go through a multi-layer cache, so a new runner can load a base model and its adapters in setup() without downloading them again. A few details from the persistent storage guide:

• fal sets HF_HOME to /data/.cache/huggingface, so base models downloaded with transformers, diffusers or huggingface_hub persist automatically.
• /data is shared, so write each file to a temporary path beside the destination, then os.rename it into place. Other runners then never read a half-written adapter.
• Upload adapters from outside an app through the Files page in the dashboard or the CLI:

fal files upload my-style.safetensors loras/my-style.safetensors

Uploads don't overwrite: uploading to a path that already exists returns an error, so remove the old file first with fal files rm. The CLI handles files of about 100 GB. Writing from inside a running app isn't subject to these upload limits.

One endpoint, many adapters

Many products train one LoRA per customer or per user. Deploying one app per adapter means thousands of separate fleets, most of them cold. Reloading the adapter on every request makes every request slow. fal's per-user LoRA example avoids both. It runs one warm fleet that keeps a bounded set of adapters on each GPU, and asks the router to prefer the runner that already has a user's adapter loaded:

• /data holds every adapter file, and a KV store record maps each user to their adapter. Any runner can look up any user.
• Each runner keeps a least-recently-used set of adapters loaded and reports them through provide_hints(). fal re-evaluates the hints on every response.
• Clients send the hint back. The example app returns each user's hint in an x-fal-runner-hint response header. Clients pass it back as the hint= option on that user's next request, and the router then prefers a runner that already has the adapter loaded.

Hints are best effort. The router can only steer a request to a preferred runner when that runner is free, and with a single runner every request lands on it anyway. See optimize routing behavior for how hints work. The companion train-serve-lora example covers the training half of the same loop. Nothing on /data is pruned automatically, so a production fleet should delete adapters for users who leave.

Choose hardware for training and serving

fal's machine types guide recommends:

• Training: GPU-A100, GPU-H100 or GPU-H200, depending on model size.
• Serving image generation: GPU-L40 or GPU-RTXPRO6000.

machine_type can be a list, such as ["GPU-L40", "GPU-H100"]. fal tries the types in order until it finds capacity, which is what the per-user LoRA example does.

Some training runs need the same hardware for hours or days. For these, fal Compute offers dedicated H100 SXM instances that you SSH into, billed per hour. Serverless suits API endpoints and on-demand jobs, including a trainer exposed as an API, where runners scale automatically and bill per second.

Decide who can call your model

fal deploy uses private auth by default: only you and your team can call the app, with an API key, and all compute is billed to you. Public lets anyone with the URL call it, still billed to you, which is useful for demos. Shared lets anyone call it with their own API key, so each caller pays for their own usage. This is how apps in fal's model gallery work. fal enables shared mode on request, and it is the first step toward publishing to the marketplace. Listing also needs billable units and a review by the fal team.

FAQ

How do I fine-tune a FLUX LoRA and call it as an API?

Use the FLUX LoRA fast training Model API (fal-ai/flux-lora-fast-training) to train on a ZIP of images, with is_style set for a style. It returns LoRA weights, not an endpoint. Pass diffusers_lora_file.url in loras[].path to a hosted LoRA inference API such as fal-ai/flux-lora, or deploy your own serving app. Both hosted models need an API key and credits, not Serverless access.

Can I train a Wan video LoRA on fal?

Yes. The WAN LoRA training example deploys a trainer for Wan 2.1 T2V 1.3B as an API. Its training_data_url points to a ZIP of .png, .jpg, .jpeg, .gif or .mp4 files with optional caption files. auto_scale_input fits videos to 81 frames at 16 fps, and the result includes the LoRA file URL.

Where do I keep LoRA weights so every runner can load them?

On /data, the persistent volume every runner mounts. Files there survive runner restarts and deployments. Reads are served through a multi-layer cache, so a new runner can load weights in setup() without downloading them again.

How do I serve many LoRAs from one endpoint?

Keep every adapter on /data, keep a small set loaded on each GPU, and use routing hints to send each user's repeat requests to a runner that already has their adapter loaded. fal's per-user LoRA example implements this with provide_hints(), a KV store registry and the hint= client option.

Should I train on Serverless or on fal Compute?

Serverless suits API endpoints and on-demand jobs, including a trainer exposed as an API: runners scale automatically and bill per second. fal Compute gives you dedicated H100 instances over SSH, billed per hour, for training runs and batch jobs that need the same hardware for hours or days.

Do I need approval to deploy my own fine-tuned model?

To deploy your own apps on fal Serverless, yes. The fal team approves Serverless access for each account. Request it on the Serverless Get Started page. Creating a fal account is free. The hosted trainers and LoRA inference APIs need no approval, only credits on your account.