Python-Native Serverless GPUs: Deploy Your Own Model Without a Dockerfile

Many teams who deploy their own models want to stay in Python. They do not want to write a Dockerfile, pick a CUDA base image or run a cluster. They want to write a class, say which GPU it needs, and get an API. That style is usually called Python-native serverless GPUs. This page explains how it works, what you give up and what you keep, and how fal Serverless implements it.

What "Python-native" means

On a Python-native platform, three things that are usually separate files become Python:

• The environment. Dependencies are a list in your code, not a Dockerfile.
• The hardware. The GPU type and count are attributes, not a node pool.
• The API. Decorated methods become HTTP routes, with schemas from type hints.

A command-line tool ships the code, builds the environment remotely and runs it on GPUs that scale with traffic. You never build or push an image yourself.

A model as one Python class

A complete fal app:¹

import fal
from pydantic import BaseModel

class Input(BaseModel):
    prompt: str

class Output(BaseModel):
    result: str

class MyModel(fal.App):
    machine_type = "GPU-H100"
    requirements = ["torch", "transformers"]

    def setup(self):
        from transformers import pipeline
        self.pipe = pipeline("text-generation", model="gpt2", device="cuda")

    @fal.endpoint("/")
    def generate(self, input: Input) -> Output:
        result = self.pipe(input.prompt, max_length=50)[0]["generated_text"]
        return Output(result=result)

• machine_type picks the GPU. num_gpus gives a runner more than one.²
• requirements lists pip packages. fal installs them at build time and caches the result.³
• setup() runs once per runner, so the model loads once and serves many requests.⁴
• @fal.endpoint turns a method into an HTTP route. The Pydantic models become the request and response schema.

For a single function with no state to keep between calls, @fal.function takes the same requirements and machine_type arguments as a decorator.⁵

How the concepts map

If you have used Python-native GPU tools before, most ideas carry over. fal's names for them:

One difference matters for callers. A fal app is called over HTTP through a queue, with the fal client SDKs or plain REST, not as a Python remote procedure call. Any language can call it.⁸

Dependencies without a Dockerfile

The managed runtime handles CUDA and PyTorch setup for you. A few patterns cover most needs:

• Pin versions for reproducible builds: "torch==2.4.0".
• Use a GPU wheel index by putting "--extra-index-url" and the index URL before the packages that need it.
• Install a prebuilt wheel with a direct URL, or name@url so other packages can depend on it.
• Ship local code with app_files (files and folders at the same relative paths) or local_python_modules (a simple module).⁹

You need a Dockerfile only for system packages, a set base image or non-Python tools. Then set image = ContainerImage.from_dockerfile_str(...) on the same class. Local code then goes in with COPY, because app_files and local_python_modules work only with the managed runtime.¹⁰

What runs on your machine and what runs on the GPU

When you run fal run or fal deploy on a file, fal pickles your class on your machine and sends it to the cloud. Top-level code in the file runs locally. The methods, including setup() and your endpoints, run on the runner.

That gives three rules:

• Import heavy libraries inside methods. An import torch inside setup() runs on the runner, where the package is installed.
• Create GPU tensors, models and connections in setup(). They do not pickle well.
• Read secrets at runtime with os.getenv(). A secret read at the top level would be pickled into the payload. Store it with fal secrets set instead.

Pick a GPU in one line

Pick the smallest GPU your model fits on. fal's published rates, billed per second:¹¹

"As low as" rates are quoted through fal's sales team. To fall back when a type is busy, give a list: machine_type = ["GPU-H100", "GPU-H200"]. fal tries them in order.

Scale to zero, and when not to

A new app scales to zero by default: min_concurrency is 0.¹² After a request, an idle runner stays up for keep_alive seconds (60 by default), then shuts down. With no runners alive, nothing is billed.¹³

Billing covers the time a runner is alive: setup, idle, running, draining and teardown. Waiting for a GPU and pulling the image are not billed. Requests that fail with an HTTP 5xx status are not charged.

The cost of scale to zero is a cold start on the next request. If users cannot wait, keep one runner warm with min_concurrency = 1. On an H100 at list price that is $4.50 an hour, all day. Raise max_concurrency (2 by default) before real traffic arrives. For the trade-offs, see What Is a Serverless GPU Cold Start?.

Deploy with one command

pip install fal
fal auth login
fal run app.py::MyModel      # temporary URL, runs setup() as production does
fal deploy app.py::MyModel   # permanent endpoint: your-username/my-model

fal run starts your app on a temporary runner and prints a URL and a Playground. Import errors and model-load failures show up here, not in production.¹⁴ fal deploy builds the image in the cloud, so Docker does not need to be installed on your machine. The app is private by default, so callers need an API key. Call it from Python with fal_client.subscribe("your-username/my-model", arguments={...}).

Serverless access is approved per account. Request it at fal.ai/dashboard/serverless-get-started. For a broader tour that also covers custom containers and existing HTTP servers, see How to Choose a Serverless GPU Platform.

FAQ

Do I need a Dockerfile to deploy a model on fal?

No. List pip packages in requirements and fal builds the environment for you, including CUDA setup. Bring a Dockerfile only when you need system packages or a set base image.

Do I need Docker installed locally?

No. fal deploy uploads your code and builds the container image remotely. You only need the fal CLI and your code.

Does a Python-native app scale to zero?

Yes, by default. min_concurrency defaults to 0. Idle runners shut down after keep_alive seconds, and an app with no runners has no compute charge.

How do I call the deployed app from another language?

Over HTTP. Every fal app gets a queue endpoint at https://queue.fal.run/your-username/app-name, and client SDKs exist for Python and JavaScript.