Dedicated GPUs vs Serverless Inference: How to Find Your Break-Even

When is a dedicated GPU cheaper than serverless?

Reserved GPUs and serverless GPUs do not compete. They serve different parts of the same traffic curve.

• A reserved GPU costs less when it is busy for most of the hours that you pay for.
• Per-second billing costs less when a GPU would otherwise sit idle.

Billed utilisation decides which option costs less. Billed utilisation is the share of time that your runners are alive, not the share of time that they are busy. A runner is one running copy of your app on one or more GPUs. At fal's published prices, the break-even is about 55% for an H100 and about 69% for a B200.

In practice, idle time moves the break-even line:

• On per-second billing, you pay for setup time and for the keep_alive window.
• On a reserved GPU, you pay for every quiet hour.

Our recommendation. GPU capacity is scarce across the market. Per-second billing is the right price for demand that you cannot predict. A reservation is the right price, and the right guarantee, for demand that you know. Production inference should use both. Reserve the floor: the number of GPUs that your traffic needs for most of the day. Use per-second capacity for the peaks.

What is billed GPU utilisation?

Billed utilisation is billed time, not busy time. fal bills a runner for all of the time that it is alive:

• setup (your model loading),
• idle time, including the keep_alive window,
• running requests,
• draining and teardown (the runner finishes its last requests and shuts down).

fal does not bill pending time or image pulls.

So billed time is always more than busy time. You can make this difference smaller. The sections below show how.

The break-even formula

Let s be the per-second rate, expressed per hour. Let d be the reserved rate per hour. Let u be the share of the month that your runners are alive.

• Per-second monthly cost = u × 720 h × s
• Reserved monthly cost = 720 h × d
• Break-even utilisation = d ÷ s

We use a 30-day month of 720 hours. Costs are per GPU.

The prices in this guide are fal's published prices on October 19, 2026:

For the H100: 2.49 ÷ 4.50 = 55.3%. For the B200: 5.49 ÷ 7.99 = 68.7%.

The reserved rate depends on the size and length of the commitment. Check fal.ai/pricing for current rates.

Break-even at other reserved rates

The break-even is the reserved rate as a share of the per-second rate. This is true for any provider and any negotiated rate:

A smaller discount moves the line up. A larger reservation can get a larger discount, and a larger discount moves the line down.

Monthly cost at each utilisation

H100 (80 GB)

Reserved at $2.49/h for 720 h = $1,792.80 per month, at any utilisation.

B200 (192 GB)

Reserved at $5.49/h for 720 h = $3,952.80 per month, at any utilisation.

The B200 line is higher because its reserved rate is a larger share of its per-second rate. The share is 69% for the B200 and 55% for the H100.

Idle time that you pay for: setup, keep_alive and min_concurrency

keep_alive

keep_alive is how long an idle runner stays up after its last request. The default for a fal.App is 60 seconds.

Take one H100 that serves a 10-second job every 5 minutes (300 seconds). The GPU is busy 3.3% of the time. Assume 60 seconds of setup. Setup can take from seconds to minutes. It depends on your model size and your setup() code.

A keep_alive that is shorter than your typical gap gives you the idle cost and the cold start together. In this example, the 60-second default costs $648 more per month than keep_alive 0. Every request still gets a cold start.

At 290 seconds or more, the runner never shuts down. You pay per-second rates for an always-on GPU. A reserved GPU at $2.49/h costs less.

The rules:

• If a gap is shorter than your setup time, it costs less to keep the runner warm than to restart it. Set keep_alive to cover the gap.
• If a gap is longer than your setup time, it costs less to restart the runner. Keep the runner warm through the gap only if you need low latency on the next request.
• A keep_alive that is shorter than your typical gap is the worst choice. Make it longer or make it shorter.

To make setup shorter:

• Use smaller images.
• Cache your model weights.
• Load the weights lazily where you can.

See What Is a Serverless GPU Cold Start?

min_concurrency

min_concurrency keeps a minimum number of runners alive when there is no traffic. fal bills these runners at the per-second rate. One H100 held warm all month costs $3,240.00. The same GPU at the lowest reserved rate costs $1,792.80.

A permanent warm floor on per-second billing is a reservation that you pay list price for.

Steady traffic and spiky traffic

Steady traffic keeps GPUs busy all day and night. Utilisation is high and predictable. Above the break-even line, a reserved GPU costs less.

Spiky traffic has peaks and long quiet periods. Launches, campaigns, batch drops and daytime-only use all have this pattern. A reserved fleet sized for the peak is idle most of the day. Per-second billing charges only for the peak.

Most real traffic is both: a steady base with bursts on top.

To see your traffic pattern, calculate one ratio:

Average GPUs ÷ peak-hour GPUs

This ratio is the utilisation of a reserved fleet sized for your peak hour. If the ratio is below the break-even line, a peak-sized fleet costs more than per-second billing.

How should I size reserved GPU capacity?

Size the reservation from the floor, not from the peak. Reserve the GPUs that your traffic needs for most of the day. Send the rest of the traffic to per-second capacity.

Example: one day of H100 traffic

Your traffic needs:

• 3 H100s for 8 hours of the night,
• 6 H100s for 12 hours of the day,
• 12 H100s for a 4-hour evening peak.

The average is 6 GPUs. The peak is 12.

The monthly cost at each reservation size, with the remainder billed per second at $4.50:

The lowest cost is at 6 reserved GPUs. This is the highest level that the traffic holds for more than 55% of hours. It is 28% less than all per-second capacity, and 35% less than a fleet sized for the peak.

The per-second figures assume that runners are alive only while you need them. Add your real setup time and keep_alive.

Check your latency and throughput requirements

Cost gives one answer. Your service requirements give a second answer. fal bills capacity above a reservation per second. That capacity is not reserved for you, and it can have cold starts.

If your product has a latency or throughput target, the cost-only answer can be too small. For example:

• a p95 latency target that a cold start would break,
• a throughput commitment to your own customers, such as requests per second or jobs per hour,
• a launch or campaign peak that must not wait in a queue.

Traffic inside a reservation has guaranteed capacity on warm runners. To guarantee a level of throughput, reserve enough GPUs to deliver it at your latency target. Do this even if some of those GPUs are idle for part of the day. That idle time is the price of availability. It often costs less than a missed target.

In the example above, the cost-only answer is 6 GPUs. Suppose that you must have 10 warm GPUs at peak to meet your latency target. Reserve 10 GPUs. The reservation costs $17,928.00 and per-second use costs $1,080.00, for a total of $19,008.00 per month. That is $5,011.20 (36%) more than the cost-only answer. It guarantees the capacity that your target needs.

Size the reservation in two steps:

1. Cost. Reserve each GPU level that your traffic holds for more than the break-even share of hours.
2. Requirements. Find the number of GPUs that must be warm to meet your latency and throughput targets at peak. Then make sure that the burst band in your contract covers the rest of your peak.

Reserve the larger of the two answers. If the answers are far apart, you must choose between cost and performance. A latency-critical, customer-facing workload usually needs the larger answer. A batch workload or an internal workload can use the smaller answer.

Other differences: capacity, cold starts and price certainty

Cost is not the only difference between reserved capacity and per-second capacity.

• Capacity. Per-second capacity is shared. Across the industry, per-second GPU pools get smaller during capacity shortages. Reserved capacity is held for you. Other customers' traffic cannot use it. Ask any provider: what capacity do you guarantee for me during a shortage?
• Cold starts. Requests inside a reservation run on warm runners, so they do not get cold starts. Per-second runners above the reservation can get cold starts, and you pay for their setup time.
• Price certainty. A reservation locks its rate and its capacity for the term. In a tight GPU market, that certainty has value of its own.

fal reports 99.99% historical uptime.

How reserved capacity works on fal

On fal Serverless, a reservation contract has three parts for each machine type:

• Reservation. Your floor of always-on GPUs. fal bills it every hour of the term at the reservation rate, and guarantees it for the term.
• Burst band. Capacity above the reservation, sized as a share of it. fal bills it per second at a negotiated burst rate.
• Further scale. Above the burst band, you can scale further at the per-second rate. This capacity is not reserved for you.

We size the reservation with you from your usage over 24 hours and over 7 days. We also ask whether cost or performance matters more for the workload. Contact sales to start.

fal also sells Compute instances: full H100 virtual machines at a fixed hourly rate. They suit training, and jobs that need a whole machine. This guide is about inference on Serverless.

Find your own break-even

1. Get a week of runner usage. Add up the GPU-seconds that your runners were alive, not only busy. The Platform API usage endpoint returns billed seconds by machine type.
2. Divide by the GPU-seconds in that week. The result is your billed utilisation.
3. Compare it with the break-even line for your GPU: about 55% for an H100, about 69% for a B200, at the prices above.
4. Plot your GPU count by hour, for 24 hours and for 7 days. Weekends are often different.
5. Calculate average GPUs ÷ peak-hour GPUs. The result is the utilisation of a fleet sized for your peak.
6. Size the reservation from the floor. Reserve each level that your traffic holds for more than the break-even share of hours.
7. Make sure that your burst band covers your peak.
8. Do this again when prices or traffic change.

Decision checklist

• Is your billed utilisation above the break-even line for your GPU?
• Is your warm floor permanent, every day of the week?
• Is the workload latency-critical and customer-facing?
• Do you have latency or throughput targets that a cold start or a wait for capacity would break?
• Can you forecast your floor for the next 6–12 months?
• Does your burst band cover your peak?
• What capacity does your current provider guarantee for you during a shortage?

More "yes" answers point to a larger reservation. More "no" answers point to more per-second capacity.

FAQ

Is serverless GPU cheaper than a dedicated GPU?

Below the break-even utilisation, per-second billing costs less. The break-even is about 55% for an H100 and about 69% for a B200, at fal's published prices. Above that line, a reserved GPU costs less.

How do I calculate GPU break-even utilisation?

Divide the reserved hourly rate by the per-second rate, expressed per hour. For an H100 on fal: $2.49 ÷ $4.50 = 55.3%.

At what utilisation does a reserved B200 beat serverless?

At about 69% billed utilisation, at fal's published prices. That is $5.49 ÷ $7.99.

Where does the $2.49 H100 rate come from?

$2.49 is fal's lowest published reserved rate for an H100. $4.50 is the published per-second rate. Reserved rates depend on the size and length of the commitment.

Can I combine reserved and serverless GPUs?

Yes. On fal, a reservation covers your floor. Traffic above the floor runs per second in a burst band. For most production workloads, this costs less than either option alone.

Does a reservation remove cold starts?

For traffic inside the reservation, yes. Runners above the reservation can get cold starts.

Why would I reserve more than the lowest-cost amount?

To guarantee availability. If you have latency or throughput targets, reserve the GPUs that must be warm to meet those targets at peak. Capacity above a reservation is not reserved for you, and it can have cold starts.

Does keep_alive time count toward my bill?

Yes. fal bills idle time, including the keep_alive window, at the machine type's rate.

Is setup time billed?

Yes. fal bills setup time, including model loading.

Are image pulls or failed requests billed?

fal does not bill pending time or image pulls. On Serverless, you pay for runner time, so fal bills the time that a runner spends on a failed request. On per-request Model APIs, fal does not charge for requests that fail with a 5xx error.

How are multi-GPU runners billed?

fal bills GPU count times duration. A 2-GPU runner that is alive for 60 seconds is billed for 120 GPU-seconds.

Should I use min_concurrency or a reservation for my warm floor?

For a permanent floor, use a reservation. fal bills a warm floor on per-second billing at the per-second rate all month. A reservation at a lower hourly rate costs less.

Sources

1. fal pricing: https://fal.ai/pricing
2. fal Serverless: https://fal.ai/serverless
3. fal docs, Serverless pricing: https://fal.ai/docs/documentation/serverless/pricing
4. fal docs, scaling parameter reference: https://fal.ai/docs/documentation/deployment/scale-your-application
5. fal docs, Serverless FAQ: https://fal.ai/docs/documentation/serverless/faq
6. fal docs, Why fal: https://fal.ai/docs/documentation/why-fal
7. fal docs, Compute: https://fal.ai/docs/documentation/compute

All sources accessed October 19, 2026.