NVIDIA H100 Price in 2026: Specs, Hourly Rates and Buy vs Rent

Four years after NVIDIA announced it, the H100 is still the reference point for AI infrastructure: newer GPUs are pitched as multiples of it, and most inference stacks have been tuned on it. What has changed is the buying question. Supply is no longer the main constraint, so the real decision is how to pay for H100 hours, and that comes down to how many of those hours your GPUs are actually busy.

H100 specs compared with the A100 and H200

The H100 was the first data-center GPU on NVIDIA's Hopper architecture, announced on March 22, 2022.¹ Its biggest step over the A100 was FP8 Tensor Core math, which doubles peak throughput over FP16 for models that can run at 8-bit precision.

Hopper figures are NVIDIA's sparsity figures halved to dense.²³⁴ Retail listings are PCIe cards. NVIDIA's current one, the H100 NVL, has 94 GB at 3.9 TB/s and 350–400 W.² SXM modules ship inside 4- or 8-GPU servers, and fal's dedicated H100 instances use them.⁵

When an H100 is the right GPU

• Models that fit in 80 GB. At FP16, weights take about 2 GB per billion parameters, so models up to about 30B fit on one H100 with room for KV cache. At FP8 it is about 1 GB per billion, so a 70B model with long contexts or large batches spills onto two H100s.
• LLM serving and training. The H100 remains a strong default for both. For the largest models, the H200's extra memory is often the better buy, as the comparison below shows.⁶
• Not for video generation. The H100 has no hardware video encoder, so video pipelines are better served by GPUs that do, such as the RTX PRO 6000 or L40.⁶

What an H100 costs to rent

Renting by the hour is now the common way to get H100 capacity. On fal, an H100 lists at $4.50 per GPU-hour, billed per second:

Source: fal.ai/pricing.⁷ At list price that is $0.00125 a second.

Per-second or reserved by the hour

H100 rentals come in two shapes across the industry: serverless capacity billed per second, and dedicated instances billed by the hour whether used or not. fal offers both:

Sources: fal Machine Types, Compute and Compute pricing docs.⁶⁵⁸

On Serverless, time waiting for hardware or pulling your image is not billed, and requests that fail with a 5xx error are not charged. A multi-GPU runner bills GPU count times duration, so a 2× H100 runner costs $9.00 per hour at list price.⁹ Scaling to zero means a cold start on the next request. To pick between the two for your traffic, see Dedicated GPUs vs Serverless Inference: How to Find Your Break-Even.

What common H100 workloads cost

Keeping the 8B runner warm all month would cost eight times the 3-hours-a-day pattern. That holds on any per-second platform: for bursty traffic, how long GPUs stay warm moves the bill more than the hourly rate does.

Two H100s or one H200?

A 70B model at FP8 that needs two H100s for KV cache headroom usually fits on one H200 (141 GB):

The H200 costs a third less here and avoids splitting the model. The H100 stays cheaper whenever the model fits on one H100 with room to spare. See the NVIDIA H200 price guide.

Should you buy or rent an H100?

Break-even is the card price divided by the hourly rental rate:

Purchase prices are two US retail listings.¹⁰¹¹ Months use 720 hours.

This flatters buying. It leaves out the server, power, cooling, networking, space and staff. Eight cards alone cost about $218,000 to $296,000, and retail PCIe cards lack the 900 GB/s NVLink of the SXM modules in fal's H100 instances.²⁵ Buying makes sense only if you will keep the GPUs busy for years and already run data-center space. Otherwise, rent.

H100 vs A100 on price

One US retailer lists the A100 80 GB PCIe card at $16,535.¹² The H100 SXM has about 1.6 times the A100's memory bandwidth and adds FP8, so for a bandwidth-bound model an H100 hour does about 1.6 times the work. If an H100 hour costs less than 1.6 times an A100 hour, the H100 is usually cheaper per output.

Running a model on H100s with fal

Set the machine type in your app; Serverless access is approved per account.⁶

class MyApp(fal.App):
    machine_type = "GPU-H100"
    num_gpus = 1

• num_gpus: use the smallest count your model fits on. Each extra H100 adds $4.50 per runner-hour at list price.
• machine_type as a list, such as ["GPU-H100", "GPU-H200"], falls back to the next type when the first has no free capacity.⁶ Check each type's rate.
• keep_alive, min_concurrency and max_concurrency decide how many runners stay warm. See Scale Your Application.

New to fal? Start from the Quick Start, or compare options in How to choose a serverless GPU platform.

FAQ

How much does an NVIDIA H100 cost?

US retailers list a new H100 80 GB PCIe card at about $27,000 to $37,000. SXM H100s are sold inside 4- or 8-GPU servers, priced by quote.

What is the H100 price per hour?

On fal Serverless, $4.50 per GPU-hour at list price, billed per second ($0.00125 a second). It goes as low as $2.49 per GPU-hour.

Is it cheaper to buy or rent an H100?

For most teams, renting. At fal's list price a card costs about 6,000 to 8,200 rented GPU-hours: 8 to 11 months of round-the-clock use, or 5 to 8 years at 3 hours a day, before server, power and staff costs.

How much does an A100 cost compared with an H100?

Less to buy: one US listing has the A100 80 GB PCIe at $16,535, against about $27,000 to $37,000 for an H100. The H100's extra bandwidth and FP8 often make it cheaper per output.

References