The H200 is the GPU that made memory, not compute, the headline spec. It runs the same Hopper compute as the H100 but carries 76% more memory, which changes how many GPUs a large model needs. For inference, that makes the price of one model replica a better guide than the price of one GPU.
What the NVIDIA H200 is
The H200 is NVIDIA's Hopper-generation data-center GPU. It keeps the H100's compute and power profile, up to 700 W per GPU, but has 141 GB of HBM3e instead of 80 GB, and 4.8 TB/s of memory bandwidth, about 1.4 times the H100.12 The H200 SXM goes into 4- and 8-GPU HGX servers. The H200 NVL is a PCIe card for air-cooled servers, at up to 600 W.1
| Spec | H200 (SXM) | H100 (SXM) | B200 |
|---|---|---|---|
| Architecture | Hopper | Hopper | Blackwell |
| GPU memory | 141 GB HBM3e | 80 GB HBM3 | 192 GB HBM3e |
| Memory bandwidth | 4.8 TB/s | 3.35 TB/s | 8 TB/s |
| FP8 dense (TFLOPS) | ~1,979 | ~1,979 | ~4,500 |
| FP16/BF16 dense (TFLOPS) | ~989 | ~989 | ~2,250 |
| NVLink per GPU | NVLink 4 (900 GB/s) | NVLink 4 (900 GB/s) | NVLink 5 (1.8 TB/s) |
| Max power | Up to 700 W per GPU | Up to 700 W per GPU | ~14.3 kW per 8-GPU DGX B200 |
Hopper dense figures halve NVIDIA's with-sparsity numbers. B200 throughput is NVIDIA's 8-GPU figure divided by eight; B200 memory is fal's GPU-B200 machine type.12345
With the same compute, the H200 beats the H100 only on memory-bound work, which covers most inference. NVIDIA reports up to 1.9 times faster Llama 2 70B inference.1
When an H200 is the right GPU
- 70B-class models on one GPU. At FP8, a 70B model's weights take about 70 GB. That leaves an 80 GB H100 little room for KV cache, so long contexts push the model onto two GPUs. On an H200 it fits on one.
- Long-context and high-batch LLM serving. The extra 61 GB holds more KV cache, and the bandwidth reads it faster. This is where the H200 earns its premium over the H100.
- Not for most image or video work. Image models rarely need 141 GB and run well on cheaper GPUs such as the L40 or RTX PRO 6000. Video pipelines benefit from hardware video encoders, which the H200 lacks.4
What an H200 costs to buy
Reseller and negotiated prices move with supply.
| Source | Price per H200 | Date |
|---|---|---|
| US online market prices | ~$30,000 to $34,000 | March 2026 |
| Reported price offered to buyers in China | ~$27,000 | December 2025 to March 2026 |
| Lowest prices reported by industry sources | ~$17,000 to $19,000 | March 2026 |
All three come from one policy analysis, which notes that large buyers negotiate discounts.6 A December 2025 news report gave the same ~$27,000 China price.7 The GPU is only part of the cost. You also pay for the 4- or 8-GPU server, networking, power, cooling and operations staff.
Rental prices have not fallen. Across 28 cloud providers, the median on-demand H200 rate rose from about $3.20 per GPU-hour at the end of 2024 to about $4.00 at the end of 2025.6
What an H200 costs to rent
On fal, an H200 lists at $6.00 per GPU-hour, from fal.ai/pricing:8
| GPU | VRAM | List price | As low as |
|---|---|---|---|
| H200 | 141 GB | $6.00/h | $2.99/h |
$6.00 is the per-GPU list rate, billed per second while a runner is alive, idle keep-alive included.9 To compare per-second platforms, see how to choose a serverless GPU platform.
At list price, an H200 second costs about $0.00167.
| Workload | Rate | Billed GPU time | Cost |
|---|---|---|---|
| 10-minute test run on 1 GPU | List | 600 GPU-seconds | $1.00 |
| 1 hour on an 8-GPU runner | List | 8 GPU-hours | $48.00 |
| Bursty traffic: 3 runner-hours a day for 30 days | List | 90 GPU-hours | $540.00 |
| 1 GPU kept warm for a 30-day month | List | 720 GPU-hours | $4,320.00 |
| 1 GPU held for a 30-day month | As low as, $2.99/h | 720 GPU-hours | $2,152.80 |
The "as low as" rate is about half the list rate, so holding a GPU all month at that rate beats per-second billing at list only when the GPU is busy more than about half the time. The break-even guide works through that line.
What a 70B model replica costs on H100, H200 and B200
Memory sets how many GPUs one copy of a model needs. For a 70B model at FP8 with long contexts:48
| GPU | Memory per GPU | GPUs per replica | List price per replica-hour | As low as per replica-hour |
|---|---|---|---|---|
| H100 | 80 GB | 2 | $9.00 | $4.98 |
| H200 | 141 GB | 1 | $6.00 | $2.99 |
| B200 | 192 GB | 1 | $7.99 | $5.49 |
H200 vs H100. An H200 replica costs a third less than two H100s at list price, and 40% less at "as low as" rates. For a small model that fits on one H100, the H200 lists at 1.33 times the price for 1.43 times the bandwidth, so it wins only if the model is bandwidth-bound. For the H100 side, see the NVIDIA H100 price guide.
H200 vs B200. A B200 replica costs 1.33 times an H200 replica at list price and 1.84 times at "as low as" rates, for 1.67 times the bandwidth. The B200 wins on cost per token only if its extra bandwidth turns into at least that much more throughput. If your model fits in 141 GB, benchmark both. The NVIDIA B200 price guide covers that GPU.
Buy or rent an H200?
The purchase price divided by the hourly rate gives the rental hours the same money buys:
| Purchase price per GPU | Rent at $6.00/h (list) | Rent at $2.99/h (as low as) |
|---|---|---|
| $27,000 | 4,500 h (~188 days of 24/7 use) | 9,030 h (~376 days) |
| $30,000 | 5,000 h (~208 days) | 10,033 h (~418 days) |
| $34,000 | 5,667 h (~236 days) | 11,371 h (~474 days) |
The purchase price covers the GPU alone, so this flatters buying.68 Buy only if you can keep H200s busy for years and already run data-center hardware.
Running a model on H200s with fal
On fal Serverless the H200 is the GPU-H200 machine type, with 141 GB of VRAM, 112 GB of system RAM and 12 CPU cores.4 Set machine_type = "GPU-H200" in your app, or list ["GPU-H200", "GPU-B200"] so fal falls back to the next type when H200s have no capacity.4 fal approves Serverless access per account. Request it on the Serverless get-started page, then follow the Quick Start.10 Keep-alive settings drive most of an H200 bill; see What Is a Serverless GPU Cold Start?
FAQ
How much does an NVIDIA H200 cost?
About $30,000 to $34,000 per GPU at US online prices in March 2026, and negotiated deals go lower.6 On fal Serverless it rents for $6.00 per GPU-hour at list price, or as low as $2.99 per GPU-hour.
What is the H200 price per hour?
On fal, $6.00 per GPU-hour at list price, billed per second, which is about $0.00167 per second. It goes as low as $2.99 per GPU-hour.
What is the H200 vs B200 price?
On fal the B200 lists at $7.99/h and the H200 at $6.00/h. At "as low as" rates, the gap widens to $5.49/h against $2.99/h. Choose a B200 when the model needs more than 141 GB, is strongly bandwidth-bound, or can run in FP4.
References
References
-
NVIDIA H200 GPU, NVIDIA. ↩ ↩2 ↩3 ↩4
-
NVIDIA H100 GPU, NVIDIA. ↩ ↩2
-
NVIDIA DGX B200, NVIDIA. ↩
-
NVIDIA HGX Platform, NVIDIA. ↩
-
Georgia Adamson and Tim Fist, When Do More AI Chips for China Mean Fewer for the United States?, Institute for Progress, March 27, 2026. ↩ ↩2 ↩3 ↩4
-
Reuters, Nvidia sounds out TSMC on new H200 chip order as China demand jumps, sources say, via The Standard (Hong Kong), December 31, 2025. ↩
-
fal Pricing. ↩ ↩2 ↩3



