NVIDIA H200 GPU: Full Specs, Price, and Cloud Rates for 2026

What is the NVIDIA H200 GPU?

The NVIDIA H200 is a data center GPU built on the same Hopper GH100 die as the NVIDIA H100, with one change that matters: it replaces 80GB of HBM3 memory with 141GB of HBM3e running at 4.8 TB/s, up from the H100 SXM5's 3,350 GB/s. NVIDIA designed it for workloads that run out of memory before they run out of compute, which describes most large language model training and inference today. For a broader primer on this category of hardware, see our explainer on what an AI accelerator is.

It ships in two form factors, matching the H100 lineup:

Both variants use the same GH100 silicon as the H100, with 80 billion transistors fabricated on TSMC's 4nm process. NVIDIA did not redesign the compute pipeline. It widened the memory pipe.

"The NVIDIA H200 Tensor Core GPU is the first GPU to offer HBM3e memory, delivering the largest capacity and fastest memory in the world." (NVIDIA, H200 product page, 2024)

NVIDIA H200 full specifications

The H200's spec sheet looks almost identical to the H100's outside of the memory subsystem. Here's the full comparison against the previous two generations:

Look at the FP8 throughput row. It's identical between H100 and H200, because both run on the same GH100 die at the same clock targets. The only specification that changed between generations is memory capacity and bandwidth. For a workload that already fits in 80GB and doesn't bottleneck on memory bandwidth, an H200 will not run noticeably faster than an H100. For one that doesn't fit, the difference can be the gap between needing two GPUs and needing one.

How much does the NVIDIA H200 cost in 2026?

NVIDIA does not publish list pricing for the H200, so the numbers below come from OEM quotes, reseller listings, and cloud provider rate cards tracked through 2026.

Outright purchase, per GPU:

Cloud rental, per GPU-hour, varies more than purchase price because providers bundle networking, storage, and support differently:

The number most guides don't show

AWS raised pricing on its 8-GPU H200 instances in January 2026. The p5e.48xlarge instance went from $34.61/hr to $39.80/hr, a $5.19/hr increase, according to Introl's analysis of the change. Run that instance continuously for a year and the increase alone adds $45,464 in extra cost, for hardware that didn't change at all between December 2025 and January 2026. That's close to the price of a brand new H200 PCIe card, lost entirely to a pricing adjustment in a single month. Introl frames the hike as a sign of structural supply constraints in high-end AI accelerators, not ordinary demand growth, which is the clearest evidence that H200 capacity, not B200 capacity, was the actual bottleneck heading into 2026.

H200 vs H100: what actually changed

Memory is the only meaningful difference between these two GPUs, but it shows up differently depending on the workload.

For memory-bound workloads, Fluence's H200 deep dive reports training speedups around 1.4x and inference speedups up to 1.8x over the H100, purely from the bandwidth increase letting the GPU feed its tensor cores faster. For compute-bound workloads that already fit comfortably inside 80GB, the gain shrinks toward zero, since both GPUs run the same compute pipeline at the same clocks.

The practical effect: a 70B-parameter model that needed two H100s to fit comfortably with room for a long context window can often run on a single H200. That's not a speed improvement so much as an elimination of a problem. Fewer GPUs needed per model means less cross-GPU communication overhead and simpler deployment math.

Who actually needs an H200 over an H100?

The H200 is worth the premium for workloads that hit memory limits on an H100, not for workloads that are simply slow.

Good fit for H200:

• Training or fine-tuning models above 70B parameters where activation memory and KV cache eat into the 80GB H100 ceiling
• Long-context inference, 32K tokens and up, where the KV cache grows large enough to force batch size down on an 80GB card
• Multi-tenant inference serving where higher memory per GPU means more concurrent users without adding GPUs

Better off on H100 or A100:

• Fine-tuning smaller models, under 13B parameters, that fit in 80GB with room to spare
• Inference workloads with short context windows and small batch sizes
• Budget-constrained teams where the 15-20% price premium doesn't pay back in fewer GPUs needed

If your workloads fit comfortably in less memory, the NVIDIA A100 remains the better value choice, especially on the secondary market. Teams training smaller models on a tighter budget should look at our guide to the best GPU for AI training, which covers consumer and prosumer cards like the RTX 4090 that cost a fraction of an H200. Hyperscalers, model labs, and large enterprises running frontier-scale LLM training are the buyers actually driving H200 demand. By early 2026, AWS, Google Cloud, Microsoft Azure, and Oracle had all rolled out H200-backed instances, alongside specialized providers offering per-GPU rental for smaller teams that don't want to commit to a multi-year reserved instance.

Should you wait for the B200 instead?

NVIDIA's Blackwell-based B200 is the direct successor to the H200, and it's already shipping in volume to large customers as of 2026, which raises an obvious question for anyone pricing out H200 capacity now.

B200 offers more memory, better performance per watt, and roughly double the total system throughput of H100-class hardware in NVIDIA's own positioning, at a similar or only slightly higher price per GPU. The catch is availability. H200 is a drop-in replacement for H100 in existing HGX and DGX infrastructure, so it ships faster and integrates into systems that are already racked and validated. B200 generally requires newer power and cooling infrastructure.

For teams that need capacity now and already run H100 fleets, H200 remains the path of least resistance. For new builds with no existing Hopper infrastructure to protect, waiting for B200 allocation is increasingly the better economic call, assuming the wait doesn't cost more in delayed training runs than the price difference saves.

Common misconceptions about the H200

A few claims about the H200 circulate widely and don't hold up against NVIDIA's own specifications.

It's not a new architecture. Some marketing material describes the H200 as a generational leap, but it uses the identical GH100 die, the same 16,896 CUDA cores, and the same Transformer Engine as the H100. The only architectural change is the memory subsystem.

It doesn't have more CUDA cores. A handful of vendor listings claim up to 30% more CUDA cores than the H100. Independent hardware databases, including TechPowerUp, list the same 16,896 shading units for both GPUs. The compute side is unchanged.

It's not a gaming or workstation card. The H200 has no display outputs and doesn't support DirectX, Vulkan, or OpenGL. It's built exclusively for data center racks, not desktops.

It doesn't make every workload faster. The performance gain is workload-dependent. Memory-bound jobs see a real uplift. Compute-bound jobs that already fit in 80GB see little to no difference, because the underlying tensor core throughput didn't change.

Frequently Asked Questions

Related Articles