NVIDIA A100 GPU: Specs, Price, and Performance in 2026

What Is the NVIDIA A100 GPU?

The NVIDIA A100 Tensor Core GPU is a data center accelerator built on NVIDIA's Ampere architecture, succeeding the V100 (Volta, 2017). Its core chip, the GA100, contains 54.2 billion transistors manufactured on TSMC's 7nm process node. The A100 was designed as a single platform for three workload categories: AI training, AI inference, and high-performance computing (HPC).

Three capabilities distinguish the A100 from its predecessor. First, third-generation Tensor Cores supporting TF32, BF16, FP16, INT8, and FP64 data types in a single chip, eliminating the need for separate training and inference GPUs. Second, Multi-Instance GPU (MIG) technology, which allows a single A100 to be partitioned into up to 7 fully isolated GPU instances, each with its own memory, compute, and bandwidth allocation. Third, structural sparsity acceleration, which doubles effective throughput for AI models with at least 50% sparse weights.

The A100 comes in two memory configurations and two physical form factors:

The SXM4 form factor is designed for NVIDIA DGX A100 and HGX A100 systems, where 8 GPUs connect via NVLink at 600 GB/s total bidirectional bandwidth. The PCIe variant fits standard server slots and costs less, but lacks NVLink connectivity for multi-GPU training.

From 2020 to 2023, the A100 was the GPU running every major AI training cluster. GPT-3 (175B parameters), Stable Diffusion 1.x, and early Meta LLaMA models were all trained on A100 infrastructure. According to NVIDIA's official A100 product page, the chip was deployed across research institutions, cloud providers, and enterprise data centers on every continent.

NVIDIA A100 Full Technical Specifications

The following table covers all three A100 variants against the H100 SXM5 for direct comparison. All TFLOPS figures are without sparsity unless noted.

A note on sparsity figures: when AI model weights have at least 50% near-zero values (the sparsity threshold NVIDIA uses), all tensor core throughput figures double. FP16 on the A100 reaches 624 TFLOPS with sparsity, and INT8 reaches 1,248 TOPS. Most production transformer models do not achieve the 50% sparsity threshold required for these figures, so the non-sparsity numbers are the ones that apply in practice.

The most meaningful difference between the A100 40GB and 80GB is not just memory capacity. The 80GB variant uses HBM2e rather than HBM2, delivering 2,039 GB/s versus 1,555 GB/s, a 31% bandwidth improvement. For memory-bandwidth-bound workloads, including large transformer inference at long sequence lengths, the 80GB delivers measurably higher throughput, not just larger model support.

NVIDIA A100 Price: What You Pay to Buy or Rent in 2026

The A100 trades across three markets in 2026: new hardware from authorized resellers, used hardware on secondary markets, and cloud rental by the hour. Each has distinct economics.

New hardware (2025-2026 market prices):

Used hardware (secondary market, 2025-2026):

• A100 80GB: $4,000-$9,000 depending on condition, original warranty status, and seller
• Used SXM4 modules require compatible HGX A100 server boards, adding significant integration cost

Cloud rental per GPU per hour (2026):

• A100 80GB on RunPod, Lambda Labs: $1.49-$2.00/hour
• A100 80GB SXM4 on CoreWeave: approximately $2.00-$2.50/hour
• AWS p4d.24xlarge (8x A100 40GB, full instance): approximately $32.77/hour ($4.10 per A100)
• H100 80GB SXM5 comparison: $2.50-$4.50/hour depending on provider

The Number Most Guides Don't Show

The per-hour cloud price comparison does not tell you which GPU is cheaper for a complete training run. Here is the full calculation.

The H100 SXM5 delivers 989 TFLOPS in FP16. The A100 SXM4 delivers 312 TFLOPS. The H100 provides 3.17x more raw training throughput.

Training a 7B parameter language model (comparable to Llama 2 7B) on a 128-GPU cluster:

• On A100 at $2/hour per GPU: 128 GPUs x 168 hours (7 days) x $2 = $43,008 total
• On H100 at $3.50/hour (3.17x faster, so 2.2 days): 128 GPUs x 52.8 hours x $3.50 = $23,654 total
• H100 saves approximately $19,000 per training run at this scale despite the higher hourly rate

For inference at low utilization, the math reverses. An A100 handling 15 inference requests per second uses roughly 30% of its compute capacity. The same load on an H100 uses about 9% of its capacity. Both GPUs sit mostly idle, but the H100 costs 75% more per hour for identical output. Below approximately 40% sustained GPU utilization, the A100 is the more cost-efficient inference server by a factor of 1.5 to 2.

This divide explains the current market split: large-scale training has migrated to H100 and H200 clusters, while stable inference workloads at moderate traffic levels continue running cost-efficiently on A100 hardware.

What AI Workloads Run on the A100?

The A100 covers the full range of deep learning workloads, though its position in the compute hierarchy has shifted since the H100 arrived in 2022-2023.

AI training:

• Large language model pre-training: GPT-3 (175B parameters), DALL-E, Stable Diffusion 1.x, and Meta's original LLaMA were trained on A100 clusters. These represented the frontier in 2020-2022.
• Fine-tuning models up to 13B parameters: the A100 80GB handles full-parameter fine-tuning of 7B-13B models in FP16. LoRA fine-tuning of 70B models is feasible with gradient checkpointing.
• Vision transformer training: standard image classification, object detection, and segmentation training workloads fit within the A100's memory and compute envelope.

AI inference:

• API inference for models up to 13B parameters: providers including Together AI, Baseten, and smaller cloud GPU operators use A100 clusters for serving Llama 2 7B, Mistral 7B, and similar models.
• MIG-based multi-tenant inference: a single A100 splits into 7 isolated GPU instances, each capable of serving a separate model. This makes a single A100 economically efficient for diverse inference API deployments.
• Real-time NLP: NVIDIA's own benchmarks show the A100 delivering inference throughput 249 times faster than a CPU server on BERT-class models (NVIDIA, 2020), making sub-100ms API response times achievable.

High-performance computing (non-AI):

• Molecular dynamics simulation and protein structure prediction
• Climate and weather modeling
• Financial risk modeling requiring FP64 double-precision support (9.7 TFLOPS FP64)

For context on the hyperscale facilities where A100 clusters are deployed, see our overview of what hyperscalers are. For organizations considering colocation of owned A100 hardware, our article on colocation data center costs covers the full pricing structure.

NVIDIA A100 vs H100: When Does the Upgrade Make Sense?

The H100 (Hopper architecture, TSMC 4nm) represents a full architectural generation over the A100. The key additions are: FP8 training precision (not available on A100), fourth-generation Tensor Cores, a dedicated Transformer Engine that automatically optimizes between FP8 and FP16 precision mid-operation, and NVLink 4.0 at 900 GB/s versus the A100's 600 GB/s.

Performance gap by metric:

"The H100 with its Transformer Engine is the first GPU designed specifically for transformer model architecture. The A100 is a general-purpose accelerator that happens to work well for transformers." (NVIDIA Hopper Architecture Technical Brief, 2022)

When upgrading to H100 makes sense: 1. Training runs exceeding 72 hours on A100 clusters, where the 3.2x throughput difference converts to direct cost savings (as shown in the pricing section above). 2. Models requiring FP8 precision for training efficiency, particularly models above 30B parameters where memory becomes the constraint. 3. Organizations where researcher and engineering time cost more than GPU time, and faster iterations justify the higher hardware spend.

When staying on A100 makes sense: 1. Inference-dominant workloads at less than 40% sustained GPU utilization. 2. Existing owned A100 hardware not yet fully depreciated, where the replacement cost does not recover through training speed savings within the depreciation window. 3. Budget-constrained research teams where $1.49-$2.00/hour A100 rental is the practical ceiling for compute spend.

Multi-Instance GPU and Ampere Architecture Details

The A100 introduced two architectural features not present in the V100 that remain significant for deployment in 2026: Multi-Instance GPU (MIG) and structural sparsity support.

Multi-Instance GPU (MIG)

MIG allows a single A100 to be partitioned into up to 7 fully isolated GPU instances, each with its own dedicated slice of the GPU's compute, memory, and bandwidth resources. Each instance appears to the operating system and running software as a separate physical GPU, with complete isolation: a crash or memory error in one instance does not affect others.

MIG partition sizes on an A100 80GB (selected options):

For inference API operators, MIG is a material cost saver. A single A100 at $2/hour that serves 7 concurrent model instances costs $0.29/hour per effective GPU equivalent. The same workload on 7 separate cloud GPU instances would cost $10.43/hour.

Third-Generation Tensor Cores and Sparsity

The A100's Tensor Cores added TF32 (TensorFloat-32) precision, a 19-bit format designed to match FP32 range with FP16-equivalent computation speed. TF32 delivers 156 TFLOPS on the A100 and requires no code changes from standard FP32 training, making it an automatic speed improvement for PyTorch and TensorFlow users.

Structural sparsity support accelerates computations where at least 50% of matrix values are zero. The A100 can prune and compress sparse matrices at the hardware level, theoretically doubling FP16 throughput to 624 TFLOPS. The limitation: generating the 2:4 sparsity pattern that the A100 hardware requires (exactly 2 non-zero values in every group of 4) requires specific training procedures that most production models do not currently use.

For a full picture of how the A100 fits into the broader GPU data center hardware landscape, the NVIDIA A100 product specifications page remains the authoritative reference for official benchmark figures.

Is the A100 Still Worth Using in 2026?

The A100 is over five years old in 2026. In GPU hardware terms, that is a significant generational gap. The H100 and H200 are both commercially available, and NVIDIA's Blackwell architecture (B100, B200) is entering volume production. Yet the A100 continues to serve a defined and economically rational role.

Three conditions favor continued A100 use in 2026:

1. Inference at moderate scale: models up to 13B parameters in FP16 inference, at request volumes below roughly 40% GPU utilization, run more cost-efficiently on A100 than H100. The math was shown in the pricing section above. This covers a large fraction of production AI API deployments that are not serving millions of requests per hour.

2. Owned hardware not yet depreciated: organizations that purchased A100 hardware in 2022-2023 at peak prices have assets with remaining useful life. Replacing them with H100s before the A100s are fully depreciated incurs a double capital cost that cannot be recovered through training speed savings for most workload profiles.

3. Budget-constrained teams: at $1.49-$2.00/hour for cloud rental, A100 is the lowest-cost entry point for running 7B-13B parameter model training and fine-tuning without resorting to consumer GPU workarounds. University research labs, small AI startups, and individual developers operate in this range.

The conditions where A100 is no longer competitive:

• Training runs for frontier models (above 30B parameters) where FP8 precision and H100 Transformer Engine give a 4-6x effective throughput advantage.
• Multi-node training at scale (128+ GPUs), where the NVLink bandwidth gap (600 GB/s vs H100's 900 GB/s) adds communication overhead that lengthens training wall-clock time.
• Applications requiring the latest CUDA features and toolkits that NVIDIA is targeting specifically at Hopper and Blackwell architectures.

The A100's market position in 2026 is similar to the V100's position in 2022: clearly not the leading chip, but far from obsolete, and more cost-effective than its successor for a defined set of workloads.

Frequently Asked Questions

Related Articles