What Are AI Chips? GPUs, Pricing, and Chip Wars Explained
Key Numbers
Key Takeaways
- 1An AI chip is any processor built for the parallel matrix math behind machine learning. NVIDIA GPUs dominate the category, holding an estimated 70 to 80% of the data center AI accelerator market.
- 2A new NVIDIA H100 GPU costs $25,000 to $40,000 in 2026. Epoch.ai estimates the B200's component cost at $5,700 to $7,300, meaning street prices run 4 to 7 times the bill of materials.
- 3Nearly every advanced AI chip is fabricated by one company, TSMC in Taiwan. That concentration is why US export controls on China and the 2022 CHIPS Act both treat chip manufacturing as a national security issue.
An AI chip is a processor built to run one specific kind of math very fast: the matrix multiplication that powers neural networks. The category technically includes GPUs, TPUs, NPUs, and custom ASICs, but in practice "AI chip" usually means an NVIDIA GPU, since NVIDIA holds an estimated 70 to 80% of the data center AI accelerator market by revenue.
Here is the detail most coverage skips. A single company, Taiwan Semiconductor Manufacturing Company (TSMC), fabricates nearly every advanced AI chip on the market, including NVIDIA's, AMD's, and the custom designs built by Google, Amazon, and Microsoft. That concentration, not the chips themselves, is the real reason the US and China have spent the past three years fighting over export rules.
By the end of this article you will know what actually separates an AI chip from a regular CPU, why a single H100 GPU costs as much as a down payment on a house, why governments treat a chip factory in Taiwan as a matter of national security, and whether the AI chip shortage you keep hearing about still exists in 2026.
In This Article
What Is an AI Chip, Exactly?
An AI chip is a processor designed around one task: running massive numbers of parallel calculations, mostly matrix multiplication, fast and efficiently. A regular CPU does the opposite. It runs a small number of complex instructions one after another, which works well for web browsers and spreadsheets but poorly for the math a neural network needs.
The Center for Security and Emerging Technology (CSET) at Georgetown University defines AI chips as chips that trade away general-purpose flexibility for speed and efficiency on AI-specific calculations. That trade-off is the entire story.
| CPU (e.g. Intel Xeon) | AI chip (e.g. NVIDIA H100) | |
|---|---|---|
| Core count | 8 to 192 complex cores | 16,896 simpler CUDA cores |
| Designed for | Sequential, branching logic | Parallel matrix math |
| Memory type | DDR, lower bandwidth | HBM, up to 3.35 TB/s |
| Typical AI role | Orchestration, light inference | Model training, heavy inference |
GPUs are the most common AI chip, but the category also includes TPUs, NPUs, and ASICs. We cover the full breakdown of accelerator types, specs, and prices in our guide to AI accelerator cards. For this article, the short version: when a headline about export bans or NVIDIA earnings says "AI chip," it almost always means a GPU.
What Kind of Chips Does AI Actually Use?
Most AI today runs on one of four chip types: GPUs, TPUs, NPUs, or ASICs. GPUs handle the overwhelming majority of training workloads. The other three show up in more specific places.
| Chip type | Who makes it | Where it runs | Best at |
|---|---|---|---|
| GPU | NVIDIA, AMD | Cloud data centers, on-prem clusters | Training large models, flexible workloads |
| TPU | Google (custom) | Google Cloud only | Training inside Google's own stack |
| NPU | Apple, Qualcomm, Intel | Phones, laptops | On-device inference, low power |
| ASIC | Amazon (Trainium), Microsoft (Maia) | AWS, Azure only | Cost-efficient training and inference at one company's scale |
NVIDIA's H100 and the newer B200 are GPUs, originally designed to render video games, repurposed for AI because the same parallel architecture that draws millions of pixels at once also multiplies millions of matrix values at once. Tensor cores, the specialized circuits inside modern GPUs, perform that multiplication in hardware rather than software, which is the main reason a GPU beats a CPU on AI workloads by an order of magnitude or more.
If you are choosing hardware to run a model yourself rather than reading about the industry, the calculus is different from anything in this article. Our guide to the best local LLM models covers what you can realistically run on consumer GPU memory, which for most people means a single RTX-class card, not a data center chip.
Why Are AI Chips So Expensive?
A new NVIDIA H100 costs $25,000 to $40,000 per unit as of Q1 2026, and the newer B200 runs $30,000 to $40,000, according to NVIDIA's H100 product specifications and current market pricing. Three things drive that price: scarce manufacturing capacity, a software moat, and margin.
Manufacturing capacity is the most concrete constraint. Every H100 and B200 is fabricated by TSMC on advanced process nodes (4nm for H100, a custom 4NP node for B200) and packaged using CoWoS, a technique that stacks high bandwidth memory directly onto the chip. TSMC's CoWoS packaging capacity was the primary bottleneck on H100 supply through 2023 and 2024, and building new packaging capacity takes years, not quarters.
The software moat compounds the price. NVIDIA has invested in its CUDA software platform for more than 15 years, and most AI frameworks, including PyTorch and TensorFlow, run on CUDA first. Switching to a cheaper chip from AMD or Intel means re-engineering software, not just swapping hardware, which lets NVIDIA price well above the cost of materials without losing customers.
The math nobody walks you through
NVIDIA's H100 costs $25,000 to $40,000 to buy versus a market median of $2.29 per hour to rent. (Fluence Network, 2026)
That single comparison explains why renting beats buying for most teams. At $2.29 an hour, the purchase price of an H100 equals roughly 15 to 24 months of nonstop rental at the median rate. Most training runs last weeks, not years, which is why most AI companies rent capacity instead of buying chips outright, and why a healthy secondary market for used H100s exists at all.
The margin side of the story is just as direct. Epoch.ai estimated the bill of materials for a B200, the components, packaging, and assembly, at $5,700 to $7,300 per unit. Against a $30,000 to $40,000 street price, that puts NVIDIA's markup at roughly 4 to 7 times the cost of parts. We break down what that markup means for NVIDIA's actual profit margin in our Blackwell architecture deep dive.
Why Are Governments Fighting Over AI Chips?
Governments are fighting over AI chips because almost all of them come from one factory complex, in one country, that sits roughly 130 kilometers from a rival superpower. TSMC fabricates nearly every advanced AI chip on the market: NVIDIA's, AMD's, and most of the custom chips built by Google, Amazon, and Microsoft. Taiwan's near-monopoly on leading-edge chip manufacturing makes it, by most estimates, the single most consequential piece of geography in the AI industry.
The United States has responded on two fronts. Domestically, the 2022 CHIPS and Science Act committed $52 billion to subsidize US chip manufacturing, an attempt to reduce dependence on any single country. Internationally, the US has restricted which AI chips can be exported to China.
The timeline matters. In October 2022, the US Bureau of Industry and Security imposed export controls on NVIDIA's A100 and H100, blocking sales of chips above a certain performance threshold to China. NVIDIA responded with the A800 and H800, China-specific variants with reduced interconnect bandwidth designed to fall just under the threshold. In November 2023, updated rules closed that gap and restricted the H800 and A800 as well. As of 2026, NVIDIA has no H100-equivalent product approved for the Chinese market, a topic we cover in more depth in our H100 specs and pricing guide.
China was an estimated 20 to 25% of NVIDIA's data center revenue before the controls began. The company has absorbed that loss without much difficulty, since demand from the US, Europe, and the rest of Asia has grown faster than the lost China revenue. China, in turn, has accelerated its own chip industry. Huawei's Ascend 910C is the most capable domestic alternative, delivering an estimated 60 to 80% of H100 performance on standard training workloads, though its software ecosystem remains far behind CUDA.
Researchers at Georgetown's Center for Security and Emerging Technology have made a similar case in their chip policy work: as AI models grow more capable, access to advanced chips increasingly determines which countries can compete at the frontier of AI development. None of this is really about the chips. It is about who controls the only factory complex capable of making them.
Is There Really an AI Chip Shortage in 2026?
Not in the way the 2023 headlines described it. H100 order lead times have fallen to roughly 6 to 12 weeks as of Q2 2026, down from the six-plus-month waits that defined the chip famine of 2023, when demand for ChatGPT-style models outpaced everyone's production capacity at once.
What looks like scarcity today is really an allocation problem on the newest chips, not a supply problem across the board. NVIDIA's B200 remains tightly allocated to hyperscalers and large cloud providers as production ramps, with most enterprise buyers accessing it through multi-year cloud contracts rather than open market purchases. H100 supply, on the other hand, has normalized enough that a healthy secondary and rental market exists, including used units priced well below new ones.
The underlying constraint behind the 2023 shortage, TSMC's CoWoS advanced packaging capacity, has expanded substantially since. That expansion is the main reason H100 availability improved faster than B200 availability: B200 needs the newest packaging generation, which is still catching up to demand, while H100 production runs on a more mature line.
So the honest answer is that the original AI chip shortage has largely eased, while a narrower allocation squeeze on the newest, highest-demand chips persists, and probably will every time NVIDIA launches a new architecture faster than TSMC can scale packaging for it.
Common Misconceptions About AI Chips
Three misconceptions come up constantly in coverage of AI chips, and they are worth correcting directly.
- Any fast chip is an AI chip. It is not. An AI chip's value comes from architectural choices, mainly parallel cores and tensor math units, built specifically for matrix operations. A fast CPU can still be slow at the exact calculations a neural network needs.
- AI chips think or contain intelligence. They do not. An AI chip executes linear algebra faster and more efficiently than a general-purpose chip. The intelligence, such as it is, lives in a model's weights and training, not in the silicon.
- CPUs are obsolete now that AI chips exist. They are not. CPUs still orchestrate workloads, run lighter inference tasks, and handle everything outside the matrix math that GPUs specialize in. Every AI data center runs CPUs and GPUs together, not GPUs alone.
Each of these misconceptions traces back to the same root confusion: treating "AI chip" as a marketing label rather than a specific set of engineering trade-offs.
Frequently Asked Questions
What are AI chips and why are they expensive?
AI chips are processors built for the parallel matrix math behind machine learning, most commonly NVIDIA GPUs. They cost $25,000 to $40,000 per unit because of scarce TSMC manufacturing capacity, NVIDIA's 15-year CUDA software moat that makes switching to cheaper alternatives costly, and gross margins that analysts estimate above 70% on the hardware itself.
What kind of chips does AI use?
AI primarily runs on four chip types: GPUs (NVIDIA, AMD), which handle most training and inference; TPUs, Google's custom chips available only through Google Cloud; NPUs, low-power chips in phones and laptops for on-device AI; and ASICs like AWS Trainium and Microsoft Maia, built by cloud providers for their own infrastructure.
Why are governments fighting over AI chips?
Because nearly every advanced AI chip is manufactured by one company, TSMC, in Taiwan. The US has restricted AI chip exports to China since October 2022 to limit its access to frontier AI compute, while also funding $52 billion in domestic chip manufacturing through the 2022 CHIPS Act. Both moves treat semiconductor manufacturing as a national security issue, not just an economic one.
Why is there an AI chip shortage?
The severe 2023 shortage, when H100 lead times stretched past six months, was driven by a bottleneck in TSMC's CoWoS advanced packaging capacity needed to attach high bandwidth memory to each chip. That capacity has expanded significantly since, and H100 lead times had fallen to roughly 6 to 12 weeks by Q2 2026. A narrower allocation squeeze still affects the newest chips, like the B200, as production ramps.
What is the difference between a GPU and a CPU for AI?
A CPU has a small number of complex cores designed for sequential tasks. An NVIDIA H100 GPU has 16,896 simpler CUDA cores designed to run the same operation across massive amounts of data simultaneously, which is exactly what matrix multiplication in a neural network requires. CPUs also use slower DDR memory, while AI GPUs use high bandwidth memory (HBM) that moves data up to 3.35 TB/s.
Is NVIDIA the only company that makes AI chips?
No, but it dominates the category. NVIDIA holds an estimated 70 to 80% of the data center AI accelerator market by revenue. AMD competes with its MI300X and MI355X chips, Google builds TPUs for its own cloud, Intel offers Gaudi accelerators, and Amazon and Microsoft build custom chips, Trainium and Maia, for their own platforms.
What chip does ChatGPT run on?
OpenAI's models run primarily on NVIDIA GPUs, including H100 and the newer Blackwell-generation B200 and GB200 systems, hosted through Microsoft Azure's data centers. Microsoft has also begun deploying its own Maia accelerators for some inference workloads, but NVIDIA hardware remains the primary compute behind ChatGPT as of 2026.
Can AI chips be made outside of Taiwan?
Only in limited volume today. TSMC fabricates nearly all advanced AI chips, and while it operates a growing fab in Arizona, the most advanced process nodes still run primarily in Taiwan. Samsung in South Korea is the other major advanced foundry, and Intel is investing heavily to become a third option, but neither currently matches TSMC's capacity or yields at the leading edge.
Related Articles
What Is an AI Accelerator Card? Types, Specs, and Costs for 2026
10 min read
NVIDIA H100 GPU: Full Specs, Price, and Cloud Rates for 2026
11 min read
NVIDIA Blackwell Architecture: What the B200 GPU Can Do
10 min read
Best GPU for AI Training in 2026: RTX 4090, RTX 5070 and Enterprise Picks for LLMs, ComfyUI and Inference
12 min read
Want hands-on setup guides?
These step-by-step guides relate to topics covered in this article.