AI accelerator chip: Working
What it is
The product and the job it does, in a short paragraph
An AI accelerator chip is a processor built to do the multiply-and-add arithmetic of neural networks at very high speed. The functional unit is one AI accelerator package: the logic chip or chips together with their stacked memory on one carrier [20]. The scope is HS 8542.31 and 8473.30. It covers data-centre GPUs and custom accelerators such as Google's TPU. Neural-processing blocks inside phone and PC processors, whole servers and general-purpose processors are left to their own pages.
as of 2026-11-16
How it works
How it does its job, step by step in plain words
The chip holds thousands of small arithmetic units that multiply and add numbers in parallel, at low precision such as 8-bit floating point (FP8) [13]. Each layer of a neural network is a large block of such sums, so the arithmetic units must be fed fast. For that, stacks of DRAM chips called high-bandwidth memory (HBM) sit beside the logic die on a thin silicon slab, the interposer, which links them over many short wires [20]. The interposer then sits on a substrate that carries power and signals to the board [20]. Nearly all the electricity used ends as heat, removed by a copper heat sink fitted to the package, which is 98% copper in the A100 module [17], or by liquid cooling [40].
Indicative · as of 2026-11-16
Key specifications
The numbers that describe it, each with unit, variant, year and source
| Spec | Value | Unit | Variant | Year | Source |
|---|---|---|---|---|---|
| Maximum thermal design power, H100 SXM (power the cooling must remove) | 700 | W | Data-centre GPU | 2026 | [4] |
| Memory capacity, H100 SXM (HBM3) | 80 | GB | Data-centre GPU | 2026 | [4] |
| Memory bandwidth, H100 SXM | 3.35 | TB/s | Data-centre GPU | 2026 | [4] |
| Die area of the H100 logic chip (TSMC 4N) | 814 | mm2 | Data-centre GPU | 2022 | [13] |
| Transistors on the H100 logic chip | 80 | billion | Data-centre GPU | 2022 | [13] |
| Transistors on a Blackwell GPU (two dies, TSMC 4NP) | 208 | billion | Data-centre GPU | 2024 | [14] |
| Thermal design power of the Tesla V100, for comparison with the H100 | 300 | W | Data-centre GPU | 2017 | [28] |
| HBM3e capacity per Google Ironwood (TPU v7) chip | 192 | GB | Custom accelerator | 2025 | [39] |
as of 2026-11-16. The H100 figures are the maker's current datasheet [4]; the die area and transistor count come from a report on NVIDIA's Hopper launch [13]. Power is the design limit, not measured use.
Variants
The variants whose make-up or footprint differs a lot, what defines each and its share of units
| Variant | Defining spec | Year | Source |
|---|---|---|---|
| Data-centre GPU | Logic dies and five to eight HBM stacks on an interposer: the H100 SXM5 has one die with five active HBM3 stacks and draws up to 700 W, Blackwell joins two dies, and AMD's MI300X has eight stacked GPU chiplets with eight HBM3 stacks at 750 W | 2025 | [13][4][14][51] |
| Custom accelerator | A chip a cloud firm designs for its own models and has a foundry make, such as Google's TPU; Ironwood has 192 GB of HBM3e and liquid cooling | 2025 | [39][40] |
as of 2026-11-16. No source gives unit shares by variant. NVIDIA had about 70% of the AI chip market in 2025 by TrendForce's estimate, and TrendForce expects ASIC shipments to pass GPU shipments in 2026 [7]. Neural processors built into phone and PC chips are not a variant here: they have no memory stack and sit on the processor chip page.
Limits
The physical limits set by science and the engineering limits of today's design, with today's value
| Limit | Kind | Limit | Today | Unit | Source |
|---|---|---|---|---|---|
| Largest single chip a lithography scanner can expose (one reticle); the H100 die is already close, so Blackwell joins two dies | Engineering | 858 | 814 | mm2 per die | [43][13][14] |
| Interposer size for packaging many dies and HBM stacks (TSMC roadmap for 2029, not a hard limit) | Engineering | 14 | two reticle-limit dies in one Blackwell GPU | reticle sizes | [43][14] |
| Power one package can draw while its heat is removed (set by cooling; no hard limit found) | Engineering | – | 700 | W per package (H100 SXM); 300 W for the V100 of 2017 | [4][28] |
as of 2026-11-16. Lithography resolution limits are common to all chips and are given on the processor chip page.
Science it relies on
The science principles it works by or is limited by
Relies on- Artificial neural networks The work is the layer-by-layer multiply-and-add of artificial neural networks, which the chip's arithmetic units are built for.
- Diffraction limit of resolution The finest circuit lines are limited by the diffraction of the light that prints them, which is why EUV light is used.
- Heat conduction (Fourier's law) Nearly all the electricity a chip uses ends as heat, which must conduct out through the package to a heat sink or liquid.
as of 2026-11-16
Sources
- NVIDIA H100 Tensor Core GPU: specifications NVIDIA, 2026
- Nvidia Hopper GPU architecture revealed. 4nm die & 18432 shaders HWCooling.net, 2022
- NVIDIA Blackwell Platform Arrives to Power a New Era of Computing NVIDIA investor relations, 2024
- More than Carbon: Cradle-to-Grave environmental impacts of GenAI training on the Nvidia A100 GPU (arXiv:2509.00093v3) arXiv preprint, 2025
- AI Expansion - Supply Chain Analysis For CoWoS And HBM SemiAnalysis, 2023
- NVIDIA Announces Tesla V100 with Volta GPU at GTC 2017 PC Perspective, 2017
- Google's Ironwood TPUs represent a bigger threat than Nvidia would have you believe The Register, 2025
- Google offers 7th-gen Ironwood TPUs for AI, with AI-inspired comparisons The Register, 2025
- TSMC's next-gen CoWoS roadmap: over 14-reticle packages and a 48x leap in compute power expected by 2029 Tom's Hardware, 2026
- AMD unveils Instinct MI300X GPU and MI300A APU, claims up to 1.6X lead over Nvidia's competing GPUs Tom's Hardware, 2023