AI accelerator chip
A chip built to run the arithmetic of neural networks, mainly in data centres.
Electronics and chips · No. 43
- Networked and AI age
Run at its 700 W limit for 80% of the year in a data centre with a PUE of 1.4, one H100 chip draws about 6,900 kWh a year (calculated), and use causes most of an AI chip's climate impact.
Chart: Illustration: AI accelerator chip
Where it comes from
Chart: Map of where it comes from
Intel announced the ETANN neural-network chip in 1989, but no sale of it is recorded, and Google ran its own TPU in its data centres from 2015 without selling it. NVIDIA's Tesla P100, with its memory stacks in the same package, went to cloud firms in 2016, and the V100 of 2017 added 640 tensor cores.
Three things to know
- NVIDIA's H100 has 80 billion transistors on an 814 mm2 die and draws up to 700 W; its Blackwell successor holds 208 billion transistors on two dies.
- In NVIDIA's HGX H100 board, memory makes 546 kg CO2e of its 1,312 kg CO2e footprint, 42%, against 25% for all the integrated circuits including the GPUs (values read from a chart).
- Taken apart, one A100 module held 1,374 g of copper, 1,300 g of it in the heat sink; the GPU chip with its memory held 3.7 g of silicon and about 0.001 g of gold.
The journey
- 1 Logic dieA foundry such as TSMC prints the processor on a silicon wafer; the H100 die measures 814 mm2
- 2 Memory stacksSK hynix, Samsung or Micron stack DRAM chips into high-bandwidth memory (HBM)
- 3 PackagingCoWoS mounts the logic die and HBM side by side on a silicon interposer, then on a substrate
- 4 Baseboard and serverEight packages sit on a baseboard inside a server built by contract assemblers
- 5 UseA data centre runs the chip for years at hundreds of watts
- 6 End of lifeNo recovery rate is published; under a quarter of world e-waste is documented as recycled
Try this in What if Switch on levers and see what changes in India by 2036.