Material Insights · Understanding the Material World

Large language models: Evolution

Timeline

The milestones from first idea to today

YearMilestone
1951Shannon publishes his measure of how predictable English text is [5]
2000Bengio publishes a neural probabilistic language model [6]
2017Google researchers describe the Transformer on 31 August [1]
2020OpenAI opens its API, running GPT-3 family models, on 11 June [2]
2022ChatGPT is launched in November [3]
2024Output as good as GPT-3.5 costs 7 cents per million tokens, down from 20 dollars in 2022 [4]
2025ChatGPT reaches 800 million weekly users, in October [3]

as of 2026-11-11

Lineage

The technologies it came from and the ones that came from it

Builds on
  • DL Deep learning Language models are deep networks trained by backpropagation; Bengio's neural language model of 2000, which introduced word embeddings, was an early one [6].
  • GPU Graphics processing unit Models are trained and run on data-centre chips; Nvidia says reasoning AI drives demand for its Blackwell data-centre GPUs [7].
  • Cld Cloud computing Models are offered as an online service through an API, as OpenAI's was from June 2020 [2].
Works with

as of 2026-11-11

Sources

  1. Transformer: A Novel Neural Network Architecture for Language Understanding Google Research Blog (Jakob Uszkoreit), 2017
  2. OpenAI API OpenAI, 2020
  3. Sam Altman says ChatGPT has hit 800M weekly active users TechCrunch (Rebecca Bellan), 2025
  4. AI Index 2025: State of AI in 10 Charts Stanford Institute for Human-Centered AI (HAI), 2025
  5. Prediction and Entropy of Printed English Nokia Bell Labs, publications (C. E. Shannon, Bell System Technical Journal), 1951
  6. Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award Association for Computing Machinery, news release distributed on EurekAlert!, 2019
  7. NVIDIA Announces Financial Results for Fourth Quarter and Fiscal 2025 NVIDIA Newsroom, 2025
  8. Alphabet Announces Fourth Quarter and Fiscal Year 2024 Results (Exhibit 99.1) Alphabet Inc., filed with the US SEC, 2025