# Attention and the transformer: Technologies

Source page: https://www.materialinsights.org/science/attention_transformer/sci-technologies/

#### Technologies

The technologies that rely on it, and how each one uses it
 Relied on by

- [LLM Large language models](https://www.materialinsights.org/technologies/large_language_models/) Self-attention models the relationships between all the words in a passage at once, the core of the transformer; its cost grows with the square of the passage length ^[2].
- [DL Deep learning](https://www.materialinsights.org/technologies/deep_neural_networks/) The transformer is a deep network built from stacks of attention layers; the base encoder stacks six identical layers ^[2].
- [Srch Web search engine](https://www.materialinsights.org/technologies/web_search_engine/) Google applies BERT, a transformer model, to ranking search results, and said it would help Search understand one in ten English searches in the US ^[6].

as of 2026-11-12

#### Sources

1. [Attention Is All You Need](https://ar5iv.labs.arxiv.org/html/1706.03762) ar5iv (arXiv Labs), full text of arXiv:1706.03762, 2017
2. [Understanding searches better than ever before](https://blog.google/products/search/search-language-Understanding-bert) Google, The Keyword blog (Pandu Nayak), 2019

## Other sections

- [Summary](https://www.materialinsights.org/science/attention_transformer/index.md)
- [The idea](https://www.materialinsights.org/science/attention_transformer/sci-idea/index.md)
- [Discovery](https://www.materialinsights.org/science/attention_transformer/sci-discovery/index.md)
