Web search engine
Programs crawl and index the web and rank pages for any query.
Computing and AI · AI and data · No. 289
- Digital age
- Readiness 9 of 9
- Platform
WebCrawler, a student project of 1994, indexed about 50,000 pages in its first year; one engine has long handled about 90% of US searches.
Chart: Illustration: Web search engine
Where it comes from
Chart: Map of where it comes from
Early search engines were student and university projects: JumpStation at Stirling in December 1993 and WebCrawler at the University of Washington, live in April 1994. Google began as a Stanford research project and raised its first 100,000 dollars in 1998.
Three things to know
- WebCrawler, which went live on 20 April 1994, was among the first to search the full text of pages, not just their titles.
- Brin and Page at Stanford ranked pages by the web's link graph: a page linked from many important pages ranks higher.
- In August 2024 a US federal court found that Google had acted unlawfully to maintain its search monopoly, and remedies followed in 2025.
The journey
- 1 CrawlRobot programs follow links from page to page and fetch each one
- 2 IndexEvery word is listed with the pages it appears on
- 3 RankPages are scored by their words and by the links pointing to them
- 4 QueryA user types words and gets the best-scoring pages in a fraction of a second
- 5 AdvertisingPaid adverts beside the results pay for the service