web crawler

or web·crawl·er

[ web-kraw-ler ]
/ ˈwɛbˌkrɔ lər /

noun Digital Technology.

(sometimes initial capital letter) a computer program that retrieves data from a website, as in order to index web pages for a search engine.
Also called spider.
Compare bot1.

Origin of web crawler

First recorded in 1990–95

OTHER WORDS FROM web crawler

web crawling, noun