Noun: web crawler
- A computer program that prowls the internet looking for publicly accessible resources that can be added to a database; the database can then be searched with a search engine
"The web crawler indexed millions of web pages for the search engine";
- spider, crawler
Derived forms: web crawlers
Type of: computer program, computer programme [Brit], program, programme [Brit, Cdn]
Encyclopedia: Web crawler