Definition, Meaning & Synonyms
crawler
noun
/ˈkrɔːlər/
Definition
A computer program that automatically browses the web to index content for search engines.
Examples
- The search engine uses a crawler to gather new data from websites.
- Without a crawler, search engines would struggle to keep their results up-to-date.
- Crawlers can also be used for data mining and web analysis.
Meaning
In the context of the internet, a crawler is a tool that scans websites to gather information and update search engine databases.
Synonyms
- spider
- webbot
- bot
- robot
