What is the primary method by which robots (crawlers, spiders) discover new web pages?
- Google and other search engines manually add new sites to an index on a regular basis
- User queries activate robots and cause them to begin searching the web
- Robots follow the path created by websites linking to each other
- Webmasters submit their sites to be indexed