• A combination of standards and conventions governing web crawler behavior and search engine indexing. The REP includes the robots exclusion standard (robots.txt), crawling and indexing directives for HTML meta elements, the Sitemaps protocol, and the rel=nofollow microformat. ↑ rfc-editor.org/rfc/rfc9309

1. More about this term: Web search, articles and videos, books
2. Previous term: Robot
3. Next term: Robots exclusion standard
4. Random term: TrueDoc