How does the robots metadata affect web crawling and indexing?

The robots metadata instructs web crawlers on how to index the page.

This metadata provides directives to search engine robots about whether to index the page, follow links, or cache the content. It helps control the visibility and behavior of the page in search engine results.

<meta name="robots" content="noindex, nofollow">