Crawl
A bot has to reach the URL. It fails when robots.txt blocks a directory nobody remembers adding, when a page is orphaned with no internal link pointing at it, when a faceted filter generates ten thousand near-duplicate URLs that eat the crawl budget, or when a firewall or bot-management rule quietly returns 403 to Bingbot, GPTBot, or ClaudeBot while letting Googlebot through.
What we check: robots.txt and meta directives, server logs by user agent, orphan pages, redirect chains, crawl traps, and per-bot response codes for the AI fetchers specifically.