Finding01/ 05
AI crawler access blocked
For an AI assistant to mention you, it has to have been able to read you. It’s the precondition for everything else, and it’s the one most websites fail without knowing: many security plugins and CDNs block these crawlers by default.
What it checks
- AI crawler accessFor each AI crawler in the catalog, evaluates whether the applicable rules allow it to access the URL. No finding if all are allowed; limits if some are and some aren’t; blocks if all the search and retrieval crawlers evaluated are blocked.