# Free AI readiness and SEO checkers

Each checker answers one question about how search engines, AI crawlers and agents see a site. They all run the same free CrawlWise scan.

- [AI crawler checker](https://crawlwise.dev/tools/ai-crawler-checker.md): Free AI crawler checker: test if GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot can reach your site through robots.txt, Cloudflare and your firewall.
- [llms.txt checker](https://crawlwise.dev/tools/llms-txt-checker.md): Free llms.txt checker: see if your file is reachable, is Markdown, follows the llmstxt.org layout, and whether your pages serve Markdown to AI agents.
- [Can AI read your site without JavaScript?](https://crawlwise.dev/tools/ai-javascript-checker.md): Free test of what AI crawlers see: compare your raw HTML with the rendered page to find content that ChatGPT, Claude and Perplexity miss without JavaScript.
- [Structured data checker for AI search](https://crawlwise.dev/tools/structured-data-checker.md): Free JSON-LD checker for AI search: see if your schema.org markup parses, is in the server HTML, describes the page entity and carries valid dates.
- [WebMCP and agent discovery checker](https://crawlwise.dev/tools/mcp-agent-discovery-checker.md): Free checker for agent protocols: WebMCP tools, MCP server cards, A2A agent cards, agent skills, API catalogs and OAuth discovery documents on your site.
- [robots.txt checker](https://crawlwise.dev/tools/robots-txt-checker.md): Free robots.txt checker: see if your file is reachable and valid, which search and AI bots it allows, and whether Googlebot can fetch your CSS and JavaScript.
- [XML sitemap checker](https://crawlwise.dev/tools/sitemap-checker.md): Free XML sitemap checker: see if your sitemap.xml is reachable and valid, is declared in robots.txt, lists the scanned page and carries lastmod dates.
- [Redirect checker](https://crawlwise.dev/tools/redirect-checker.md): Free redirect checker: trace every hop from the URL you enter to the final page, spot temporary redirects and long chains, and check HTTPS and HSTS.
- [Indexability and canonical checker](https://crawlwise.dev/tools/indexability-checker.md): Free indexability checker: find noindex, nofollow and nosnippet in meta tags and X-Robots-Tag headers, check the canonical URL and validate hreflang.
- [Open Graph and meta tag checker](https://crawlwise.dev/tools/open-graph-checker.md): Free Open Graph checker: see if your page has og:title, og:description and an absolute og:image, plus a title, meta description, H1 and lang attribute.
- [Security headers checker](https://crawlwise.dev/tools/security-headers-checker.md): Free security headers checker: test Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, frame protection and HSTS on your page.

---

Last updated 19 November 2026.

HTML version: <https://crawlwise.dev/tools>

Every CrawlWise page in Markdown: <https://crawlwise.dev/llms.txt>

