---
schema_version: "newruntime-reader-dossiers-v1"
type: "reader_dossier"
slug: "crawlcheck"
kind: "seo"
operator: "CrawlCheck (crawlcheck.io)"
researched_at: "2026-10-23"
confidence: "high"
json_url: "https://newruntime.com/readers/crawlcheck.json"
canonical_url: "https://newruntime.com/readers/crawlcheck/"
---

# CrawlCheck

> Free site audit that checks whether AI and search crawlers can read a site. The CrawlCheck/1.0 UA fetches machine files and control paths (robots.txt, sitemaps, llms.txt, API catalog); the homepage is additionally requested once under ~12 other crawler identities (GPTBot, ClaudeBot, Googlebot, etc.).

## Dossier

- Kind: SEO & backlink crawlers
- Operator: CrawlCheck (crawlcheck.io) (https://crawlcheck.io/)
- Documentation: https://crawlcheck.io/policy
- robots.txt tokens: CrawlCheck
- Feeds: monitoring
- Respects robots.txt: Yes, per its documentation. Docs: 'User-agent: CrawlCheck' Disallow: / or 'CrawlCheck: none' opts out on next scan. CrawlCheck/1.0 requests are Web Bot Auth-signed; measurement probes under other crawlers' UAs are unsigned. No owned IP range (egress via Cloudflare).
- Verification: No published method
- Labels matched in our logs: crawlcheck

## Why it matters to New Runtime (our interpretation)

Someone ran an AI-crawlability audit on our domain; it also means some 'GPTBot/ClaudeBot' homepage hits may actually be CrawlCheck probes.

## Observed behaviour

- Baseline 2026-10-23 (93-day window): 12 requests on 4 days; first seen 2026-10-10, last seen 2026-10-22; Returning.
- Live numbers: https://newruntime.com/bot-traffic-history.json and https://newruntime.com/crawler-journeys.json

## Sources

- https://crawlcheck.io/policy
- https://crawlcheck.io/

Identity: A family name is what a client claims in its user agent. It is verified only when its request came from the operator's published IP ranges or passed forward-confirmed reverse DNS, as reported per family in /crawler-journeys.json. Anyone can claim any name.

All dossiers: https://newruntime.com/readers.md