Crawlability & AI Crawler Access Auditor

AI crawler access audit and robots.txt checker for any domain: rules per crawler (GPTBot, ClaudeBot, CCBot, Google-Extended and 25 more), Content-Signal and Content-Usage, sitemaps with URL counts, llms.txt, meta robots on sample pages, TDM opt-out detection and the anti-bot service in front.

Publisher description, verbatim

Last 7 dayson this actor

2other changes

Each one reaches followers by email the morning after.Follow this actor

Key figures8 readings

Price / 1k$20.00per 1,000 domain-audited

Users / 30d1↑ 1 over 8 readings

Users, all time2↑ 1 over 8 readings

Runs / 30d6↑ 5 over 7 readings

Runs / user / mo6.00runs per monthly user

Recorded changes

2026-09-25schemaInput schema changed—on record
2026-09-25contentREADME updated+4,275 characters+4,275characters
2026-09-18changeListed—on record

showing 3 of 3 in the last 30 days · followers get the next one by email

History

8 readings since 2026-09-18

Users / 30d8 readings

Users, all time8 readings

Runs / 30d7 readings

Follow this actor, compare it against its rivals, and get every overnight change in one morning report.

Publish it yourself? Claim your publisher page and its full history is yours free, forever.

Open in the app Claim this page

Numbers on this page are measured only by an independent census of the Apify Store — not self-reported. Not indexed: too little has happened to this actor for us to have anything worth saying yet.