Live robots.txt Checker
Fetches the public file and evaluates the submitted path against documented crawler user-agents.
What this tool checks
Fetches the public file and evaluates the submitted path against documented crawler user-agents.
How it works
The most specific applicable group is selected and the longest matching rule determines each verdict.
Common mistakes
A successful fetch does not prove bot compliance, and a global rule can differ from a crawler-specific group.