Free tool
robots.txt checker and generator
Find out whether crawlers are being blocked by accident, whether AI bots are welcome, and whether your sitemap is declared — then generate a corrected file.
Check robots.txt
We fetch https://yourdomain.com/robots.txt and grade it against common crawl rules.
Want the whole site checked, not just one file?
SEO Scanner runs 29 evidence-backed checks across your pages, with screenshots, Core Web Vitals and a fix plan you can hand to a developer.
Free tool disclaimer
These tools are free and provided as-is. Automated results and generated files should be reviewed by an SEO or development professional before you publish them to a live site. SEO Scanner accepts no liability for changes made from this output.