Free Robots.txt Generator for AI Bots
Control which AI crawlers can access your website. Toggle bots on or off to generate a robots.txt file.
15 allowed/0 blocked
GPTBotOpenAI
ChatGPT-UserOpenAI
Google-ExtendedGoogle
GooglebotGoogle
BingbotMicrosoft
anthropic-aiAnthropic
ClaudeBotAnthropic
PerplexityBotPerplexity
BytespiderByteDance
CCBotCommon Crawl
FacebookBotMeta
TwitterbotX (Twitter)
ApplebotApple
YouBotYou.com
Cohere-aiCohere
Preview
# robots.txt — Generated by LunimRank # 2026-03-29 # All AI bots are allowed User-agent: * Allow: / # Sitemap Sitemap: https://example.com/sitemap.xml # Generated by LunimRank — https://lunimrank.com/tools/robots-txt-generator
What does robots.txt do for AI bots?
Control Access
A robots.txt file tells AI crawlers like GPTBot and ClaudeBot whether they are allowed to access and learn from your website content.
Protect Content
Block specific AI bots from scraping your content for training data while still allowing search engines to index your pages.
Easy Setup
Upload the generated file to the root of your website (e.g. yoursite.com/robots.txt). Most web hosts support this out of the box.
Want to track your AI visibility?
Controlling crawler access is step one. LunimRank monitors how AI engines actually talk about your business across an 8-engine platform, weekly.