Claude-SearchBot
Claude-SearchBot is assigned to the AI category by BotScope. Detection is based on the user-agent string. Current pattern:
Claude-SearchBot
AI training crawlers collect web content as training material for large language models. They bring no direct traffic — your content ends up in the model and influences future AI answers. Most honour robots.txt, some are more aggressive than search engine crawlers.
BotScope maps Claude-SearchBot requests onto your session timeline — you spot bursts, can filter them per path, and compare against other bots in the same category.
robots.txt directive for Claude-SearchBot
If you don't want Claude-SearchBot to crawl your site, add the following block to your /robots.txt. This works only for bots that honour robots.txt — malicious crawlers ignore it.
User-agent: ClaudeBot Disallow: /
This page is being enriched with detail knowledge about Claude-SearchBot.