Baidu Spider
Baidu Spider is assigned to the Search category by BotScope. Detection is based on the user-agent string. Current pattern:
Baiduspider
Search engine crawlers index publicly available content for their results. They typically fetch /robots.txt first and respect crawl-delay and Disallow rules. High crawl frequency usually correlates with good visibility in those search results.
BotScope aggregates every Baidu Spider request in your session: which URLs it prefers, how many 4xx/5xx responses it gets, whether it crawls from one or many IPs, and what time window it's particularly active in.
robots.txt directive for Baidu Spider
If you don't want Baidu Spider to crawl your site, add the following block to your /robots.txt. This works only for bots that honour robots.txt — malicious crawlers ignore it.
User-agent: Baiduspider Disallow: /
This page is being enriched with detail knowledge about Baidu Spider.