Semrush

Semrush is assigned to the SEO category by BotScope. Detection is based on the user-agent string. Current pattern:

SemrushBot

SEO tool crawlers build internal databases for tools like Ahrefs, Semrush, Sistrix or Majestic. They have no direct SEO impact — only the tool accesses the data. Crawl frequency grows with your domain authority.

Once Semrush shows up in your logs, BotScope lets you analyze its crawl paths, status codes, IP distribution and temporal activity — by hour, day or week.

robots.txt directive for Semrush

If you don't want Semrush to crawl your site, add the following block to your /robots.txt. This works only for bots that honour robots.txt — malicious crawlers ignore it.

User-agent: SemrushBot
User-agent: SiteAuditBot
User-agent: SemrushBot-BA
User-agent: SemrushBot-SI
User-agent: SemrushBot-SA
User-agent: SemrushBot-SD
User-agent: SemrushBot-IO
User-agent: SemrushBot-News
User-agent: SemrushBot-BM
User-agent: SemrushBot-BotX
Disallow: /

This page is being enriched with detail knowledge about Semrush.

Similar crawlers in the same category

Discover more crawlers