Which user-agent strings identify Semrushbot?

The following user-agent strings identify Semrushbot: SemrushBot.

Does Semrushbot respect robots.txt rules?

Yes, Semrushbot respects robots.txt directives according to official documentation.

How do I block or allow Semrushbot in robots.txt?

You can control Semrushbot access using robots.txt rules. See the blocking snippet on this page for specific directives.

How can I verify Semrushbot traffic in server logs?

You can verify Semrushbot requests by checking your server access logs for the documented user-agent strings. For accurate verification, correlate user-agent patterns with IP ranges or verification methods provided by Semrush.

What is Semrushbot?

Direct Answer: SemrushBot is the search bot software that Semrush sends out to discover and collect new and updated web data.

Operator: Semrush Purpose: SEO Analysis

User-Agent Identification

The following user-agent strings identify Semrushbot in your server logs:

SemrushBot

robots.txt Rules for Semrushbot

Respects robots.txt: Yes

Use the following robots.txt rules to control Semrushbot access:

User-agent: SemrushBot
Disallow: /

Crawl Behavior

Request Pattern: Not Documented

Log Verification

To verify Semrushbot traffic in your server logs:

Search access logs for the user-agent strings listed above
Check if the IP addresses match documented ranges (if provided by Semrush)
Verify the crawl pattern matches documented behavior
Use reverse DNS lookup for additional verification if available

Note: Observed behavior in production environments may differ from official documentation. Log-based monitoring provides the only reliable verification of actual bot behavior.

Missing Information

The following information is not officially documented for Semrushbot:

requestBehavior

Official Documentation

View Official Semrushbot Documentation →

Information extracted on February 2, 2026 from official sources. Content generated with AI assistance.