Start Here

Find the Right Tools

Platform

Top Apps

Grow with Semrush

Explore Free Tools

About Semrush

Main page /Product News

Log File Analyzer – Open Beta

December 3, 2018
Log File Analyzer – Open Beta
What do ‘sea’ and ‘SEO’ have in common aside from their similar spelling? You’re going to need reliable equipment to dive deep into both. The solution for the first was invented a long time ago, and now it’s time to deal with the second.
Today we’re celebrating the beta launch of our new and very promising product. Please meet our innovative Log File Analyzer – a tool that will allow you to take a look at your site from the perspective of a Googlebot.

What is it for?

  • What errors were found during the crawl?
  • What are your most crawled pages?
  • Do you spend your crawl budget efficiently?
No questions unanswered and stones unturned now! With Log File Analyzer you will clearly understand what happens when a search engine crawls your website.
The tool will allow you to:
  • eliminate structural and navigational issues affecting the accessibility of your site’s pages
  • track the occurrence of technical problems (broken pages, incorrect redirects, etc.)
  • optimize consumption of your crawl budget
  • track and analyze the behavior of different bots on your site separately (mobile, desktop)
  • check if your website has successfully gone through migration
  • understand how Googlebot prioritizes content
  • make sure you spend resources on content production wisely

How does it work?

Find the Log File Analyzer in our left-hand menu in the On Page & Tech SEO block, or just select All Tools. Make sure that your log file is unarchived and in the proper access.log file format. We’ve described the proper file format in this PDF manual. Use the link in the description.
6.png
Then you simply drag and drop your file into the form on the Log File Analyzer page to upload your file to the tool.
After all of your files have been uploaded, click on Start Log File Analyzer and ... here you go: all the useful data, including how a Googlebot’s behavior changed over time, consistency of response statuses for tracking issues, the paths that received the most hits by the bot in the time frame, and much more.
7.png
8.png
You can use the All Google Bots filter to show only desktop or mobile bot activity, and also adjust the time period. The charts on the right will tell you how many of each status code and file type the bots interacted with.
9.png
That's all for today. There’s no need to analyze log files manually, no matter how skilled you are. The formula is simple: take Site Audit, add our new Log File Analyzer and find answers to any questions related to technical SEO!
Enough with the theory, let’s get to the practice! Give our new tool a try and let us know what you think by hitting us up with an email at log-analysis@semrush.com.
Want to learn more about crawlability and indexability of a website? Check out this article.
Become a technical SEO guru. Try out the tool.

Relevant news

Semrush for Enterprise Unifies Content Optimization Capabilities Across SEO and AI Search

New workflows shared between Enterprise SEO and AI Optimization enable teams to grow brand visibility through high-performing content for both search channels.
Product News
May 27, 2026

Semrush Expands AI Visibility Database To 32 Countries, With 17 New Regional Markets

Global coverage expansion now provides international teams with real AI search data for previously unmeasured markets, growing the Semrush prompt database to more than 261 million LLM prompts.
Product News
May 14, 2026

Semrush Announces Partnership With Lovable, Bringing Search Intelligence Into The Building Experience

In a new partnership with Semrush, all Lovable users will have access to Semrush’s search intelligence natively inside the building experience, so founders can create projects for brand visibility and growth from the outset.
Product News
May 13, 2026