Fact: The upcoming mobile-first index is going to shake rankings up like no other update in history.

While typical algorithm changes target specific strategies or a set of quality factors, the new index will pretty much turn EVERYTHING upside down.

And judging by the recent updates from Google, we won’t have to wait for very much longer for that to happen.

The thing is, the search engine already announced that they’d evaluate sites “independently on their readiness for mobile-first indexing [...] and transition them when ready.”

Moreover, as John Mueller pointed out in a recent Hangout:

“[...] that's probably not something that would just like go from one day to the next but rather be more kind of a gradual thing.”

And this simply suggests that most likely you’ll have to figure out when Google decides to move your site to the new index.

In this post, we will discuss different ways to monitor when Google switches your site to the mobile-first index. And we will also cover a couple of must-do steps that all sites should do ahead of the move to the Mobile-First Index to make sure you’re prepared.

How to Monitor Your Site's Move to the Mobile-First Index

The three strategies we recommend are:

  1. Analyzing bot crawling to spot an increase in the smartphone Googlebot activity.
  2. Assessing search performance and mobile traffic, and
  3. Monitoring changes in mobile rankings

So, let’s go through them in detail.

Analyzing Bot Activity

By far, this is the most surefire way to identify when your site has moved to the new index. At the core of the index change is the shift to use mobile content to crawl, index, and rank web pages. Currently, Google uses the desktop content to assess how to rank a page in search. The new index, however, intents to swap things around.

For one, as we’ve come to understand earlier last year, there will be no separate indexes. Instead, the new index aims to shift the evaluation criteria from desktop on-page factors to mobile.  

(Which pretty much means that Google will crawl your mobile pages, and use them to assess and rank them.)

And so, once moved, your site will see an increase in smartphone Googlebot activity.

(Google Search Console screenshot showing bot activity per day)

How to monitor the Googlebot activity?

You have a couple of options to do that.

One is to review the server log file regularly.

This file includes information about every server hit (or else, a request) for any file on your site. Be it a page, CSS code or an image, a server log file will contain a record of every time the file was requested. But what’s important is that, apart from information such as time and date or the response code of the request, the file also identifies the user agent - the search engine that made the request, behind it.

Here’s an example of an actual server log file with the desktop Googlebot activity highlighted:

Information about the request clearly indicates the request origin:

"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

From this data, I can see that the last time Googlebot visited the site was on the 29th of December.

But the file also contains information about Google’s mobile agent’s visits. See:

And so, by regularly analyzing the server log you could identify an increased smartphone Googlebot activity. But reviewing server logs manually is time-consuming.

So the other option is to use a crawler like Bot Clarity to monitor Googlebot activity from a simple dashboard.

Bot Clarity analyzes your server logs and allows you to dive deep into the data to correlate crawl data, rankings, and traffic.

(BotClarity - dashboard)

Most importantly, it allows you to monitor for specific crawlers, including the Googlebot smartphone.

Assessing Search Performance and Traffic

I admit that organic mobile traffic data can be somewhat misleading when it comes to mobile-first index. For one, the search engine makes other regular changes to mobile search results that can impact your traffic and other metrics.

Nonetheless, every day Google Search Console collects countless data on your site’s performance. This information allows you to analyze the number of clicks, impressions, average click-through rates and more for pages, keywords, and devices.

It helps identify changes in your mobile traffic that could signal that your site is now on the mobile-first index.

(Google Search Console showing mobile traffic for a domain)

In seoClarity, Search Analytics allows for further slice, dice and filtering of the data to also assess how the switch might have affected your rankings. 

Monitoring Mobile Ranking Changes

You know, I think this goes without saying - once a site moves to the mobile-first index, its rankings will change.

And potentially quite significantly at that.

Other factors could affect your pages’ search rankings for relevant keywords. After all, as I mentioned already, Google regularly introduces changes to its ranking algorithm that could affect your pages positions in the search results.

Nonetheless, a sudden shift at a time when no other significant Google updates occurred might indicate the switch.

Rank Clarity allows you to track changes to both mobile and desktop rankings, and spot major shifts and fluctuations.

(Rank Clarity - dashboard shows mobile ranking results)

How to Minimize a Potential Negative Impact of the Move

Finding out that your site has moved to the mobile-first index is crucial. But making sure that you retain rankings and traffic when this happens takes even further priority.

A couple of months ago, I published a comprehensive checklist for getting a site ready for the new index. It highlights everything you should do to optimize your site before Google moves it to the mobile-first index, and I strongly recommend you read it. To reiterate, here are the most important steps to prepare for the mobile-first index.

Basic Checklist to Prepare for Mobile-First Index:

  • Start by verifying that Googlebot can access your mobile site. This is particularly important if your organization uses a separate desktop and mobile sites. Make sure that Google can access, crawl, and index the mobile one.
  • Add more content to the mobile site. Often, organizations shorten mobile content to provide a better user experience. However, since in the new index, Google will assess those pages for rankings, it’s critical that at a minimum, you match their content with their desktop counterparts.
  • Improve mobile content formatting. User experience is a crucial ranking factor. And so, make sure that your mobile pages provide the same level of experience as your desktop content.

For more tips, check out my checklist for getting a site ready for the mobile-first index.

Closing Thoughts

It goes without saying, the upcoming mobile-first index will bring changes we've never seen before. And given the launch strategy Google intents to employ, you want to know when Google decides to move your site to the mobile-first index.

Monitoring the move and the results of that move are critical to ensure your search visibility is maintained. And, when you prepare your site to ensure it's ready for the mobile-first index, the transition should be seamless to you and your users searching for your site.