Mobile Site Configuration & The Vary HTTP Header

Yesterday, Google announced more changes to the Google mobile search algorithm, which we can expect to roll out shortly. This update is intended to improve the search experience for mobile users by “address[ing] sites that are misconfigured for smartphone users” — presumably by improving mobile rankings for sites that have been optimized according to their Mobile Optimization Guidelines.

In an attempt to prevent some confusion, Google also updated these guidelines to include a list of common mistakes, along with ways to avoid said mistakes. Unfortunately, confusion has been the name of the game with mobile SEO for some time now, as Google has historically sent mixed signals about what it considers mobile best practices.

In November 2011, when Google launched the GoMo project to help businesses make “m.” versions of their desktop sites, it seemed like a tacit endorsement for the development of mobile-specific pages from Google, and that was great. Unfortunately, just over 6 months later, Google announced a preference for single-URL, responsive design websites to address mobile traffic and said that “m.” pages are more difficult for Google to index correctly.

mobile seo preference

The inconsistency in Google’s messaging about mobile SEO leaves large, enterprise-level strategists wondering if it is truly possible to future-proof their mobile search strategy. Will following the Mobile SEO Guideline de Jour turn out to be a colossal waste of time and money? Will Google change its stance again once a new strategy has been painstakingly implemented across a large site? What’s an SEO to do?

The Problem With Responsive Design

Last year, many companies took Google’s new preference for responsive design mobile solutions to heart, and started readying their development teams to update from mobile-specific pages to a responsive design approach.

Unfortunately, as most developers know, the problem with responsive design is always load time — and since load time is a ranking factor, this could have negative SEO consequences. In fact, Google’s Matt Cutts announced at SMX this week that a site speed penalty was in the works for mobile.

You might not know it, but load time must be considered a bit differently on mobile than it is on desktops. It is slightly less impacted by the total file size of the page and its component parts, and slightly more impacted by the number of round-trip server requests that have to be made to retrieve all the content for the page.

According to the Google Page Speed Team, we can roughly figure that each average round trip to the server takes between 200 and 300 milliseconds (for a 3G or 4G connection).

With that in mind, you should all be looking at the number of round-trip requests for the pages you are serving to a mobile visitor. Every 3-5 external elements on a page could represent 1 second of load time on a mobile device. Unfortunately, some of the enterprise-level sites that I have looked at for responsive design have more than 50 external resources, each of which represents a separate round-trip request.

Using Selective Serving For Mobile SEO

Google’s answer to the load time problems associated with responsive design on a mobile device is what they call “selective serving” of HTML. Selective serving enables you to serve different content based on the user-agent, either redirecting users to another URL (such as or dynamically serving different HTML elements within the same URL.

selective serving

If you change elements within a page that is meant to adapt to fit multiple devices, this is generally referred to as RESS, which stands for Responsive Design + Server Side Components. (It is a horrible acronym — I know!)

If you want to address mobile visitors with a responsive design site, but have concerns about the file size of your responsive design pages, then your best option is RESS because it allows you to send smaller file size versions of content to mobile devices. Unfortunately, unless the base HTML changes, using RESS will not decrease the latency caused by numerous round trip requests.

Google is not clear how much of the HTML can change between the different versions of the page that are served on one URL; in fact, they are not clear on how much they are comparing the HTML at all.

My guess is that if the visible content is too different, you will have a problem; but, it is hard to know. This can be very stressful for companies that choose to build a variety of different landing pages to address a variety of different user-agents and serve them all from the same URL.

Google stipulates that if you are using this technique, you should be sure to actively let them know that you are changing HTML, based on the user-agent that is accessing the page. You do this by updating the Vary header that your server sends in the HTTP request to include “user-agent.” This indicates that the page content ”will vary depending on the user agent that requests [it].” According to Google, it should look something like this:

google varies http

Where Enterprise SEOs Have Trouble With Selective Serving For Mobile

The big issue here is that most enterprise-level sites are already using a CDN (Content Delivery Network) to speed up content delivery — but most/all CDNs have huge problems with changes to the “varies” attribute.

CDNs (including Akamai and others) take anything in this field as a signal that the content cannot be cached or served from the CDN and must be fetched directly from your server — thus rendering your CDN completely useless. So, how can SEOs for sites that rely on a CDN follow Google Guidelines and still use their CDN?

cdn issues

The Vary HTTP Header Workarounds

We all know the value that CDNs provide for large sites, and that benefit generally carries over to mobile content, too. According to Google, confusion over the Vary HTTP header is a problem with the CDNs and not their guidelines. If you are an enterprise-level SEO, you can’t just turn off the CDN, so how do you get around it? Here are some workaround options:

  1. Maintain your “m.” pages and set up bi-directional annotation. Link the desktop pages to their mobile counterparts with rel=alternate and rel=canonical tags (Google explains this process here). Mobile-specific pages do not need any changes to the Vary HTTP header to comply with Google Guidelines. While Google says they prefer indexing responsive design pages, they have created this method for sharing value between a desktop page and its mobile counterpart. I think Google’s preference is more philosophical than algorithmic and that “m.” pages will be able to compete against responsive design pages of the same SEO caliber.
  2. Call your CDN representative and see if they have a work-around. Someone from Akamai has written in a Google Group that there is a workaround. The workarounds have not been enough for my enterprise clients, but let me know if you have better luck.
  3. Skip the Vary header altogether. If you don’t include the “user-agent” attribute in the Vary header, it is just an opportunity cost — you are simply not sending all possible mobile ranking signals to get your mobile-ready content recognized by the mobile bot faster. According to the people that I have talked to at Google, it does not create a risk that you will be condemned for cloaking or anything like that. (Still — always approach with caution.)
  4. Serve variable content without your CDN. Vary headers are controlled by the server, and are sent with each element of the page that is requested. While it might be more complicated, you can also consider changing the Vary attribute of external page resources (like external images, JavaScripts, CSS, etc.) to include the “user-agent” if, indeed, that element does change across different devices. To be clear, the net impact of this change could still slow the page load, since those elements are coming directly from your server, rather than the CDN. It is only a good idea if there are a very low number of elements that must come directly from your server (<4) — and you should still test to see the impact before site-wide deployment.

The Follow-Up

This is obviously a difficult situation, and both Google and the CDNs have some valid interests. Ultimately, if a compromise cannot be reached, then webmasters are going to need a more useful and efficient way of letting Google know that a page is selectively serving.

I have suggested that there be an on-page signal, like a meta tag. There could also be a “mobile URL identification” function added to Webmaster Tools, or any number of other creative solutions. This is a known issue; but so far, Google has no useful response aside from either, “It is not our problem,” or, “Then don’t follow that part of the Guidelines.”

At this point, I think most SEOs that are working with responsive design mobile content are desperate for Google to really do something good here – we are tired of the confusion and inconsistencies, and we are just begging for meaningful rules and guidelines that we can actually follow without compromising load-time or the functionality of our CDNs.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: All Things SEO Column | Channel: SEO | Google: Mobile | Google: SEO | SEO: Mobile Search


About The Author: Cindy Krum is the CEO and Founder of MobileMoxie, LLC, a mobile marketing consultancy and host of the most cutting-edge online mobile marketing toolset available today. Cindy is the author of Mobile Marketing: Finding Your Customers No Matter Where They Are, published by Que Publishing.

Connect with the author via: Email | Twitter | Google+


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Brant

    Excellent post Cindy. I was unaware of those canonical tags for mobile and desktop.

  • Colin Guidi

    Holy moly, great post, the mobile best practices continue to evolve and become more confusing.

    Here’s what I had understood before your article as options for mobile:
    Responsive Web Design / RESS
    Dynamic Serving
    and now after your post we also have Selective Serving?

    Oh what to do, what to do. I know, I’ll just wait for Google’s 6th idea on mobile best practices!

  • cboulanger

    Great post. Vary headers & RESS (horrible acronym) are both pains to explain and manage with enterprise teams. Google keeps making mobile adoption harder with its unclear explanations.

    Thanks for trying to make sense of this.

  • Alan Perkins

    Good post except for this:

    > Workaround 3: Skip the Vary header altogether

    This is not a workaround – don’t do this! Otherwise you will probably find that only one version of your content – be it the desktop version or the mobile version – is served to all platforms from the CDN. And if your content isn’t responsive but really does vary from device to device, this isn’t what you want. To be fair to Google, this is why they recommend you use the Vary header.

    Allow me to suggest an alternative workaround: create a responsive site that is fast on mobile connections (and therefore obviously on desktop connections too). Then use AJAX to provide a richer desktop experience on top of each responsive page load, where necessary.

  • Cindy Krum

    LOL – Ya! It is tough to know what to do!

  • Cindy Krum

    Thanks! I have asked some Google industry representatives out-right for more clear instructions – I was hoping for something better than the most recent update!

  • Cindy Krum

    Hi Alan- The recommendation was to use mobile-specific pages, which don’t need the varies header. Most CDN’s already have systems in-place to handle mobile redirects from a desktop page to a mobile page on an ‘m.’.

  • Alan Perkins

    > The recommendation was to use mobile-specific pages, which don’t need the varies header.

    Thanks for clarifying, Cindy.

  • Colin Guidi

    Indeed, but if you utilize Ajax for a richer desktop you’ll still need to implement a crawlable solution for that. Whether its progressive enhancement/graceful degradation, utilizing a headless browser, etc, it still asynchronously will call from an external JavaScript library. You’ll need to find a crawlable solution for that content being called in via JavaScript.

  • Alan Perkins

    Possibly. It depends what the extra content is. For example, if it was video or images then sitemaps might be the way to go.

  • fallacycomp

    Google needs to refer to established content negotiation terminology on W3C before creating more junk acronyms and confusing people further.

  • Josh Bledsoe

    Hmm…. It looks like the tag could be the answer to getting responsive design right.

  • Colin Guidi

    That’s the golden ticket. Just what exactly is the Ajax calling in? That’s what the site owner will need to figure out. If it’s content that should be crawled, it’ll need a solution :)

  • Colin Guidi

    Need to move to HTML5 before using it though?

  • George

    This was a great post shared by our SEO inhouse expert. Very nice! Thank you for this.

  • Josh Bledsoe

    Just waiting on the browsers to come to the party. Chrome Version 27.0.1453.110 m is currently accepting the template tag, but it’s still no bueno for IE and Firefox. It would be tempting to maybe build up a responsive page using this since there is one browser with which to test it with though…

  • Colin Guidi

    Agreed, could be cool. Especially seeing how Chrome is making a push as a top browser globally, just need everyone to hop on the 27.0.1453.110 m bandwagon.

  • MuSTa1ne

    I was reading the google documentation and the Vary header is recommended when you are doing a redirection using the User-Agent but with responsive design you are using the same URI.

    Google has the GoogleBot Mobile, so it now which version is Desktop and which is Mobile and about the content should be different because are for different purposes and Google don’t say that should be similar or different just are different because are for different versions.

    I understood in the wrong way?

  • Gareth McConnell

    Outstanding article. Thank you for sharing the knowledge

  • Cindy Krum

    Hey Gareth – thanks! Glad you liked it! :)

  • Cindy Krum

    Some pages serve different content to different devices within the Responsive Design pages. If that is the case, then you still should have the ‘vary’ header in place. If the CDN works with it in place, I think any page that has a mobile redirect or can be viewed on mobile should be sending the ‘vary: user-agent’ header as a recognizable mobile signal for Google.

    Google says: “If your site serves content or redirects users depending on the user-agent, i.e. the response varies, we strongly recommend that your server also send the Vary HTTP header on URLs that serve automatic redirects. This helps with ISP caching and is another signal for Googlebot and our algorithms to discover and understand your website’s configuration.”

  • Mark @ Make Them Click

    Ugh My Brain hurts (lol). Great post but unfortunately raises more questions than answers. Hope we get some definitive guidelines soon.

    We demand rigidly defined areas of doubt and uncertainty.


Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide