HTTP to HTTPS: An SEO’s guide to securing a website

Back when I wrote the article, “Why Everyone Should Be Moving To HTTP/2,” it was meant to bring awareness to an awesome protocol upgrade that I thought was an easy win to make a website faster.

Since then, I have spoken to hundreds of business owners and SEOs about upgrading, performed dozens of upgrades and troubleshot dozens more. I have realized that there is still one big hurdle for both business owners and SEOs: HTTPS. The gotcha moment with HTTP/2 is that most browsers only support this new protocol over a secure connection, which means you have to migrate your website to HTTPS.

It shouldn’t come as a shock to anyone that Google and many others want the web to be more secure. Google had their HTTPS everywhere campaign, they announced HTTPS as a ranking signal, and they have started indexing secure pages over unsecured pages. They even have their own guide, “Securing Your Website With HTTPS,” which I encourage everyone to read, along with this article.

Yet with all of this push towards a more secure web, the fact remains: Less than 0.1% of websites are secure.

It seems like everyone is trying to make it as easy as possible to switch by removing barriers to entry, such as cost. Let’s Encrypt offers free certificates (Sidenote: I am very amused that Google Chrome has the only nofollow on their paid sponsorship link after being called out.) Many website hosts and CDNs are also offering free security certificates to encourage people to make the switch, but many people still aren’t moving.

Why move to HTTPS?

Google identifies several reasons to switch to HTTPS in their website migration guide:

Data sent using HTTPS is secured via Transport Layer Security protocol (TLS), which provides three key layers of protection:

1. Encryption. Encrypting the exchanged data to keep it secure from eavesdroppers. That means that while the user is browsing a website, nobody can “listen” to their conversations, track their activities across multiple pages or steal their information.
2. Data integrity. Data cannot be modified or corrupted during transfer, intentionally or otherwise, without being detected.
3. Authentication. Proves that your users communicate with the intended website. It protects against man-in-the-middle attacks and builds user trust, which translates into other business benefits.

There are other benefits, though, including the Google ranking boost previously mentioned.

Making the switch to HTTPS also helps with the loss of referral data that happens when the referral value in the header is dropped when switching from a secure website to an unsecured website. Analytics programs attribute traffic without the referral value as direct, which accounts for a large portion of what is called “dark traffic.”

The switch also prevents a lot of bad things, such as when AT&T was injecting ads into their hotspots. They would not have been able to inject these ads on a website with HTTPS.

Does HTTPS secure my website?

People hear HTTPS referred to as a secure protocol, and they think this protects their website. The fact is that your website is not protected, and you may still be vulnerable to one or more of the following:

• Downgrade attacks
• SSL/TLS vulnerabilites
• Heatbleed, Poodle, Logjam, etc.
• Hacks of a website, server or network
• Software vulnerabilities
• Brute force attacks
• DDOS attacks

Making the switch from HTTP to HTTPS

1. Start with a test server. This is important because it lets you get everything right and test without screwing it up in real time. Even if you are doing the switch without a test server, there’s almost nothing you can do that you can’t recover from, but it’s still best practice to have a plan and have everything tested ahead of time.
2. Crawl the current website so that you know the current state of the website and for comparison purposes.
3. Read any documentation regarding your server or CDN for HTTPS. I run into lots of fun CDN issues, but it can also be straightforward.
4. Get a security certificate and install on the server. This will vary depending on your hosting environment and server setup too much for me to go into details, but the process is usually well-documented.
5. Update references in content. This can usually be done with a search-and-replace in the database. You’ll want to update all references to internal links to use HTTPS or relative paths.
6. Update references in templates. Again, depending on how you deploy, this might be done with Git or simply Notepad++, but you’ll want to make sure references to scripts, images, links and so on are either using HTTPS or relative paths.
7. Update canonical tags. Most CMS systems will take care of this for you when you make the switch, but double-check, because that’s not always the case.
8. Update hreflang tags if your website uses them, or any other tags such as OG tags for that matter. Again, most CMS systems will take care of this, but it’s best to QA it just in case.
9. Update any plugins/modules/add-ons to make sure nothing breaks and that nothing contains insecure content. I commonly see internal site search and forms missed.
10. CMS-specific settings may need to be changed. For major CMS systems, these are usually well-documented in migration guides.
11. Crawl the site to make sure you didn’t miss any links and nothing is broken. You can export any insecure content in one of the Screaming Frog reports if this is the crawler you are using.
12. Make sure any external scripts that are called support HTTPS.
13. Force HTTPS with redirects. This will depend on your server and configuration but is well-documented for Apache, Nginx and IIS.
14. Update old redirects currently in place (and while you’re at it, take back your lost links from redirects that haven’t been done over the years). I mentioned during the Q&A portion of the Technical SEO Panel at SMX West that I’ve never had a site drop in rankings or traffic when switching to HTTPS, and a lot of people questioned me on this. Due diligence on redirects and redirect chains is likely the difference, as this is what I see messed up the most when troubleshooting migrations.
15. Crawl the old URLs for any broken redirects or any redirect chains, which you can find in a report with Screaming Frog.
16. Update sitemaps to use HTTPS versions of the URLs.
17. Update your robots.txt file to include your new sitemap.
18. Enable HSTS. This tells the browser to always use HTTPS, which eliminates a server-side check and makes your website load faster. This can also cause confusion at times, since the redirect will show as 307. It could have a 301 or a 302 behind it, though, and you may need to clear your browser cache to see which.
19. Enable OCSP stapling. This enables a server to check if a security certificate is revoked instead of a browser, which keeps the browser from having to download or cross-reference with the issuing certificate authority.
20. Add HTTP/2 support.
21. Add the HTTPS version of your site to all the search engine versions of webmaster tools that you use and load the new sitemap with HTTPS to them. This is important, as I’ve seen traffic drops misdiagnosed because they saw the traffic in the HTTP profile drop, when the traffic in reality moved to the HTTPS profile. Another note for this is that you do not need to use the Change of Address Tool when switching from HTTP to HTTPS.
22. Update your disavow file if you had one for the HTTPS version.
23. Update your URL parameter settings if you had these configured.
24. Go live!
25. In your analytics platform, make sure you update the default URL if one is required to ensure that you are tracking HTTPS properly, and add notes about the change so that you know when it occurred for future reference.
26. Update your social share counts. There’s a lot of gotchas to this, in that some of the networks will transfer the counts through their APIs, while others will not. There are already guides for this around if you are interested in keeping your share counts.
27. Update any paid media, email or marketing automation campaigns to use the HTTPS versions of the URLs.
28. Update any other tools such as A/B testing software, heatmaps and keyword tracking to use the HTTPS versions of the URLs.
29. Monitor everything during the migration and check, double-check and triple-check to make sure everything is going smoothly. There are so many places where things can go wrong, and it seems like there are usually several issues that come up in any switch to HTTPS.

One question I’m often asked is if incoming links should be cleaned up. This is a huge amount of outreach and effort. If you have time, then sure; but most likely you’re busy with other things, and I don’t feel it’s absolutely necessary. However, you should update the links on any properties that you control, such as social profiles.

Common problems with HTTPS migrations

Things that can go wrong include:

• preventing Google from crawling the HTTP version of the site, or preventing site crawls in general (usually happens because of failure to update the test server to allow bots);
• content duplication issues, with both HTTPS and HTTP versions of the pages showing; and
• different versions of the page showing on HTTP and HTTPS.

Most of the common problems with HTTPS migrations are the result of improperly implemented redirects. (I’ve also had fun times cleaning up websites that changed their entire structure/design while making the switch to HTTPS.)

Redirects deserve their own section

As stated above, the main problems I see with the migration to HTTPS have to do with redirects. It doesn’t help that the change can be done at the registrar level, in the server config, or even in a .htaccess file; all have their own “gotchas.”

Failed redirects and redirect chains are almost always issues. Be sure to check subpages, as well as the home page; depending on how the rules are written and where they are placed, these can be affected differently. You also need to actually look at what’s going on with these as far as the status codes and hops, not just whether they get you to the correct page.

It definitely doesn’t help when Apache’s documentation for this doesn’t include a 301 and Apache defaults to a 302. The code below should be updated to R=301.

RewriteEngine On
# This will enable the Rewrite capabilities

RewriteCond %{HTTPS} !=on
# This checks to make sure the connection is not already HTTPS

RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
# This rule will redirect users from their original location, to the same location but using HTTPS.
# i.e.  https://www.example.com/foo/ to https://www.example.com/foo/
# The leading slash is made optional so that this will work either in httpd.conf
# or .htaccess context

I’ve seen sites recover from this mistake when switching, but it seems to only happen several months later, when Google figures out what happened and corrects the mistake on their end.

Even the best of us fail at times:

Trust but verify. I use tools like Screaming Frog and Ayima Redirect Path to make quick checks on some of the old URLs — or, with some Excel manipulation, to do bulk checks on massive amounts of URLs and older redirects. This helps to ensure that everything is redirecting properly and without multiple hops.

(See the “Checking Our Work” section in “Take Back You Lost Links” for help in recreating URLs to crawl.)

Closing thoughts on HTTPS

Simply put, HTTPS is not going away. HTTP/2, Google AMP and Google’s QUIC protocol (which is likely to be standardized soon) all require secure connections for browsers to use them. The fact remains that HTTPS is being pushed hard by the powers that be, and it’s time to make the switch.

Most of the problems that I see are from poor planning, poor implementation or poor tracking. If you follow the steps I outlined, you should have little to no trouble when migrating from HTTP to HTTPS.

My favorite comment on the subject is from Gary Illyes, a Google Webmaster Trends Analyst:

If you're an SEO and you're recommending against going HTTPS, you're wrong and you should feel bad.

— Gary 鯨理／경리 Illyes (so official, trust me) (@methode) August 18, 2015