Making proper 301 permanent redirects

The Good, The Bad and The Ugly: 3 aspects of 301 (permanent) redirects

Problem

A client – one of the leading Africa’s online travel agencies – has reported a significant drop in traffic to a new site after putting 301 (permanent) redirects from 23 'old' domains.

A typical redirect scheme involved putting 301 redirects to a Main page, individual Hotel listings pages as well as Maps. What was left on the old domains is Whereabouts sections (roughly 20% of all site’s original content).

As a result of improper 301 redorect application, each of 23 'old' domains has lost its rankings against hotel listings keywords, i.e. 2/3 of its traffic. Sites’ rankings against whereabouts keywords also dipped.

THE LARGEST OLD OLD DOMAIN TRAFFIC DROP FRAPH

THE NEW DOMAIN TRAFFIC CHANGE GRAPH

Ok. What was the cure?

Approach

I have addresses the following points that arguable were critical to the site's performance.

1. The sites have had poor sitemaps and ineffective internal linking structure. After putting permanent redirects the crawl statistics was disrupted and the crawler performance has become unpredictable.

2. By the time of putting 301 (permanent) redirects on top-level (category) pages, page to page content granularity was missing. So, thousands of pages could not be found on the new site.

3. As a result, search engine result pages (SERPs) showed tons of results from the old domains with a new site in title but with the old path. The situation persisted for quite a long time as Google was unable to figure out whether or not the pages have really been moved.

A Google Wisdom part

According to Matt Cutts the 301 operates as follows. The crawler looks at as many pages as it is willing to crawl on the old domain and if it sees a 301 redirect than it will put that in a queue to craw at the new location (see below).

YouTube Video

Aspect 1 (Good): You Need a Good Sitemap

First thing to do when making massive 301 redirects is make sure you have a good sitemap. Because once you put a 301 redirect on a page Google is unable to crawl links accessible from that page.

Suppose a page is only accessible from other page that’s been redirected – it simply could not be found and crawled. My observation, if you put permanent redirect to 50% of your site’s pages, be assured your site internal linking structure & crawl stats will be devastated, no matter how good it was. So, a good site map is the only way to get your site’s pages found and crawled adequately.

Consider the situation – you have a poor internal linking structure, poor or no sitemap (‘bad scenario’) and still put tons of 301 redirects on your pages. In that case, Googlebot will find some of the 301 redirects and crawl them at the new domain. Still, due to the logics above, some pages that have 301 redirect won’t simply be accessible, because the Googlebot has been already redirected prior it had a chance to reach those.

So, Google won’t know that you was going to move the content at the new site. Accordingly, Google will keep some pages from your old domain in the search index forever – no changes despite the fact that you were using 301 (permanent) redirects.

Let’s look at the practical angle of that problem – some hotels listings were fetched by Google and moved to a new site, while some of them have been left on the old domains. So, if one wanted to consolidate listings from an array of old domains under a new site, obviously, h/she failed. And what’s more important, the site’s ranking will drop quickly and significantly.

Also, see a Matt Cutts narrative on whether or not redirecting a large naumber of domains suspicious:

YouTube Video

Why would organic rankings drop?

Aspect 2 (Bad): Mixed signals are bad for Google

Generally, Google needs to build up enough confidence to really know that the site has fully migrated from the old domain to a new one. Typically, over a period of several week (think about it as a couple of month) Google is might be able to detect that the site has entirely moved. If they get a mixed signals, like some pages are return a 200 (OK) and some pages return a 301, then Google does not know what to make of that.

Listed to Matt Cutts explaining how long does it take for Google to detect the site has been entirely moved:

YouTube Video

So, by putting a 301 redirect on an Index page (Category page) of your site, you really show Google that the site or its large portion have migrated for good. Yet, under the bad scenario above the crawler will detect that you have obviously forgotten lots of page on the old domains. Heck!

Besides, many of pages that have a 301 redirect could not be found on the new site. Once a page was not found on the new site (a 404 error), it is not counted as existing and does not pass a PageRank. Eventually, all 404 pages will disappear from Google search results page. And the PageRank passed on to them the old domain will be wasted. So, Google sees that you neither migrated fully, nor maintain your sites technically valid. And obviously, you have lost your PageRank. That’s the worst things with 301 redirects. Heck!

Aspect 3 (Ugly): no content granularity is ugly

Another aspect of most epic 301 mistakes is an absense of content granularity. In the most common case if you moving from one site to another you can put 301 redirects from site’s pages to go to the root of a new domain. But that’s kind of waste because it will seriously undermine user experience.

Here’s what Matt Cutts says about it (min 4:10):

YouTube Video

Obviously, the best way to make a great user experience and preserve a PageRank is to make the redirect in order to do a granularity of level to level content.

So, Google specifically recommends to do page level to page level redirects because it is a great user experience and a Page Rank will flow relatively well from the old site to a new one. Apparently, you shall start at the lowest level pages, for example, a listing page. Otherwise, if you put a 301 redirect on a Category page level, the crawler could not go deeper with it and would not crawl all linked Subcategories and Listing pages.

Obviously, the logic of 301 operation dictates only two options: ultimate migration or page to page redirect

The first option is make an ultimate migration and tell Google that you have really moved (use Google Webmaster Change Address Tool). To that end, you have to redirect all existing pages to the corresponding pages of the new site, which means that you will have to create them. Second option is ensure a proper page to page level redirects, without 404 errors and wrong signals to Google.

Key takeaways

To sum it up, I recommend several steps applicable to 301 (permanent) redirects if used massively, especially to channel traffic / PageRank from several domains to a single one.

1. Create a good sitemap that will encompass all of the old domain pages. To that end, it’s necessary to either lift the 301 redirects from a while or do that on a separate folder.

2. Submit that sitemap to Google.

3. Specifically ping Google that the site has been updated so fasten the indexation up.

4. Ensure page to page level redirects

5. Spot and remove 404 error pages - they 'steal' your pageRank.