Two Thousand Redirect Rules Read on Every Visit

A file that grew with every redesign

Every time a site is rebuilt, some addresses change. A careful developer adds a redirect for each old address, so that old links, bookmarks and search results land somewhere useful.

On most small business hosting, those redirects live in a file called .htaccess, read by Apache, the server software most shared hosts run. After a few redesigns and a platform move, the file can hold thousands of lines.

Nobody removes anything, because nobody is sure what is still needed. So the list only grows.

And the server reads it far more often than anyone imagines.

Why the length costs time

Apache reads the .htaccess file afresh for every request it handles. Not once a day, and not once a visit: once for the page, then again for every image, stylesheet and script the page asks for.

For each request it works down the rules in order, checking whether the address matches. Rules written as patterns take more work to check than plain addresses, and thousands of either add up.

A typical page makes dozens of requests. Each one waits while the server compares it with addresses that have nothing to do with it, before a single byte is sent.

Where the lines come from

• A line for every product when the shop changed platforms.
• A line for every blog post when the address format changed.
• Redirects to redirects, one layer from each redesign.
• Rules a plugin wrote and left behind when it was removed.
• Campaign addresses from promotions that ended years ago.

The third also costs the visitor directly. An old address that redirects to another old address, which redirects again, sends the browser back and forth before the real page even starts, and every trip is slower on a phone.

How to see the cost

Time to first byte, the wait before a server starts sending a file, is where a long rule list shows up. WebPageTest and the browser's developer tools both report it for every file.

If even tiny images wait noticeably before arriving, the rules are a likely suspect. Ask your developer or host how long the file is.

Patterns instead of lists

Old addresses usually follow rules of their own. Every product page on the old shop might have ended in a product code, every post in a date and a title.

Where the new address can be worked out from the old one, a single pattern replaces hundreds of individual lines: anything in the old products folder goes to the matching page in the new one.

Patterns need care, because one written too broadly will catch new pages too. Written well, a handful does the work of the whole list.

Check the rest only when a page is missing

Some old addresses follow no rule at all and need a list. That list does not have to be read for every request.

The site's own software can check it only when nothing else matches, at the moment it would otherwise show a not found page. Ordinary requests, which are nearly all of them, skip the list entirely.

Many content management systems and shop platforms can work this way, which moves the cost from every visit to the rare one that needs it.

A worked example

A cleaning supply wholesaler sells floor finishes, chemicals, paper products and equipment to janitorial companies, schools and restaurants, from a warehouse with a trade counter and an online catalogue of several thousand products.

The catalogue had moved twice: from a custom system to a shop platform, then to a new design with a new address structure. Each move added a redirect for every product, category and page.

By this year the .htaccess file held more than two thousand lines, many sending one old address to another. Every product photograph waited while the server worked through them.

The developer found that the old product addresses all ended in the product code, which had survived both moves, so one pattern replaced most of the list.

A year of server logs showed which remaining addresses anyone still requested. The rest were removed, apart from any with links from other sites, and the survivors went into a list the shop checks only for missing pages.

Time to first byte fell on every file. The developer then tested the most visited old addresses, and each now reaches the right page in one step.

Removing what nothing uses

The server's access logs record every address requested. A year of them shows which old addresses still bring anyone, and which have been silent since the last redesign.

Keep any redirect that still gets requests, and any for an address other sites link to, which the links report in Search Console helps find.

The rest can go. An address nobody has asked for in a year, with no links pointing at it, is not worth checking on every request.

Flatten the chains

Wherever a redirect points to an address that itself redirects, change it to point straight at the final page. Each old address should reach its destination in one step.

That spares visitors the extra trips, and search engines too, which may give up partway along a long chain.

Test the important ones afterwards

Before changing anything, list the old addresses that matter most: the most visited, the most linked, and any printed in brochures or on vehicles.

Afterwards, check each one. It should return a single permanent redirect to the right page, not a chain, an error or the home page.

Then load a few new pages to make sure no pattern caught them by mistake. Keep the list, because it is the test for the next redesign too.

The counter-case

A site with a few dozen redirects has nothing to gain here. The cost only becomes noticeable when the list runs to hundreds or thousands.

Servers that keep their rules in the main configuration, read once at startup, pay far less for a long list, though chains still cost visitors time.

And no redirect should go just to shorten a file if it still brings visitors. Turn lists into patterns, check the leftovers only for missing pages, and test the addresses that matter.

The order to do it in

1. Count the redirect lines in the file.
2. List the old addresses that matter most.
3. Replace repeated lines with a few patterns.
4. Move the leftovers to a missing-page check.
5. Remove redirects unused for a year.
6. Point every chain straight at its final page.
7. Test the important addresses after each change.

Step two comes before any deleting. Without a list of the addresses that must keep working, a clean-up can quietly break the links that bring people in.

Why every extra request costs time on a slow connection is covered in connections closed after every file.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.