The Staging Site Nobody Blocked From Google

How the second site appears

A developer makes a copy of your site at a working address to build on, which is exactly correct practice.

The copy is a complete, functioning website. It has pages, links, and content, and nothing about it announces that it is not meant for the public.

If anything anywhere links to it, even once, it can be discovered and added to the search index like any other site.

Now there are two versions of your business online, with nearly identical content, and the search engine has to decide which one is real.

How it gets found

Rarely by anybody typing the address.

A link in an email that passes through a system which follows links. A reference in a shared document. A developer's own portfolio page. An analytics or monitoring tool that publishes what it sees. A link from the staging site to the live site, noticed in reverse.

None of that requires anybody to be careless in an obvious way, which is why this happens to competent teams.

What it costs

Two problems, and the second is worse than the first.

The mechanical one is duplication. Two sites with the same content force a choice about which to show, and occasionally the wrong one wins, so your customers land on a development copy with placeholder text and a broken contact form.

The one that actually causes damage is exposure. A staging site is where unreleased prices sit, where draft content nobody has approved sits, and where the client list you were going to publish next quarter sits.

It is also usually less carefully secured than the live site, because everyone treats it as temporary.

Why the usual methods fail

• A robots file requests that crawlers stay away; it does not stop the address being listed if others link to it.
• A noindex instruction works, and stops working the moment somebody copies the site to live with the setting still in place, or removes it too early.
• An obscure address is not protection, since nothing stops it being discovered.
• A platform tick-box for discouraging search engines is a request rather than a barrier.
• Remembering to do it later is the method that fails most often.

Each of these is a polite instruction to well-behaved software, and none of them prevents a human or a badly behaved tool from reading everything.

The method that works

Password protection at the server level, applied to the whole staging address before any content is put on it.

It works because it does not ask anybody to cooperate. Nothing without the password sees anything, including crawlers, including tools you have never heard of, including anybody who finds the address.

It is a few minutes of setup in most hosting control panels, it does not interfere with development, and it can be shared with the client easily.

Crucially, it fails safely. If somebody forgets to remove it at launch, the site is inaccessible and you find out within minutes. The opposite mistake, forgetting to add protection, is silent and lasts for months.

The reverse failure at launch

Worth naming separately, because it is the more expensive version of this problem.

A site built with a blocking instruction in place gets copied to live with the instruction still set, and the new site tells every search engine not to index it.

Nobody notices, because the site looks perfect. The symptom is that traffic falls away over the following weeks for no visible reason.

This is behind a large share of rebuilds that appear to have destroyed a business's search presence, and it is a single setting.

Check it within an hour of launch, and check it again the next day.

A worked example

A manufacturer discovered a staging copy of their site in the search results eight months after a rebuild finished.

It had been left running because the developer used it for occasional testing, and it had been found through a link in a public project management board.

The copy contained a full price list that had been drafted and never published, and an unannounced product page.

Their duplicate content problem was mild and would have resolved on its own.

A competitor having eight months of visibility into their pricing was not mild, and there was no way to establish who had looked.

The fix took ten minutes. The exposure was not reversible.

Cleaning up one that is already indexed

If a staging copy is already in the results, order matters.

Do not simply delete it, because that leaves the listings in place for a while with nothing behind them.

Add a noindex instruction to the staging site first and let it be recrawled, or use the removal tools in your search console for an immediate result.

Then apply password protection so it cannot recur.

Then, and only then, decide whether to keep the environment at all, and treat anything that was visible on it as having been public, because it was.

Applying it to everything else

Staging is the common case and it is not the only one.

Old sites left running at a subdomain after a migration. A second copy created for a redesign that was abandoned. A test installation from three years ago. A backup directory sitting inside the live site where anybody can browse it.

Search your own domain in a search engine, restricted to your site, and read the whole list. Most businesses find at least one thing they did not know was published.

The counter-case

There is one situation where a staging site should be reachable without a password: when it is being shown to people who cannot be given credentials easily, such as a wide group of stakeholders during a review.

Even then, the correct answer is usually a shared password rather than an open site, because the convenience is small and the exposure is total.

If it genuinely must be open, it should contain no unpublished pricing, no draft content, and no customer data, which in practice means it is not really your staging environment. It is a demonstration.

The rules

1. Password-protect staging before any content goes on it.
2. Never rely on a robots file or an obscure address.
3. Check the live site is indexable within an hour of launch.
4. Search your own domain and read everything listed.
5. Retire old environments rather than leaving them running.
6. Treat anything that was reachable as having been public.

The third item takes two minutes and prevents the most expensive silent failure in this entire process.

Getting the pages you do want found is covered in telling Google your pages exist.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.