Explaining a Drop That Is Not Real

Two kinds of drop

A figure falls and there are only two explanations: fewer people, or less measurement.

They look identical in a chart and are entirely different problems, and reacting to one as though it were the other wastes months.

The distinguishing evidence is usually available within ten minutes, and it starts outside the analytics entirely.

The harder part is not working out which it is. It is being able to show somebody else convincingly, when they have already decided the business is in trouble.

Check enquiries first

The fastest and most persuasive test, and it costs nothing.

If recorded traffic fell by a third and enquiries did not move, the visitors are still arriving and something stopped counting them.

If enquiries fell in proportion, that is a business change and the analytics is describing it accurately.

That single comparison resolves most cases, which is why an enquiry log kept independently is worth more than any configuration.

It is also the version of the argument that a non-technical person accepts immediately, because it rests on the phone rather than on an interface.

Look for an edge

The second test, and it identifies the cause once you know it is measurement.

Measurement changes produce a step: normal, then lower, from a specific day, with a sharp edge.

Business changes produce a slope: a gradual decline over weeks as seasonality or competition takes effect.

So the shape tells you a great deal before you know anything about the cause.

Find the exact date of the step and ask what happened that day: a site change, a plugin update, a banner, a migration, a supplier doing something.

The answer is nearly always on that date, and nearly always something nobody connected to analytics.

What causes false drops

• Tracking removed during a redesign.
• A tag firing on some pages only.
• A consent banner being added.
• A filter applied too broadly.
• Duplicate tracking removed, halving inflated figures.
• A reporting change, such as a different date range.

The fifth is the pleasant one: figures fall because they were wrong before, and the new lower number is the accurate one.

A worked example

A business reported that traffic had fallen by about forty percent and asked what had gone wrong with their marketing.

Enquiries for the same period were slightly above the previous month.

The chart showed a clean step on a Tuesday, not a slope.

That Tuesday, a developer had rebuilt three page templates, and the tracking code had been present on the homepage template and absent from the other three.

So the site had been recording roughly the homepage and nothing else for six weeks.

The fix took ten minutes and the figures returned to their previous level.

The valuable part was the enquiry comparison, which had told them within a minute that this was a counting problem, before anybody had opened a template.

Showing somebody else

The presentation problem, since being right privately is not the whole job.

Lead with the enquiry figures, because they are the outcome the reader cares about and they are not in dispute.

Then show the shape: a step on a date, next to the thing that happened on that date.

Avoid technical explanation until asked, since a detailed account of tag firing sounds like an excuse to somebody who came in worried.

Say plainly what it was, what you did, and whether the recorded figures will return to their previous level or stay lower permanently.

That last part matters: a consent banner is a permanent step, and a broken tag is a temporary one, and the reader needs to know which they are looking at.

Some drops are permanent and fine

Worth separating, because not every measurement change reverses.

Removing duplicate tracking, adding a consent banner, applying an internal traffic filter, or excluding a bot source all reduce the recorded number permanently and correctly.

In each case the new figure is the better one, and the previous series was overstated.

The right response is a note on the chart and a fresh baseline rather than an attempt to recover the old level.

Anybody comparing next year against the year before will otherwise see a decline that is entirely an artefact of getting the measurement right.

Write it down at the time

The habit that makes all of this easy the next time.

A single line whenever anything changes: the date, what changed, and whether it affects the figures.

Site rebuilds, plugin changes, banners, filters, new tracking, and supplier work all belong on it.

When a drop appears months later, that list is the first place to look and usually contains the answer.

Without it you are reconstructing from memory under pressure, with somebody waiting for an explanation, which is when people reach for the wrong one.

Check the range before anything else

The most embarrassing cause, and worth ruling out in the first thirty seconds.

A comparison of an incomplete period against a complete one shows a drop that is entirely an artefact: the current month against last month, six days in, will always look catastrophic.

The same happens with the last day or two of any range, since data is still being processed and recent figures are provisional.

And a default range that changed, or a report opened with a different period than usual, produces the same effect with no cause at all.

Confirm both periods are the same length and both are complete before investigating anything further.

The counter-case

Not every drop is measurement, and looking for a technical explanation first is its own trap.

Businesses do decline, competitors do improve, seasons do turn, and a search update can genuinely remove a page's traffic.

A drop with a matching fall in enquiries is real, and treating it as a tracking fault delays the response to an actual problem.

There is also a version where somebody blames measurement for every unwelcome number and never investigates anything, which is worse than the opposite error.

Check enquiries, look at the shape, and be willing to conclude that the number is telling the truth.

The ten minutes

1. Compare enquiries over the same period.
2. Look for a step rather than a slope.
3. Find the exact date of the edge.
4. Ask what happened on that date.
5. Check the tracking on several page types.
6. Say whether it recovers or is permanent.
7. Add a line to your change log.

Step one settles most cases in about a minute, and it is the version of the argument that convinces somebody who does not care how tags work.

Managing that conversation is covered in reporting to somebody who liked the old reports.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.