Comparing Two Analytics Tools That Disagree

They will never match

Put two analytics tools on the same site and they will report different numbers for the same month, sometimes by a substantial margin.

That is not a fault in either, and it is not a sign that one is broken.

They are measuring different things, from different positions, with different rules about what counts.

Which means the correct response is not to reconcile them but to understand the shape of the difference and then stop trying.

Where they measure from

The most fundamental difference, and it explains the largest gaps.

Server-based tools, including the statistics your host provides, count requests arriving at the server. That includes bots, scanners, and anything that never rendered a page.

Script-based tools, including the main analytics products, run in the visitor's browser and only count when the page loads and the script executes.

So server statistics are almost always higher, frequently by a lot, because they include a great deal that is not a person.

Neither is wrong. One is counting requests and the other is counting rendered page views, which are genuinely different quantities.

The five reasons they differ

• Vantage point: server against browser.
• Blocking: some visitors block the script.
• Definitions: what counts as a session or a visit.
• Filtering: different lists of known bots.
• Time zone: where the day boundary falls.

The last is small and produces persistent confusion, since two tools set to different time zones will disagree about every daily figure and about every month boundary.

Blocking is the modern reason

Worth its own note, because it has grown considerably.

A meaningful proportion of visitors now block analytics scripts, through browser settings, extensions, or a browser that does it by default.

Those people visit your site, read it, and may contact you, and they are invisible to a script-based tool.

Which means your analytics undercounts, systematically, and by an amount that varies with your audience: a technical audience blocks far more than a general one.

Server-based figures capture those visits, which is one genuine argument for looking at both.

It also means a gradual decline in reported traffic over years may be a decline in measurement rather than in visitors, which is worth remembering before concluding anything about a long trend.

Definitions do the rest

The differences that are neither technical nor philosophical, just arbitrary.

How long a session lasts before it is considered ended, whether a visit continues across midnight, and whether arriving from a new source starts a new session are all decisions each tool makes differently.

The same is true for how a returning visitor is recognised, which affects every figure about unique visitors.

None of that has a correct answer, and a difference of a fifth between two tools can be entirely explained by definitions without either being inaccurate.

Which is why comparing a specific figure between tools is close to meaningless.

A worked example

A business had their host's statistics reporting roughly four times the visits their analytics showed, and spent a fortnight convinced their tracking was broken.

The tracking was fine.

The host's figures counted every request including bots, and the site was being scanned regularly, which accounted for most of the difference.

Removing known automated traffic from the host's figures brought the two to within about twenty percent, and the remaining gap was blocking and definitions.

They picked their analytics as the reference for trends, kept the host's figures for spotting unusual load, and stopped comparing the totals.

The fortnight had produced nothing except an understanding that they should not have expected the numbers to agree.

Pick one as the reference

The practical resolution, and it should be a deliberate decision rather than a default.

Choose whichever tool you will use consistently, and report from that one only.

Internal consistency is what makes a trend readable: this quarter against last quarter, measured the same way, is meaningful even if the absolute number is imperfect.

Use the other for what it is better at rather than as a check: server statistics for load, bandwidth, and bot activity; script-based analytics for behaviour, sources, and conversions.

Write down which is your reference and why, since otherwise somebody will quote whichever number is higher.

Never average them

The specific mistake worth naming.

Averaging two figures that count different things produces a number that describes nothing at all.

The same applies to presenting both in a report, which invites the reader to ask which is right and gets you no closer to an answer.

If somebody asks why the two disagree, the honest answer is that they measure differently and both are internally consistent, which is more credible than an attempt to reconcile them.

And if a supplier presents figures from a tool you do not use, ask which tool before comparing anything, since a large apparent improvement may simply be a different measurement.

Your own records are the third number

Worth introducing, because two disagreeing tools invite a tiebreaker and there is one available.

The count of enquiries you actually received is not an estimate produced by any tool, and it is not subject to blocking, definitions, or bots.

When two analytics figures disagree about traffic, your enquiry log still tells you what the month produced, which is the question underneath the argument.

It also lets you sanity-check either tool: a month reported as excellent by one and ordinary by the other can be judged against whether the phone rang.

Keep it, and it turns an unresolvable comparison into an unnecessary one.

The counter-case

Sometimes a disagreement is a fault.

A gap that appears suddenly, rather than existing consistently, is worth investigating: tracking removed during a redesign, a tag firing on some pages only, or a script installed twice all show up as a sudden divergence.

A tool reporting close to zero when another shows normal traffic is broken rather than differently calibrated.

And where two script-based tools of the same kind disagree substantially, that is more suspicious than a server-against-script gap, since they are measuring in the same way.

A stable difference is normal. A change in the size of the difference is a signal.

What to do

1. Expect a gap and stop trying to close it.
2. Know which is server-based and which is script-based.
3. Check the time zones match before anything else.
4. Pick one as your reference and write it down.
5. Use the other for what it is better at.
6. Never average or present both.
7. Investigate a change in the gap, not the gap.

Step seven is the useful discipline, since a consistent difference is expected and a difference that suddenly widens is usually a tracking fault.

Reports that flatter their subject are covered in the report that made a bad decision look good.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.