Measuring Email Beyond the Open Rate

The number went up on its own

A great many businesses saw their open rate rise noticeably during the autumn of last year.

Nothing about their sending had changed: same list, same subject lines, same content, same schedule.

That is because the measurement changed rather than the performance, and the rise applied to almost everybody at once.

Which matters for anybody comparing this year against last, or reporting a trend to somebody who will read the improvement as a result.

What actually happened

Open tracking has always worked the same way: a tiny invisible image is included in the message, and its being requested is recorded as an open.

That was always approximate, since somebody could read a message without loading images, or load images without reading.

The change is that some mail applications now fetch those images automatically, in advance, whether or not anybody opens the message.

So a message delivered to those recipients is recorded as opened regardless, and the reported figure rises without a single additional person reading anything.

The share of recipients affected depends on which applications your list uses, and it is large enough to move the headline number substantially.

Why this is different from the old problem

Worth separating, because the two are frequently confused.

The old problem was undercounting: opens were understated, because people read messages without loading images.

The new problem is overcounting, in the other direction, for a portion of your list.

Which means the error is no longer a consistent bias you could mentally adjust for. It is a mixture of over and under, in a proportion you do not know.

A figure that is wrong in one direction is still useful for comparison. A figure that is wrong in both directions, by an unknown mix, is not.

What it breaks

• Year-on-year comparison across the change.
• Any target expressed as an open rate.
• Automations triggered by whether somebody opened.
• Inactivity cleaning based on opens.
• Benchmarks published before and after.

The third and fourth are the practical ones: a sequence that sends a follow-up only to non-openers now excludes people who never read the first message, and a cleaning rule that removes non-openers now keeps people who have not looked in a year.

What still works

The useful half, and it is enough.

Clicks are a deliberate act, and while some scanning systems follow links automatically, the distortion is smaller and more consistent.

Replies are entirely reliable and are the strongest signal a small list produces.

Enquiries and bookings that follow a send are the actual outcome, and they are counted by you rather than by any tool.

Unsubscribes are accurate, being a deliberate action recorded when it happens.

Between those four you can tell whether email is working, which is the question the open rate was standing in for.

A worked example

A business reported to their partner that email performance had improved by about a third over the previous year, based on open rates.

Clicks were flat. Replies were flat. Enquiries attributable to email were flat.

The improvement was entirely the measurement change, and the report had presented it as a result.

They rewrote the reporting around clicks and enquiries, marked the date of the change on the chart, and noted that figures either side were not comparable.

The revised picture was that email was steady rather than improving, which was less pleasant and was true.

The useful outcome was that they stopped attributing an improvement to work they had not done.

Opens are still useful for one thing

Worth keeping rather than discarding entirely.

Within a single send, comparing two subject lines, the distortion applies equally to both groups.

So a subject line test remains valid, because you are comparing like with like under identical conditions.

What is no longer valid is comparing an open rate across time, across lists, or against a published benchmark.

Use it as a relative measure inside one send, and never as an absolute figure in a report.

Fix your automations

The practical task, and it is the part with a real cost.

Find any sequence with a condition based on opening: send this only if they did not open, or tag somebody as engaged if they opened.

Change those to use clicks where the sequence supports it, which is a genuine action.

Where a condition cannot be changed, remove it and send to everybody, which is cruder and more honest than a rule that no longer means what it did.

The same applies to any list-cleaning rule: base inactivity on clicks over a longer period, or on time since the address was added, rather than on opens.

Those rules have been quietly wrong since last autumn on most accounts.

Mark the date in your reporting

The step that prevents somebody misreading a chart in two years.

Draw a line at the autumn of last year on any open rate chart spanning it, and label it as a measurement change.

Add one sentence saying figures either side are not directly comparable.

Then start a fresh baseline from that point, in the same way as for a consent banner or any other change to how something is counted.

Anybody looking at an unmarked chart will read a step as performance, which is exactly the mistake this article exists to prevent.

The counter-case

None of this makes email harder to judge in practice.

A small business was never going to make a decision on an open rate, and the figures that matter, which are replies and enquiries, are unaffected and always were the better measures.

There is also a risk of overcorrecting into distrusting every figure, when clicks and unsubscribes remain perfectly usable.

And the change is genuinely good for the recipients, which is why it happened, so treating it as an inconvenience misses the point.

Report clicks and enquiries, keep opens for subject line tests, and mark the date on your charts.

What to do

1. Stop reporting open rate as a result.
2. Mark last autumn on any chart spanning it.
3. Start a fresh baseline from that point.
4. Report clicks, replies, and enquiries.
5. Change automations that depend on opens.
6. Change cleaning rules to use clicks or age.
7. Keep opens for subject line tests only.

Step five is the one with an ongoing cost, since a follow-up sent only to non-openers is now missing people who genuinely never read the first message.

What email measurement was always worth is covered in measuring whether email is working.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.