What a Dashboard Should Not Contain

Two kinds of bad number

Some figures on a report are simply not useful: follower counts, page views on the terms page, time on site.

Those waste attention and do no other harm.

A second group actively produces wrong conclusions, because they look precise and describe something other than what the reader assumes.

Removing the second group matters more than shortening the report.

Averages that hide the distribution

Which is the most common of these.

An average order value of two hundred dollars can describe forty orders around two hundred, or thirty-eight around ninety and two at two thousand.

Those are entirely different businesses and the average is identical.

The same applies to average time to reply, average job value, and average anything with a long tail.

Report the median alongside it, or the range, or simply the count in three bands.

Where only one figure is available, the median is nearly always the more honest one for a small business.

Percentages on small numbers

• A conversion rate on eleven enquiries.
• A change of forty per cent on two.
• A rate shown without its count.
• Any percentage of a percentage.

The third is the fixable one. A percentage always needs the count beside it, since a conversion rate of nine per cent means something different at eleven enquiries and at four hundred, and only one of them is worth reacting to.

Totals that combine incomparable things

Which quietly produce the wrong priority.

Total conversions, where a newsletter signup counts the same as a quote request.

Total enquiries, where a two-hundred-dollar job and a twenty-thousand-dollar one are one each.

Total traffic, where a visitor from your own town and one from another country are the same.

Each of those combines things that should be separate, and the combined figure moves for reasons the reader cannot see.

Split them or weight them, and where neither is possible, report the components rather than the total.

A worked example

A business reported total conversions monthly and watched it rise steadily for a year.

The rise was almost entirely newsletter signups from a popup added the previous spring.

Quote requests, which were the only conversions that produced revenue, had fallen slightly across the same period.

The combined figure had concealed a real decline behind an irrelevant increase, and the popup had been credited with a success it had not produced.

Splitting the two took ten minutes and reversed the conclusion.

Nobody had been careless; the figure was simply combining two things that should never have been added.

Ratios where the denominator moves

A subtler trap worth knowing.

A conversion rate improves when enquiries hold and traffic falls, which is not an improvement in anything.

So a rate rising is ambiguous between the numerator rising and the denominator falling, and the report shows only the rate.

Always show both components alongside any ratio, which is two extra columns and removes the ambiguity entirely.

A business celebrating a conversion rate improvement caused by a traffic collapse is not a rare occurrence.

Anything without a comparison

Since a single figure carries no information.

Eleven enquiries is neither good nor bad until it sits beside the same month last year.

A dashboard showing current values with no comparison invites the reader to supply one from memory, which is where most wrong conclusions start.

Every number should appear with a comparison attached, and where no comparison exists yet, say so rather than showing the figure alone.

That single rule improves most reports more than removing anything.

Charts that mislead by construction

Worth checking on anything visual.

An axis not starting at zero makes a small change look dramatic, which is occasionally appropriate and usually not.

A chart with a different time range from the one beside it invites a comparison that is not valid.

And a trend line drawn through four points implies a confidence the data does not support.

Look at any automated chart with those three in mind, since defaults produce all of them without anybody choosing to mislead.

The person who built the tool made a reasonable general decision, and it happens to be wrong for your numbers.

The rounding matters more than it looks

A small point that changes how a report is read.

Reporting a conversion rate as two point seven per cent implies a precision the underlying eleven enquiries cannot possibly support.

A reader treats a decimal place as evidence that somebody measured carefully, and reacts to a move from two point seven to two point one as though it meant something.

Round to whole numbers, or better, report the counts and let anybody who wants the rate work it out.

Precision beyond what the data supports is a form of misleading that nobody intends and everybody does.

Say what the number does not include

Which is the honest addition to almost any figure a small business reports.

Enquiries from the website does not include telephone calls, walk-ins, or referrals who called directly.

Traffic does not include anybody blocking the tracking, which is a meaningful and unmeasured share.

One line under the report saying what is excluded prevents the reader treating a partial figure as a complete one.

That is particularly important where somebody outside the business reads it, since they have no way of knowing what the number covers.

The counter-case

Some of these are practical compromises.

An average is easier to produce than a median in most tools, and for a stable distribution it is fine.

Combined totals also serve a purpose where the components genuinely are equivalent, which occasionally they are.

And a report showing only perfectly constructed figures may end up showing nothing at all, which helps nobody and is its own kind of failure.

Show counts beside every percentage, split combined totals, show both components of any ratio, and attach a comparison to everything.

The audit

1. Find every average.
2. Add a median or a range.
3. Add the count beside every percentage.
4. Split combined totals.
5. Show both parts of any ratio.
6. Attach a comparison to each figure.
7. Check the axes start at zero.

Step four is where the wrong decisions actually come from, since a total that combines a signup with a quote request will eventually credit the wrong thing with a rise.

How this goes wrong in practice is covered in the report that made a bad decision look good.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.