Deciding What Success Looks Like First

The problem with judging afterwards

Any set of numbers contains something that went up.

Which means a change evaluated after the fact can always be presented as a success, by selecting the metric that moved favourably.

That is not usually dishonest. It is what happens when nobody decided in advance what they were looking for.

The consequence is that nothing is ever stopped, because nothing is ever clearly judged to have failed.

What to write down before

• Which number should move.
• In which direction, and by roughly how much.
• By when.
• What the number is now, as a baseline.
• What you will do if it does not move.

The fifth is the one that makes the exercise real. A target with no consequence attached is a hope.

Deciding in advance that a directory listing will be cancelled if it produces nothing by March is what makes the measurement worth taking.

Who agrees it

Worth settling, since a target agreed by one person is not a decision the business made.

Where a supplier is involved, the target should be agreed with them before the work starts, not imposed afterwards.

That conversation is useful in itself: a supplier who will not commit to any expected outcome is telling you something, and one who commits to something implausible is telling you something else.

Where the work is internal, writing it down is enough, provided whoever will make the eventual decision has seen it.

Choosing the number

It should be the one closest to money that you can actually measure.

Enquiries, ideally. Jobs won, if you record them. Tracked conversions, if that is what you have.

Traffic is the wrong choice for most changes, because it can rise while enquiries fall, and a change judged on traffic can be declared successful while making the business worse.

Where traffic is the only available number, say so explicitly and treat the conclusion as provisional.

Being honest about having a weak measure is considerably better than pretending a strong one exists.

Setting a realistic figure

Harder than it sounds, and the common failures run in both directions.

Too ambitious, so that a genuine improvement is recorded as a failure and something worth keeping gets stopped.

Too modest, so that ordinary variation clears the bar and nothing has been tested.

The useful anchor is your normal range: look at twelve months of the figure and see how much it moves on its own.

A target should be outside that range, because anything inside it will be reached eventually whether or not the change did anything.

A worked example

A business about to spend on a directory listing for a year.

Before starting, they wrote down: this should produce at least one job per quarter, measured by asking every enquiry, and if it produces none by six months it will be cancelled at renewal.

At six months it had produced two enquiries and no jobs.

The conversation about renewal took two minutes, because the decision had been made in advance by people who were not yet invested in the answer.

Without that note, the same conversation would have involved the two enquiries being described as promising, and the listing renewing.

The value of writing it down was not the measurement. It was removing the decision from a moment when nobody wanted to admit the money had been wasted.

Setting the time period

Longer than instinct suggests, and decided before rather than during.

Most marketing changes take a season to show anything, and search changes take longer.

A change judged at four weeks will usually look like a failure, and one judged at two years has been running unexamined.

Three to six months is a reasonable default for most things a small business does, with a note that search work sits at the longer end.

What matters more than the exact period is that it was agreed at the start, because a period chosen during the evaluation will be chosen to suit the result.

The counter-case

Where formal criteria are disproportionate.

Small changes that are obviously worth making regardless: fixing a broken form, adding a phone number, correcting contrast. Nobody needs a target to justify those.

Anything cheap and reversible, where the cost of testing exceeds the cost of trying.

And work required for reasons other than return, such as compliance or accessibility, which is not being judged on its numbers.

The exercise is for decisions involving real money or real time, which for a small business means anything recurring or anything taking more than a week.

When the answer is genuinely unclear

Worth planning for, since it is a common outcome and people are unprepared for it.

The number moved a little, or moved in the right direction by less than the target, or moved while something else also changed.

The temptation is to declare it a partial success and continue, which is how things run indefinitely without ever being judged.

The better response is deciding in advance what an unclear result means: extend once for a stated period, then stop regardless.

That converts an ambiguous outcome into a scheduled decision rather than an open question, and it prevents the extension being renewed repeatedly.

Writing it somewhere

The mechanism matters, and it is unglamorous.

A single document listing each thing tried, the date it started, the target, and what happened.

Two lines per entry, kept for years.

What that produces after three years is a record of what has and has not worked for this specific business, which is more valuable than any general advice about what usually works.

It also prevents repeating something that failed in 2018 because nobody remembers it was tried.

Before the next change

1. Write down the current figure.
2. Write down what would count as working.
3. Write down by when.
4. Write down what you will do if it does not.
5. Put it somewhere you will find it.
6. Look at it on the date, not before.

The sixth prevents the most common failure of all, which is checking weekly, seeing ordinary noise, and either abandoning something too early or convincing yourself it is already working.

Why the numbers move on their own is covered in comparing this month to last month badly.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.