Keeping a Record of What You Changed

The question that always arrives late

A figure moves, and somebody asks what changed around then.

The honest answer, months later, is that nobody knows. There was a plugin update, possibly a redesign, an advertisement that ran for a while, and somebody thinks the form was altered.

So the investigation becomes archaeology: checking file dates, looking through emails, and asking a supplier what they did in October.

All of which is avoided by one line written on the day, which takes ten seconds and is the single highest-return habit in this whole subject.

What to record

• Site changes: pages, templates, structure, redesigns.
• Tracking changes: tags, filters, new properties, consent.
• Advertising: campaigns starting, stopping, or changing budget.
• Technical: migrations, plugin updates, outages, certificates.
• Business: price changes, a service dropped, staff leaving.
• External: a mention in the press, a supplier's change.

The fifth is the one people leave out because it does not feel like a website matter, and it explains more movements than any of the technical entries.

The format

Deliberately minimal, because anything elaborate will not be maintained.

A date, one sentence describing what happened, and who did it.

Optionally a word about whether you expect it to affect the figures, which is useful later for distinguishing what you predicted from what actually happened.

That is the whole format, and adding fields to it is how these become documents nobody updates.

Write it in the same place every time, in reverse date order so the recent entries are at the top.

Where it should live

Outside the analytics account, which is the point people get wrong.

Analytics tools offer an annotation feature, marking a date on a chart with a note, and it is genuinely useful for reading that chart.

It is also inside a product you may stop using, on a property that may be replaced, visible only to people with access, and lost entirely if the account changes hands.

Use annotations if your tool has them, and keep the real record in a document you own: a spreadsheet, a text file, or a page in whatever notes system you already use.

The tool annotation is the convenience. The document is the record.

A worked example

A business found their enquiries had roughly halved over a quarter and could not account for it.

They spent an afternoon checking the form, the tracking, and their search positions, all of which were fine.

Eventually somebody remembered that the telephone number on the site had been changed in that quarter, when a line was switched, and the old number had been left on two pages.

Nobody had recorded the change, so it had not been a candidate explanation.

Afterwards they started a single-page log and filled in the previous year from memory, which took twenty minutes and immediately explained two other movements nobody had understood.

The reconstructed entries were less reliable than contemporaneous ones and were still considerably better than nothing.

Record the boring ones

The discipline that determines whether the log works.

The temptation is to record only significant changes, which means the log contains redesigns and campaigns and omits the plugin update that broke a template.

Small technical changes are exactly what causes unexplained movements, and they are the entries you will be grateful for.

The same applies to things that seem unrelated: a supplier's change, a staff departure, a period when nobody was answering the phone.

If you are unsure whether something belongs, put it in, since the cost is a line and the benefit is an explanation you would not otherwise have.

Who writes it

A practical point, since a log depending on one person will have gaps whenever that person is away.

Anybody who can change something should be able to add a line, which means the document has to be somewhere shared rather than in an individual's notes.

Ask suppliers to tell you when they change something, which most will do if asked once and reminded occasionally.

For a maintenance arrangement, a monthly note of what was updated is a reasonable thing to request and costs the supplier nothing.

Where that does not happen, add a line yourself when you notice something has changed, dated to when you noticed rather than when it happened, and say so.

What it is worth

Beyond explaining movements, which is the obvious benefit.

It tells you what you actually did over a year, which is frequently less than anybody remembers and is useful when planning the next one.

It shows which changes were followed by anything, which is the closest a small business gets to knowing what works.

It answers a client or a partner asking what has been done, without reconstruction.

And it protects a supplier as much as a client, since a documented change with a documented date is how a dispute about causation gets settled.

Note what you expected to happen

A small addition that turns the log into something more useful than a history.

Alongside the change, write in a few words what you expect it to do: more enquiries from the service pages, faster loading, no visible effect.

Then, when you next read the log, you can see which predictions were right, which is the only cheap way a small business learns what works.

Most predictions turn out to be wrong, and discovering that is worth more than the individual outcomes, because it calibrates how confident to be next time.

It also stops the common pattern of remembering the changes that appeared to work and forgetting the ones that did not.

The counter-case

A log is worth only what it costs to maintain.

A business changing something twice a year does not need a document, and can reasonably rely on memory and the occasional email.

There is also a failure mode where the log becomes elaborate, gains categories and owners and a review process, and is abandoned within three months.

And it is not evidence of causation: something appearing on the same date as a movement is a candidate rather than an explanation, and treating the log as proof produces confident wrong conclusions.

One line, one place, low ceremony, and treat it as a list of candidates rather than a list of causes.

The habit

1. Start today rather than at a tidy moment.
2. Fill in the last year from memory, roughly.
3. One line: date, what, who.
4. Keep it outside the analytics account.
5. Record the boring changes too.
6. Make it shared, not personal.
7. Ask suppliers to tell you what they change.

Step two is worth the twenty minutes, since a reconstructed year is imperfect and still explains movements nobody had understood.

Using it to diagnose a fall is covered in explaining a drop that is not real.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.