Testing With People Who Use Assistive Technology

What the ceiling looks like

A site can pass automated scanning, pass a manual keyboard check, and satisfy every item on a standard, and still be genuinely difficult to use.

Because conformance is measured element by element, and using a site is a task carried out in sequence.

The failures that survive an audit are things like: the form is technically operable but the order makes no sense once you cannot see the layout. Every control is labelled, and the labels use words nobody outside the business uses. The error is announced, and by then the session has timed out.

Nobody finds those with a checklist. They are found by watching somebody try.

What you learn that you cannot otherwise

Three things, consistently.

Where people give up. Not where they struggle, which you can guess, but the specific point at which continuing stops being worth it. That point is usually earlier than expected.

What they do instead. Experienced assistive technology users have workarounds and shortcuts, and watching them reveals both what your site makes hard and what it makes unnecessary.

How fast the real thing is. Somebody proficient with a screen reader moves through a page far faster than you did when you tried it. Which means your own test was not a simulation of their experience, only of your unfamiliarity.

That last point is worth sitting with, because self-testing tends to produce sympathy rather than information.

Finding participants

The practical obstacle, and it is smaller than it looks.

Local organisations serving people with disabilities can frequently connect you, and many are glad to, particularly where a local business is genuinely trying.

Specialist testing services recruit participants with specific technologies and needs, and cost more.

Existing customers, if you know of any who use assistive technology. Asking has to be done carefully and it is usually welcomed when the reason is stated plainly.

Community groups and forums, where a respectful request for paid participants is generally received well.

Pay people. This is professional expertise, the participant is doing work, and unpaid research asking disabled people to fix a business's website is not a good look and does not produce committed participants.

Running it without wasting their time

1. Give them a task, not a tour. Book an appointment, find the price, send an enquiry.
2. Say nothing while they work. The urge to help is strong and it destroys the data.
3. Let them use their own setup, their device, their software, their settings.
4. Ask them to narrate what they are doing and expecting.
5. Watch what happens, rather than asking what they think afterwards.
6. Keep it short, an hour at most.

The third matters more than it sounds. People configure assistive technology heavily, with their own speech rate, verbosity settings and shortcuts. Testing on your setup tests your setup.

How few people you need

Fewer than expected.

Three to five participants surfaces most of the significant problems, because serious barriers affect nearly everybody who hits them rather than a random few.

Which makes this affordable for a small business. Three sessions is a day, and it produces a list of concrete things rather than a conformance report.

Different technologies matter more than more people using the same one. A screen reader user, somebody using magnification, and somebody using voice control will each find different things.

What to do with what you find

Separate the two kinds.

Blockers, where the task could not be completed. Those get fixed, and they are usually few.

Friction, where it took longer or was unpleasant. Those get prioritised against everything else you are doing.

Resist the urge to explain why something happened. A participant who could not complete a booking has told you something true regardless of whether the cause was your site, their configuration, or a third-party widget.

Fix the blockers, then test again with somebody new. The second round finds a different set, and the second set is smaller.

The reason to do it beyond the site

Watching somebody use your site changes how the people who build it think, permanently and more than any training does.

A team that has watched one session writes different markup afterwards without being asked to, because the abstraction became a person.

Which is arguably the larger return, and it is not something a report produces.

The self-testing that comes before this, and its limits, is set out in checking your own site for accessibility problems.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.