A number can be necessary and still insufficient. A conformance percentage, completion rate, or incident count helps organize discussion and notice change, but it cannot by itself show whether a person participated independently. When a metric becomes the only compass, it can lead a team toward the appearance of success while the journey remains closed to people using it under different conditions.
Accessibility is tested in the relationship between a person, a task, and a particular environment. A form may pass an automated review and remain impossible to complete on an older phone, with limited connection, by keyboard, or under time pressure. Measurement matters; interpretation with people, contexts, testing, and real barriers turns a datum into a responsible decision.
What a metric says and what it leaves out#
Metrics summarize a complex reality. The count of pages without detected errors may indicate technical improvement; average task time may reveal friction; support requests may warn that one channel is inadequate. None of these signals equals the whole experience. An average can conceal abandonment, and a high success rate may count only people who managed to enter the system.
Metrics summarize a complex reality. The count of pages without detected errors may indicate technical improvement; average task time may reveal friction; support requests may warn that one channel is inadequate.
The first step is defining the question before selecting an indicator. Asking whether a site loads quickly differs from asking who can book an appointment without help. The second question requires observing entry, understanding, error recovery, help, and confirmation. It also requires separating outcomes by channel and acknowledging that people who never gain access rarely appear in analytics.
Quantitative data should be combined with qualitative evidence. Numbers provide scale and continuity; interviews, usage diaries, and consented observation explain why something happens. One comment does not establish a trend, yet it can uncover a serious barrier that an average dissolves. The purpose is not to accumulate dashboards, but to build an accurate enough picture to act.
Interpreting the journey in context#
The same service changes with place, time, and available resources. An accessible video call in a quiet office may not work for somebody sharing a room, caring for another person, or having little data. A clear on-screen instruction may be useless in bright light or for someone switching between the process and a document reader. Context is not an exception: it is part of the requirement.
Testing should therefore reproduce plausible conditions rather than only the project ideal. It is useful to include a range of phones, text enlargement, keyboard navigation, assistive technologies, slow connections, and interrupted tasks. Messages before and after the main screen also need review: invitations, emails, attachments, calls, and in-person assistance.
Privacy requires care. Diagnoses need not be recorded and people need not be turned into categories to establish that a barrier exists. The obstacle, affected task, channel, consequence, and correction priority can be documented instead. Collecting only what is needed, explaining the purpose, and allowing non-response protects contributors and improves the evidence.
Testing with people and real barriers#
Case 1: the success rate that excluded abandonments#
A booking portal reported 94 percent completed operations. The dashboard looked excellent until a local association explained that several people could not select a date by keyboard or understand an expired-session notice. People encountering the problem left before beginning a booking, so they were absent from the denominator. The number described the behavior of an already filtered part of the population.
The review added moderated sessions with keyboard, screen-reader, and mobile users. Blocking points were recorded without asking for medical information: calendar, error message, recovery, and contact alternative. Labels, understandable notice, saved data, and a free phone line were added. The team then measured completion alongside failed attempts and the time required to obtain help.
The lesson was not that percentages lack value. They need an explicit reference population and critical reading. The team began reporting who was outside the sample, which assumptions supported the indicator, and what evidence required review before improvement could be claimed. The dashboard became a learning tool rather than a trophy.
Indicators that guide without oversimplifying#
A balanced set includes outcome, effort, and equity. It can observe whether a task is completed, how many steps it requires, how many people need intervention, how quickly an alternative response arrives, and whether the outcome is equivalent across channels. Repeated incidents, complaints, returned documents, and abrupt post-update changes also matter. Each measure needs a shared definition and a named person responsible for review.
Targets must avoid perverse incentives. Rewarding only shorter average duration can remove necessary explanation or push out people proceeding slowly. Valuing only fewer support contacts can hide the phone line. A safer target combines speed with understanding, ability to correct, access to help, and quality of outcome. What is never measured disappears from discussion; what is measured badly can distort it.
Comparisons demand caution. Fewer requests may mean improvement, or may mean that information about support became invisible. More use of one channel may follow a campaign, or follow the other channel becoming inaccessible. Every variation should be checked against operational changes, user accounts, and task testing. Causality does not come from an isolated chart.
Governance, correction, and continuous improvement#
Case 2: automated conformance that hid a decisive instruction#
An organization published a PDF guide and celebrated because its checker found no errors. In practice, reading order was confusing, tables lacked headers, and a critical instruction was embedded in an image. Screen-reader users called to ask someone to read the document. The technical indicator was useful, but it did not cover understanding or completion of the task.
The correction went beyond the file. The guide was rewritten in plain language, a structured HTML version was published, and a contact with a response deadline was offered. A test group checked whether it could find, understand, and apply the instruction without assistance. The team documented what failed, which publication control was added, and who would check future changes.
Governance turns learning into sustained practice. Procurement should request verifiable criteria and acceptance testing; content teams need time and templates; support needs authority to escalate incidents; leadership needs unvarnished risks and progress. A barrier is prioritized by harm, frequency, and absence of an alternative, not by how easily a quick improvement can be displayed.
Deciding with sufficient evidence#
Case 3: the improvement that moved the barrier#
To reduce abandonment, a service replaced a long form with identity verification by a code sent to a mobile phone. Completion rose immediately and the project presented it as a victory. Days later, problems emerged for people without stable coverage, with shared numbers, or unable to receive messages during work. One visible difficulty had been removed and another, less recorded one had been created.
The response restored several equivalent routes: codes through different means, assisted verification, and in-person help with privacy safeguards. Every alternative was tested end to end and announced before the process. The principal indicator ceased to be total completion and included initial access, success by channel, waiting times, and later outcome. Decisions were reviewed with participants and service staff.
Four institutional sources support this approach. The United Nations Convention on the Rights of Persons with Disabilities places accessibility and participation among conditions of equality. W3C WCAG provides testable technical criteria. The European Commission brings together the European accessibility framework for products and services. The World Health Organization describes disability as interaction between conditions and environmental barriers. None replaces local testing, but together they show that a number must serve rights rather than replace them.
A useful metric ends in a question: which human experience explains this result, and which barrier remains? Asking it regularly prevents conformance from being confused with inclusion. Quality appears when people can begin, understand, request help, correct, and finish a journey in real conditions, without disproportionate burdens or a need to prove why an alternative is deserved.
Follow-up must also include what never arrives as a complaint. People may not report difficulty because they do not know whom to contact, because they found informal help, or because they expect nothing to change. Regularly reviewing priority journeys, opening accessible observation channels, and sharing completed corrections in understandable language reduces that silence. Accountability does not ask every person to prove the problem again: it requires the organization to seek evidence, correct it, and check that the solution still works after release.
It is reasonable to publish limitations beside results. If a sample excludes users of an alternative route, if a test was short, or if a change coincides with a campaign, that information changes the interpretation. Explaining uncertainty does not weaken a decision; it allows it to be treated as provisional and schedules the next check. Accessibility matures when indicators retain the memory of context and decisions retain the memory of affected people.
Teams should make this review practical. At a regular interval they can select one important task, inspect its evidence, invite paid and supported participation where appropriate, and record a small number of accountable actions. The record should say what will change, by when, how equivalence will be tested, and what will happen if the preferred channel fails. This modest discipline prevents accessibility from becoming a report written after a project is over. It keeps inclusion close to operational work: writing a notification, renewing a supplier contract, changing an identity check, or responding to an incident. A number then has a proper role. It signals where to look, while people and their real circumstances determine what should be done.
Evidence should remain open to revision. When a new barrier is found, the proper response is to update the measure, preserve an alternative route, and tell affected people what changed. That cycle makes measurement a support for access rather than a substitute for it.
