The school meal service’s quarterly report arrives with a reassuring figure: the system estimating how many special meals to prepare was correct for ninety-seven percent of requests. Management celebrates that there is almost no waste and that cooks have more time. Lucía, whose child has an uncommon allergy, reads the figure after her son received an ordinary menu for two weeks. The error was corrected after several calls, but official precision makes her experience appear to be an unimportant exception. The uncomfortable question is not whether the percentage is high. It is what happens in the remaining three percent, and who carries the work of correction.
An average may describe general performance, but it cannot decide whether an error is acceptable. It mixes cases that do not carry equal weight: a prediction that produces one extra piece of fruit and a classification that misses a health need share a column if only accuracy is counted. It can also gather under one number schools with straightforward forms and families communicating a condition in another language, through a written note, or after a conversation. Apparent neutrality is created by removing context before measurement begins.
A figure needs a question before it becomes an answer#
The school does not need to abandon a tool because it reveals a limit. It needs to decide what it is trying to protect. Here, optimizing purchases is not enough; it must ensure every child receives a safe meal and every family can correct information without becoming an expert in the system. That purpose makes more useful measures possible than a global percentage: how many special requests were checked, how many changes reached the kitchen on time, which routes failed, and how long the centre took to repair each situation.
The school does not need to abandon a tool because it reveals a limit. It needs to decide what it is trying to protect.
Breaking results down does not mean invasively labelling people. It can be done through process paths: request received by form, telephone, or mediation; temporary change, declared allergy, cultural adaptation; outcome confirmed before or after a meal was served. If non-standard communications reach the kitchen late, there is a design problem even if the average remains impressive. The school must see that difference because the system is not there to win a metrics competition. It exists to support a specific obligation of care.
Lucía should not have to prove that her case represents many families before receiving attention. A human route with authority can verify the information, halt the automatic recommendation, and leave a clear instruction for meal staff. This exit is not a sentimental addition for when technology fails. It is part of the procedure that makes its safe use possible. If review depends on knowing a hidden address, persisting for days, or repeatedly explaining an intimate problem, the tool has moved its uncertainty onto the person least able to bear it.
Following the full chain avoids blaming the last click#
When the centre investigates Lucía’s case, it may find several pieces. Perhaps the system recognizes the allergy in a field but not in an attached note. Perhaps an amendment is stored in one profile while the kitchen receives an earlier copy. Perhaps a person followed a sensible procedure without knowing the recommendation had changed. Saying “an AI error occurred” neither explains this chain nor shows where to intervene. A useful review retains the connection between what the family communicated, the transformation made, the practical decision, and the meal served.
That reconstruction needs limits. It does not require keeping conversations forever or opening personal records to all staff. It requires reviewers to know which request version reached the kitchen, which rule intervened, and when someone altered the result. With that traceability, the centre can correct the present case and look for similar paths without making one family the symbol of a public failure. Privacy and accountability do not compete when access is designed proportionately.
The process should also be rehearsed with cases that do not fit an ideal input: a change reported by telephone, a family asking for help to read the form, a temporary condition, a name written differently, or an instruction needing clarification. The aim is not to make every ambiguous message trigger an alarm. It is to know when the system should request a human check rather than fill a gap with an assumption. Safety is demonstrated in exceptions, not only in clean examples from a presentation.
Repairing includes finding the people nobody called#
After an incident, the school could limit itself to updating Lucía’s profile. That would be insufficient. It needs to identify which group may have travelled the same path: requests with attached notes, changes made after a date, messages sent through one channel. Review should not wait for every family to complain again. Some people may have brought food from home, accepted a confusing explanation, or never learned that the problem was misread information. Recovering those cases is part of repair, not an optional gesture of reputation.
Communication should separate facts, uncertainties, and measures. The centre can say that it found a problem in transferring certain instructions, that it is reviewing potentially affected requests, and that a temporary direct line exists to confirm needs. It need not promise that nothing will recur or publish sensitive details. It must explain who will respond, by when, and what alternative exists if the system cannot guarantee an outcome. Families need a concrete solution, not a defence of the average figure.
Reversibility matters before and after deployment. If a mistaken recommendation can be stopped, corrected, and recorded, possible harm is limited. If it spreads through purchasing, lists, and communications with no person able to intervene, speed multiplies exposure. Before adding automation, leadership should ask: which signal will make us pause, who can do it, and how will we help somebody who has already received a consequence? Those questions do not slow care. They prevent a quick decision from becoming a routine that is hard to undo.
Official precision must answer to lived experience#
At the close of the review, the school may retain the system if it meets better conditions: mandatory review for sensitive changes, a screen showing the source of each instruction, periodic tests through different routes, and a person with time to resolve discrepancies. It may also compare whether a less automated solution protects the goal better. The decision need not defend a technological ideology. It must answer evidence about which method cares for children and respects families.
Ninety-seven percent may remain in a report, but it should no longer occupy the whole picture. Beside it should be the stories revealing what the measure did not count, the cases reviewed, and the time taken for repair. A mature institution does not use a statistic to silence an experience. It uses experience to improve what it measures, what it asks, and what it is willing to stop. Dignity is not an acceptable margin of error.
Lucía does not need every meal service to abandon its tools. She needs efficiency not to turn her child’s safety into a statistical rarity. When an organization can see who stands behind a figure, offer a human exit, and correct without demanding heroism, artificial intelligence stops hiding harm behind official precision. It becomes help that accepts limits and answers for them.
