A well-told robotic failure is not a defeat for innovation but a source of knowledge for better care. An accurate account makes it possible to understand what happened, who was exposed, which controls failed, and how repair should proceed. Concealing it to preserve an appearance of perfection destroys trust and enables repetition. Dignity requires listening to the affected person, safety requires learning about the system, and repair requires turning explanation into verifiable change.

The appearance of perfection prevents learning#

Robots intended for care are frequently introduced through smooth demonstrations. This image can create expectations of complete reliability even though every physical and digital system has limits. When a failure appears, the team may minimise it, call it an anomaly, or attribute it to the user in order to protect the project. The organisation keeps a favourable story for a few days but loses the information needed to prevent the next incident.

Robots intended for care are frequently introduced through smooth demonstrations.

Trust does not come from claiming that nothing goes wrong. It grows when capabilities and limits are communicated, responses are timely, and problems demonstrably produce change. A person understands that a machine can fail; the relationship deteriorates when they discover that an institution knew, failed to warn them, or denied their experience. Transparency should be proportionate and protect privacy, but it cannot become strategic silence.

A system recording only serious harm learns too late. Near misses, human interventions, stops, confusion, and refusals contain early signals. Collecting them does not require treating every imperfection as a catastrophe. It means distinguishing severity, frequency, exposure, and ability to recover. Classification guides priority without erasing smaller events that form a pattern when combined.

What it means to tell an error well#

A useful report separates facts, interpretations, and uncertainty. It describes task, place, version, configuration, environmental conditions, sequence, signals, and actions. It states what each person expected and what actually happened. Physical, emotional, economic, and relational consequences are included, along with lost opportunities. Blaming language is avoided, while a person's own words are retained where relevant and authorised.

Context prevents misleading conclusions. Saying a robot stopped because of an obstacle does not explain whether the obstacle was foreseeable, the sensor should have detected it, or the route changed without a map update. Referring to user error does not establish whether instructions were understandable, the interface accessible, or training sufficient. Investigation examines conditions enabling the event rather than searching only for the last human action.

Technical traceability helps but has limits. Software logs, video, audio, and telemetry can reconstruct a sequence while invading privacy if collected without restraint. Before deployment, necessary data, access, retention, and protections are defined. An incident does not authorise unlimited surveillance. Where records are incomplete, uncertainty is stated instead of replaced by invented certainty.

Dignity for the affected person#

The first obligation is to attend to the person rather than secure the machine. Risk is stopped, human care provided, privacy preserved, and known facts explained. Nobody should have to repeat their story to multiple departments. One coordinator gathers essential information and maintains contact through the chosen channel. The person can receive support and decide how far they wish to participate in investigation.

Institutional communication should not reduce someone to a case or use their experience as promotional material. If the incident is reported, anonymity should be effective and consultation offered where recognition is possible. Records attributing unfounded inappropriate behaviour are corrected. Dignity also means recognising fear, anger, and distrust as legitimate reactions rather than obstacles to analysis.

A useful apology names harm and responsibility without defensive wording. It does not demand forgiveness or promise that nothing similar can recur. Immediate measures, next steps, deadlines, and routes for review are explained. Rights to complaint or compensation are communicated accessibly. Material remedy does not replace acknowledgement, and acknowledgement does not replace correction.

Safety through a systems perspective#

Errors rarely belong to one component. A sensor may fail alongside poor lighting, delayed maintenance, an ignored alert, and inadequate supervision. Investigation follows connections among design, data, environment, people, procedures, procurement, and leadership. This avoids superficial solutions such as retraining a user where the problem lies in an interface or organisational incentive.

A root cause is not always a single root. A network of necessary and contributing conditions may exist. Analysis identifies missing controls, existing controls that failed, and signals that did not reach someone able to act. It also asks what prevented greater harm. Recognising effective defences allows reinforcement rather than only listing failures. Conclusions express confidence levels and unresolved issues.

Psychological safety is part of operational safety. Workers, users, and families need channels protected from retaliation. A person stopping a device for reasonable concern should not be punished for reduced throughput. Anonymous alerts may help but require visible follow-up. If reporting produces no response, the system manufactures silence. Accountable teams communicate receipt, assessment, decision, and change.

Case 1: a dispenser released the wrong dose#

A home medication robot releases an earlier dose after a new prescription fails to synchronise. The user notices that the package colour is different and does not take it. The supplier records a minor interruption because no clinical harm occurred. Exposure, anxiety, and an unanswered call nevertheless took place. Classifying only the outcome would erase a near miss with potentially high severity.

Investigation reconstructs timing, connection, confirmations, and escalation. The interface displayed completion before validating the update with the server. The function is suspended, other people are contacted, records corrected, and support offered. Redesign requires dual confirmation, visible status, a constantly available human channel, and a manual procedure. The user decides whether to continue. Her attention prevented harm but does not become a compulsory system control.

Case 2: a transfer robot and delayed stopping#

In residential care, a robot assists a transfer from bed. During a turn, the sling shifts and a care assistant presses stop. The device takes two seconds longer than expected, causing no injury. The first report cites incorrect placement and recommends repeated training. The assistant insists that the button had little tactile feedback and the acoustic signal was inaudible in surrounding noise.

Independent testing reproduces the delay under a particular load and confirms interface problems. Slings, maintenance, button, sound, staffing, and usage criteria are reviewed. The device remains unavailable until corrected. The person being transferred receives an explanation and can choose another method. The assistant participates without blame. The near miss creates changes at other sites before an injury, showing the value of hearing a minority account.

Case 3: a private conversation was stored#

A social robot talks with residents and sends audio fragments to improve recognition. One person shares intimate information believing that the exchange stays in the room. A relative finds transcripts on a portal and complains. Technically, processing appeared in extensive conditions accepted at installation. Ethically, the expectation of privacy and absence of understandable choice reveal a serious failure.

The service stops transmission, restricts access, and requests deletion by the supplier. Affected people are informed, possible consequences reviewed, and non-recording alternatives offered. Default settings now minimise data; any secondary use needs specific consent. Explanations are redesigned in plain language with visible signals. The incident is reported in aggregate to governance. Repair acknowledges that a formal signature did not ensure understanding.

From investigation to repair#

Individual repair addresses harm, expense, time, opportunity, and trust. It may include clinical attention, reimbursement, restored services, corrected documentation, apology, and independent support. The desired outcome is discussed without claiming that every consequence can be reversed. The organisation does not make remedy conditional upon silence or participation in publicity.

Systemic repair changes the environment. Design, testing, maintenance, training, staffing, contracts, and supervision are updated. Every action receives an owner, deadline, and evidence of closure. Temporary withdrawal may be necessary, with human continuity maintained. Restart requires verification under realistic conditions and communication of limits. If risk cannot be reduced sufficiently, the function is abandoned despite high investment.

Learning is shared without exposing identities. Related teams receive practical alerts, scenarios, and changes. Suppliers inform other clients where appropriate. Regulators receive required notifications. Aggregate publication strengthens common knowledge and permits scrutiny. Transparency does not mean releasing every detail; it prevents each organisation from repeating the same harm in isolation.

Indicators that encourage honesty#

Measuring only availability, tasks, or satisfaction creates incentives not to record failure. Dashboards should include incidents, near misses, human interventions, detection time, recovery, recurrence, people affected, and completion of repair. An initial increase in reports may show greater confidence rather than worse safety. Interpretation combines figures with context and avoids competitive rankings that reward concealment.

Denominators matter. Ten failures in one hundred uses do not mean the same as ten in one hundred thousand, but severity does not vanish at a low rate. Information can be broken down by task, environment, and population while protecting privacy. Differences may reveal that a sensor works less well with particular bodies, assistive devices, or communication methods. A broad average must not silence a concentrated pattern.

Action effectiveness needs later verification. Administrative closure of a recommendation does not demonstrate lower risk. Tests are repeated, uses observed, people consulted, and recurrence monitored. Controls may introduce new problems. Review dates remain even if no incident appears. The absence of events is interpreted alongside exposure and confidence in reporting.

Learning without making blame the method#

Avoiding blame does not eliminate accountability. Deliberately unsafe conduct requires a response, but most human errors occur within designed conditions. A just approach distinguishes error, a risk-taking decision, and reckless conduct. This distinction encourages honest accounts and proportionate action. Automatic punishment drives evidence out of the system; ignoring responsibility also destroys trust.

Leadership answers for priorities, resources, and culture. Suppliers answer for products, information, and support. Professionals answer within their training and authority. Users do not carry quality assurance. A clear matrix prevents everyone pointing elsewhere afterwards. Contracts cannot erase the ethical or legal duties of the organisation providing care.

Training uses cases and decisions rather than instructions alone. It explains how to recognise signals, stop, protect a person, preserve evidence, and escalate. Accessible simulations occur under realistic schedules. Participants then identify confusion and procedures are corrected. Competence is maintained through updates because both robot and environment change.

Institutional sources for structured learning#

The World Health Organization addresses ethics and governance of artificial intelligence for health, including safety and accountability. The UNESCO Recommendation on the Ethics of Artificial Intelligence offers a framework concerning rights, transparency, and oversight. Both help place failure investigation within human obligations.

The NIST AI Risk Management Framework organises governance, identification, measurement, and continuous management. The United States Food and Drug Administration provides institutional information about reporting medical-device problems. These sources apply according to context, device classification, and local rules; they do not replace specialist advice.

A useful robot also shows its limits#

A robot that knows its place does not pretend to be infallible. It communicates uncertainty, records what is needed, accepts a stop, and refers to a person when it exceeds its capability. The organisation using it does not pretend to perfection either. It protects reporters, attends to harmed people, and allows evidence to alter decisions.

Telling an error well turns a painful experience into knowledge without instrumentalising the person who lived it. Dignity guides treatment, safety widens analysis, and repair connects words to change. Trust depends not on an empty incident history but on reliable, verifiable response. Robotics can then contribute to care from a modest position: useful when it works, transparent when uncertain, and accountable when it fails.