Marta received the email at 6:12 p.m., after filing a claim for a household accident. “Your request does not meet the applicable criteria,” it said. It did not name the criteria, attach the file, or offer a contact beyond a link to frequently asked questions. Marta called the number on her policy. The agent told her that an outside company performed the assessment and that she could neither see nor alter its score. The vendor, in turn, dealt only with the insurer. Between them, a denial affecting an urgent repair had no visible owner, readable explanation, or workable route back in.
That is not merely a messy handoff. It is what happens when an organization buys a decision tool as though it were also buying away the duty to answer for the tool’s effects. An insurer may outsource software, data processing, maintenance, or specialist support. It cannot outsource its relationship with the person whose coverage it denies. The insurer chooses to use the output, sets the contractual terms, and retains the power to stop using it. It therefore retains the duty to explain, reconsider, repair, and suspend the process when necessary.
A contract assigns work; it does not erase accountability#
There is nothing unusual about an insurer not building every system itself. A vendor may be well placed to sort documents, flag inconsistencies, or identify files that deserve closer attention. The trouble begins when a contract promises an “intelligent solution” or a “proprietary model” but never says who does what after a particular decision causes harm.
There is nothing unusual about an insurer not building every system itself. A vendor may be well placed to sort documents, flag inconsistencies, or identify files that deserve closer attention.
Before deploying a tool, the organization should be able to answer plain questions. What information does the vendor receive, for what purpose, and for how long? What recommendation comes back? Who has authority to confirm a denial? Which versions, documents, and logs will reconstruct the case six months later? Who will speak to Marta in language she can understand? If those answers are scattered through technical appendices and terms of service, the process is already unable to support an ordinary appeal.
The useful division is specific. The vendor must provide evidence about its product: the version used, fields involved, known limits, similar incidents, recent changes, and how records are retained. The insurer must account for the decision sent to the customer: why it applies to that policy, which condition it relies on, what information the customer can correct, and what interim remedy is available while review occurs. Technical detail without a changeable decision is hollow; a courteous letter without evidence cannot be tested.
A denial should create a route, not a wall#
An opaque message turns the first day of a problem into an obstacle course. A denial should instead contain five visible elements: the precise item refused, the date and scope of the decision, reasons in ordinary language, the evidence or data the person may submit, and a channel with a response deadline. Source code is not needed to say that an invoice is missing, a date conflicts, or a particular rule classified a file in a certain way. Opacity does not necessarily protect a trade secret; it often protects a workflow nobody has made ready for scrutiny.
A readable case file matters. Marta should be able to download the material considered, distinguish data supplied by her from data received from a third party and from data inferred by the system, and flag an error without guessing a technology company’s name. If the insurer cannot provide that record because the external product keeps no adequate trail, this is not a minor customer-service limitation. It is a reason not to use that product for decisions that restrict coverage.
Review must also be more than a request for patience. A form that says “we will look into your case” is not enough. It should say who will read it, what power that person has to reverse the outcome, when they will reply, and what happens if more information is required. With water damage, three weeks of silence can multiply the loss. The deadline is not a courtesy. It is part of making repair possible.
Suspension is not an admission of defeat#
Organizations often imagine only two states: the tool continues to run or an expensive contract is terminated. There is an essential option between them: a limited suspension. If a new release starts rejecting cases without adequate reasons, if one type of claim accumulates reversals, or if the vendor cannot reconstruct individual decisions, the insurer should be able to prevent further denials based on that output while it investigates.
That power should be agreed before an incident. The agreement can set observable triggers: a reversal rate, missing logs, unusual increases by area or product, missed response times, or discrepancies found by human reviewers. It should also say what happens during the pause. New files may go to a human queue under a clear instruction: no one should be put in a worse position because the system created uncertainty. Suspension protects people who have not yet received a decision and creates room to repair those who have.
Turning off a screen and announcing an audit is not enough. The insurer must identify affected decisions, notify people where possible, reopen files, speed up payments or temporary measures, and record the correction. If Marta had to pay for an essential repair herself because an erroneous denial had no usable review, remedy does not end when her claim status changes. It must consider the costs created by delay and how to restore the position she would have held without the error.
Test the agreement before it is needed#
The most honest way to discover empty accountability is a simulation. An insurer can construct a file with a wrong data point, ask the vendor for the decision history, and check what the affected person receives at every stage. Claims staff, customer service, procurement, and compliance should take part. If nobody knows who may suspend the workflow, if the record arrives days late, or if only legal staff can understand the letter, pressure will not fix the defect.
The review should measure more than technical accuracy. A useful dashboard compares initial and reversed denials, time to deliver the case file, days to a human answer, repair costs caused by delay, and cases that triggered suspension. Those figures need to be read alongside complete examples. Several reversals may reveal one broken integration, a badly translated rule, or a category that cannot accommodate ordinary circumstances. The point is not to build a statistical defense. It is to find where a person is being made to carry the cost of organizational uncertainty.
Marta does not need to know who wrote every line of software to seek redress with dignity. She needs the organization that sold her the policy to stand behind a decision made in its name. The practical next step is to attach an operational accountability sheet to every external-algorithm contract: internal decision owner, evidence the vendor must supply, response channel and deadline, suspension triggers, reopening protocol, and a method for compensating harm caused by error. When that sheet survives a realistic test file, the contract stops being an alibi and becomes a tool for care.
Procurement must rehearse accountability#
Before a tool affects a real customer, the organisation can run a repair drill just as it would run a security exercise. A staff member submits a fictional denial, asks what evidence informed it, and tests whether a named person can pause the effect while the case is examined. The point is not to expose a vendor's source code. It is to discover whether the buyer can offer a timely answer, correct a record and keep the person from bearing the cost of an unresolved handoff.
That drill clarifies the minimum information the supplier must make available: the deployed version, the relevant inputs, the decision date, the applicable threshold and the channel for a correction. These are operational facts, not a demand for total technical disclosure. When the buyer has them, it can explain a decision honestly and recognise when the evidence is insufficient. When it does not, procurement has created a system in which every participant can point elsewhere while the affected person waits.
