“Our artificial intelligence removes bias, finds hidden talent, and enables data-driven decisions.” It is an attractive promise for any employer and a particularly dangerous one when it affects recruitment, shifts, productivity, promotion, or dismissal. This article provides a method for reading it carefully: require every claimed advantage to become a testable statement, identify who bears the cost of errors, and preserve human decision-making backed by real information, time, and authority.
A promise is not yet evidence#
Imagine a recruitment company demonstrating a tool that analyses CVs and recorded interviews. It says the product saves time, improves objectivity, and discovers applicants a human team would overlook. None of those claims is absurd, but none proves that the product works for this job, this population, and these conditions. “Objective” may simply mean that the same formula is applied consistently. A uniform formula can consistently reproduce historical inequality.
Imagine a recruitment company demonstrating a tool that analyses CVs and recorded interviews.
The first task is to translate sales language into measurable questions. If the supplier promises better hiring, it must define better: retention, observed performance, safety, team satisfaction, or diversity. If it promises a thirty per cent time saving, the calculation must include error reviews, appeals, disability accommodations, and supervision work that did not previously exist. A saving measured only during a smooth demonstration does not describe the system’s full cost.
The comparison matters as much as the headline number. A vendor may advertise ninety per cent accuracy without describing the task, test population, or seriousness of the remaining ten per cent. For a training recommendation, an error may be repairable. When screening an applicant before interview, the same rate may close hundreds of opportunities. A metric must be tied to a specific consequence and compared with the current process, not with an imaginary version of flawless human judgement.
Six claims that must be turned into evidence#
“It removes bias.” A system does not prove neutrality merely because it excludes explicit fields for sex, age, or ethnicity. A postcode, employment gap, school, speech pattern, or device can operate as a proxy. The employer should require results separated by relevant groups, state which inequality the product is intended to reduce, and examine false positives and false negatives. A single average can conceal excellent performance for a majority and severe failure for a smaller group.
“It is explainable.” A screen displaying a score and three keywords does not necessarily explain a result. The affected person needs to know which relevant facts were used, how they mattered, and what information can be corrected. The decision-maker needs to understand limitations, uncertainty, and situations in which the recommendation should not be followed. A useful explanation enables action. A decorative explanation merely makes an opaque conclusion appear friendlier.
“A human remains in control.” Placing somebody at the end of a workflow to click accept is insufficient. That person must see the underlying evidence, have enough time, be permitted to depart from the recommendation without penalty, and record the reason. Measure how often outcomes change. If reviewers never disagree, the system might be perfect, but routine approval, performance pressure, or an interface that presents a score as a verdict are more plausible explanations.
“It improves over time.” Learning does not automatically mean improving. A model may adapt to earlier decisions and reinforce exactly the pattern it was meant to correct. Documentation should identify which new data enter the model, who labels outcomes, when a version changes, and how the organisation checks that another group has not been harmed. Every material update needs testing before deployment and a practical route back to the earlier version.
“It is compliant.” A vendor certificate does not replace the duties of the organisation choosing to deploy the tool. The employer must understand purpose, lawful basis, information notices, retention, subcontractors, transfers, security, and rights. Furthermore, a processing operation that satisfies one data-protection requirement does not by itself resolve equality law, employment duties, collective consultation, or the specific requirements applying to artificial-intelligence systems.
“The data are anonymised.” In employment it is difficult to separate a record from an individual when it includes a shift, location, voice, career history, or distinctive combination of tasks. Ask which identifiers were removed, who retains any matching table, whether other records could re-identify a person, and how long information remains. “Anonymised” cannot serve as a magic word that makes limits and accountability disappear.
Run a small test before handing over power#
A pilot should begin with a reversible decision and a sufficiently varied group. Record the baseline before activation: time, errors, abandonment, complaints, and differences between groups. Then define which result would justify continuation and which signal would require a stop. The definition of success cannot change after results arrive. Evaluation should include the people operating the process and those who would bear the consequence of a mistaken classification.
Create test cases the supplier did not choose: a CV with a caring-related career gap, a person using assistive technology, a minority accent, a poor connection, two equivalent profiles written differently, and one deliberately incorrect record. The aim is not to trick the system. It is to discover which assumptions turn an irrelevant difference into a disadvantage. Results, investigations, and corrective actions should be documented rather than discussed only in meetings.
The team should rehearse the appeal as well. A person requests an explanation, supplies a correction, and asks for the consequence to be paused. Who responds? Within how many days? Can the person see evidence, or do they receive a standard sentence? Does the correction reach every system to which the record was copied? An appeal route that works only during an announced audit protects nobody during ordinary operations.
The contract must tell the story of the worst day#
Sales terms normally describe an ideal day. A responsible contract anticipates defective data, an outage, an unannounced model change, a security incident, a union investigation, an applicant’s complaint, and the supplier’s closure. It should guarantee access to logs, advance notice of changes, assistance with rights requests, restrictions on reuse, independent audit, and verifiable return or deletion at termination. Without a workable exit, the employer becomes trapped precisely when it discovers harm.
Responsibilities need names rather than passive verbs. An internal owner approves the purpose and affected population; another controls data protection and security; employment specialists review consequences; worker representatives participate when required; and a named person with authority can suspend the system. The supplier can explain its product, but it cannot decide what risk is acceptable for the workforce. That judgement belongs to the organisation exercising power over work.
What an affected person can ask#
An applicant or worker does not need to be an engineer to ask for clarity. A focused request can ask what decision was made, whether an automated recommendation contributed, which data were used and where they came from, which main factors influenced the outcome, who reviewed it, and how inaccurate information can be corrected. Keep the vacancy notice, communications, screenshots, and relevant personal documents. A concrete request is often more effective than demanding “the whole algorithm”.
When the consequence is serious—exclusion from recruitment, loss of shifts, discipline, or dismissal—ask for it to be paused during review. The review should rely on evidence independent of the disputed score. Depending on the case, a trade-union representative, data-protection officer, labour inspectorate, regulator, or qualified adviser may help. This article offers general education and cannot replace individual legal advice.
European rules already require more than a slogan#
The European Union Artificial Intelligence Act classifies certain systems intended for employment and worker management as high risk, including some used to recruit, allocate tasks, evaluate, promote, or terminate employment relationships. Classification brings requirements concerning risk management, documentation, logging, data quality, transparency, and human oversight. The primary source is the EU Artificial Intelligence Act, Regulation (EU) 2024/1689, which should be read with its application dates and the duties assigned to each actor.
For personal data, the General Data Protection Regulation establishes purpose limitation, minimisation, accuracy, and accountability, as well as safeguards for certain solely automated decisions with legal or similarly significant effects. The Spanish Data Protection Agency’s official guidance collection helps teams turn principles into controls. For platform work, Directive (EU) 2024/2831 provides specific rules concerning algorithmic management.
A family conversation about a first job#
Many young people will encounter these tools in their first application, internship, warehouse, delivery platform, or contact centre. Families can teach them to distinguish a score from a fact, check permissions before recording an interview, protect passwords, keep important messages, and request a specific explanation without treating a challenge as misconduct. Avoid the false reassurance that “if you have nothing to hide, it cannot affect you”: correct data can still receive the wrong interpretation.
Reading a promise carefully does not mean rejecting every technology. A system may widen access, detect inconsistencies, or reduce administrative work. The question is whether the organisation can demonstrate the benefit, recognise who is harmed, correct an error, and withdraw the product when it fails. Before buying “better, objective decisions”, demand a precise purpose, an honest comparison, group-level testing, a meaningful appeal, and a contractual exit. If one is missing, the promise is not ready to decide anybody’s working life.
