When an application decides who receives an interview, which shift each person gets, how long a task should take or who deserves a warning, this is not merely an IT upgrade. It redistributes power inside the workplace. For a family whose children are entering employment, the important question is not simply whether artificial intelligence will create or destroy jobs. It is what kind of conditions successful applicants will find: which data will be used to judge them, who will hear their account and how they can challenge an error.
Headlines often reduce the debate to a net number of jobs. That number matters, but it leaves out intensity, autonomy, pay, safety, learning and the ability to organise with other people. A warehouse can retain its entire workforce while degrading work if a system raises the pace every week without accounting for breakdowns, experience or fatigue. It can also improve a task if it distributes loads within clear limits and workers help design those limits. Technology does not determine the outcome by itself. Design choices, workplace rules and the genuine ability to challenge them do.
How to recognise when a system is acting as the manager#
Algorithmic management does not always carry an artificial-intelligence label. It may be a rostering program, a productivity score, a CV filter, a camera with analytics or an app that allocates deliveries. The International Labour Organization describes it as programmed procedures used to coordinate, allocate, monitor or assess work. Its portal on algorithmic management in the workplace identifies it in conventional organisations as well as digital platforms. If a system output influences hiring, pay, discipline, promotion or dismissal, it is participating in a consequential employment decision.
Algorithmic management does not always carry an artificial-intelligence label. It may be a rostering program, a productivity score, a CV filter, a camera with analytics or an app that allocates deliveries.
Draw the whole path. Which inputs enter: sales, location, keystrokes, calls, absences or customer ratings? What does the program create: a prediction, ranking or target? Who receives the result, and what do they do with it? The phrase “it only recommends” may conceal a great deal. If a supervisor lacks the time, information or permission to depart from a recommendation, the decision is automated in practice. Human oversight is not somebody clicking accept. It requires understanding limits, checking context and being able to say no without being penalised.
A score is never the whole person#
Indicators select one part of reality. Counting packages per hour may ignore fragile products, help given to a new colleague or faulty equipment. Measuring call minutes may punish the person who resolves difficult cases. Treating acceptance of extra shifts as commitment disadvantages somebody who cares for a relative. Even an accurate observation can support an unfair conclusion when context is missing. Before accepting a metric, ask which behaviour it rewards, which valuable work it makes invisible and who is most likely to be treated as an exception.
There are also derived data: labels such as “attrition risk”, “low potential” or “probability of absence”. These are inferences, not observed facts. They may learn from histories that contain earlier discrimination or rely on variables that proxy for age, origin, disability or family circumstances. Auditing the average is insufficient. False positives and negatives should be compared across groups, individual cases examined and cumulative effects tested. A mistaken classification can withhold training; missing that training then lowers a future score, creating a loop that appears to confirm the first prediction.
Rights and rules in the European framework#
Article 22 of the General Data Protection Regulation provides protection in relation to certain decisions based solely on automated processing that produce legal or similarly significant effects. Depending on the situation and legal basis, safeguards include human intervention, an opportunity to express one’s view and a way to contest the decision. This does not mean that every calculation is prohibited or that every workplace situation has an identical remedy. It does mean that an important decision cannot responsibly be treated as a sealed box that a person must simply obey.
The European Artificial Intelligence Act lists in Annex III certain systems used for recruitment and selection, decisions affecting work-related conditions, promotion or termination, task allocation based on behaviour or characteristics, and monitoring or evaluation of performance. The official text is available through EUR-Lex. Classification and duties depend on the exact use and the applicable timetable, so this article is not individual legal advice. The stable practical lesson is to document purpose, data, accountable people, human review, risks and the route for complaints before deployment.
Occupational health is part of the decision#
A system can optimise every minute while removing the margins that let people recover, learn or help. When targets update from the results of the fastest workers, the benchmark may rise without an ergonomic assessment. When an app changes shifts at short notice, uncertainty is transferred to families. The European Agency for Safety and Health at Work warns in its factsheet on AI-based and algorithmic worker management about possible effects on autonomy, workload, stress and privacy, while stressing prevention and worker participation.
A risk assessment must observe real work rather than technical operation alone. It should record workload peaks, missed breaks, roster changes, injuries, conflict, fear of disconnecting and time spent correcting the system. Temporary, contracted and remote workers and people with disabilities must be included because they may experience different pathways. A reduction in accidents may be valuable, but it does not justify unlimited monitoring. The right question is which data is necessary to prevent a specific risk, how long it is retained and whether a less intrusive alternative could achieve the same purpose.
An everyday case: the warning nobody can explain#
Imagine a young worker receives a warning for “low reliability”. Her manager can see a score but not its factors. She recalls two delays caused by a broken machine and several rejected shift changes because she cares for her brother. A fair response begins by suspending consequences, preserving evidence and naming someone with authority to review. Data, time periods, comparison groups and rules are then checked; context is heard; the decision is corrected; and, if the defect can recur, other affected cases are examined rather than leaving everyone else exposed.
A useful explanation need not disclose protected source code, but it must make the decision understandable and contestable: its purpose, relevant data, assessed period, approximate importance of factors, intended consequence, accountable person and appeal deadline. “The algorithm says so” is not an explanation. Referring the worker to a powerless vendor is not a remedy. If an employer cannot reconstruct why a warning arose, it should not use that warning for discipline. Recording human overrides also reveals defective rules. Disagreement is not noise to suppress; it is information needed to govern the system.
Questions a young person can ask before accepting#
An applicant may not be able to negotiate everything, but they can observe. Is an automated test used for selection? Is there video or location monitoring? How are targets and shifts set? Can a person review a score? Is there a works council, union representative, data protection lead or safety channel? A clear, proportionate written answer inspires more confidence than a general promise of objectivity. No information at all, pressure to consent or a requirement to install a work app on a personal phone are signals that justify seeking independent advice.
Families can rehearse these questions without speaking for the young person or creating panic. It is sensible to retain the job advert, contract, policies, roster messages and communications about assessment. If a problem occurs, a short chronology is more useful than a hundred disordered screenshots: date, decision, requested explanation, response and concrete effect. Nobody should extract secrets or bypass a system; they document their own experience. Depending on the issue, a worker representative, labour inspectorate, prevention service, data protection authority or qualified adviser may help. The proper route depends on the country and facts.
How a responsible employer should buy and deploy#
Before procurement, define the problem without naming a product. Then compare alternatives, including not automating. A pilot needs scope, duration, success criteria and stop conditions. Savings alone are inadequate: assess quality, errors, total time, health, autonomy, complaints and the distribution of benefits. Test difficult cases and incomplete data. If a vendor claims its model is unbiased, it should explain the evaluation, population and limitations. A sales demonstration cannot substitute for a trial in the context where a decision will affect real people, nor can a high accuracy figure reveal every harmful error.
The contract should enable audits, change logs, incident notification, subcontractor control, export and deletion. It also needs an exit plan: how to return to the former process without losing pay records, rosters or case files. Worker representatives should participate before the design is locked, rather than when only interface training remains. International research on logistics and healthcare shows that algorithmic management is already present in ordinary workplaces and should be examined alongside work organisation, job quality and industrial relations.
A dashboard that measures more than output#
Every review should answer five questions. Does the system meet its declared purpose? Which groups receive benefits and costs? Which errors are challenged, and how many are corrected? How do health, autonomy and learning change? Is every collected data item still necessary? Combine numbers with confidential interviews and observation. A stable average may conceal an exhausted shift or a job category denied training. Measure added labour too: minutes spent repairing rosters, documenting exceptions or feeding data. Automating one task saves nothing when it creates another task that nobody counts.
Thresholds should be set before results are seen. Examples include no sanction without human checking; a maximum appeal time; pausing after a pattern of errors; quarterly comparison of outcomes between groups; and deletion when the purpose expires. Each control needs an owner, evidence and authority. A committee that merely advises while another department retains every incentive to push output does not govern. Internally publishing decisions, limits and incidents creates organisational memory and allows learning when the vendor, leadership or model changes.
The conversation that deserves a headline#
Asking how many jobs will disappear is attractive because it produces a number. Governing work demands harder questions: who captures productivity gains, who carries uncertainty, which capabilities are developed and who can stop a mistaken instruction. A tool may remove repetitive work and improve safety if the benefit becomes training, rest or better service. It may instead intensify pace and weaken collective voice. The outcome must be evaluated in pay, schedules, bodies, opportunities and dignity, not solely in an aggregate balance sheet.
For a family, the conclusion is not to teach distrust of every technology. It is to prepare young people to recognise when a measure requires an explanation. For an employer, it is not to abandon innovation, but to accept that managing through data remains management and carries responsibility. For society, it means refusing to confuse employment with good employment. Whenever a system decides, somebody selected its objective, inputs and limits. Naming that person, listening to those who live with the consequences and preserving a genuine route out is the discussion missing from too many headlines.
