Important change does not always arrive as a robot walking through the door. Sometimes it appears as an app that proposes the rota, a camera that calculates timings, a questionnaire that rejects applicants or a dashboard that turns every task into a score. To somebody starting work, it may look like an unavoidable part of modern employment. Yet whenever a system recommends, ranks or monitors, an organisation remains responsible for deciding why it is used, which data it consumes and what happens when it is wrong.

Imagine Alex, nineteen, in a first warehouse job. The app allocates routes, records breaks and compares units per hour. One Friday, Alex receives fewer shifts. The supervisor says that «the system decides» and cannot explain which fact mattered. Alex fears that asking questions will create a bad impression. At home, nobody knows whether this is merely a planning tool or a decision that should be open to challenge. That uncertainty is precisely the ground on which algorithmic management expands without debate.

What it means for a system to decide at work#

A system does not have to sign a dismissal letter to exercise decisive influence. It can rank résumés, predict who will leave, recommend pay, distribute orders, set targets, trigger fraud alerts or select the calls that a supervisor reviews. A person may formally make the decision, but human involvement is ornamental if that person always accepts the recommendation, does not understand the model or has only seconds to review it.

A system does not have to sign a dismissal letter to exercise decisive influence.

It helps to distinguish automation from decision-making. Automating delivery of a payslip calculated under known rules is not the same as assigning every employee a «commitment probability». The first task can be checked through visible calculations; the second interprets behaviour through variables whose relationship to performance is contestable. The more an outcome affects access, income, reputation, health or continuity of employment, the stronger the explanation and oversight must be.

The origin of data matters too. A system may combine clock-ins, location, keyboard speed, sales, incidents, appraisals, absences or answers to personality tests. Possessing those data does not make every reuse legitimate. A measurement collected for safety should not silently become a disciplinary signal; a disability accommodation must not become evidence of low productivity; a protected absence should not feed a prediction that somebody is likely to quit.

Why new workers are particularly exposed#

Young people frequently enter through temporary jobs, platforms, internships or high-volume recruitment, which are exactly the settings most likely to be automated. They have fewer reference points for distinguishing a routine practice from an abusive one and may fear looking difficult. A polished interface conveys authority: if a screen displays 62 out of 100, the number seems to describe an objective reality. But every score embeds human choices about what to measure, whom to compare, how much each variable matters and which mistakes are tolerable.

Families can help without turning the conversation into an interrogation. Instead of asking only «How was work?», try: «Who organises your shifts?», «Can you correct a wrong record?», «Have they explained what the app measures?» or «Is there a person who reviews a sanction?». The aim is not to cause alarm but to provide vocabulary. Being able to name a score, alert or ranking makes it possible to report the problem precisely.

Some signals deserve attention: schedules changing without explanation, targets rising automatically, penalties for necessary breaks, public leaderboards, prompts designed to keep someone logged in, instantaneous rejection or answers such as «nobody can change it». No single sign proves unlawfulness. It does indicate a need to understand and document the process and to find out which safeguards exist.

The false neutrality of numbers#

A model learns from historical data or implements rules designed by people. If the past contained inequality, the pattern can reproduce it. If success means accepting every shift, people with caring duties receive lower rankings. If productivity is measured through keystrokes, time spent thinking, listening or supporting is penalised. If a filter learns from the profile of past executives, it may favour traits resembling those who already held power.

Indicators also change behaviour. Staff assessed on short calls will tend to end them early even if a customer needs more help. A courier measured by speed may take risks. A nurse scored on the number of contacts may have less time to observe. The problem is not only that the metric may be inaccurate: it can encourage people to perform the very job it claims to improve less well.

Asking about an accuracy rate is not enough. Ask who bears the errors. A false positive in a music recommendation is inconvenient; a false positive labelling somebody fraudulent can block their income. Differences between groups, unusual conditions and changes of context must also be examined. A model tested in offices may fail during remote work, and a test designed in one language may not assess fairly somebody who learned it later.

Rights and limits that already exist#

In the European Union, the General Data Protection Regulation provides safeguards concerning decisions based solely on automated processing that produce legal effects or similarly significantly affect a person. Article 22 does not make every calculation prohibited, but it requires analysis of whether there is a consequential decision, whether it is solely automated, and which exception and safeguards apply. The primary text is available from EUR-Lex: Regulation (EU) 2016/679.

The European Union's artificial intelligence legislation takes a risk-based approach. It treats certain systems used in employment, worker management and access to self-employment as high risk, including uses for recruitment, task allocation based on traits or behaviour, and performance assessment. That brings duties for those who develop and deploy systems, although the precise application depends on timing and circumstances. The primary source is EUR-Lex: Artificial Intelligence Regulation (EU) 2024/1689.

For platform work, European Directive 2024/2831 specifically addresses transparency, human oversight and review of decisions in algorithmic management. It also restricts certain particularly intrusive processing of data. Directives require implementation in national law, so workers should check current rules and seek union or professional advice where necessary. The official text is EUR-Lex: Directive (EU) 2024/2831.

These texts are a starting point, not an automatic answer. Employment law, collective bargaining, equality and non-discrimination rules, occupational safety, and national data protection rules can all be relevant. In a case involving a sanction, dismissal, discrimination or serious harm, this guide cannot replace advice from a union, worker representative, lawyer, labour inspectorate or data protection authority.

Questions that turn a black box into a contestable process#

Before accepting «the algorithm says so», request a plain description: what decision was made, which system contributed and who retains authority to change it. Then ask which categories of data were used, where they came from and whether they were current. It is not necessary to demand source code to obtain useful information. An operational explanation should reveal the main factors and allow a particular error to be identified.

The declared objective also matters. «Improving efficiency» is too broad. Was the aim to reduce waiting, predict staffing needs or sanction low performance? A tool bought for scheduling may later be used for appraisal without fresh checks. Asking about purpose exposes this function creep and supports a demand for assessment proportionate to the new impact.

Human review must be genuine. The reviewer needs competence, time, sufficient information and permission to depart from the recommendation. If the person's role is only to press «confirm», no effective safeguard exists. It is reasonable to ask how often decisions change after challenge, what deadline applies and how an outcome is communicated. A channel that never changes anything may be an empty formality.

Finally, ask about expiry. Models, rules and data grow stale. When was the system last tested? What happens when a role changes? How long does an alert remain active? Can an old incident still reduce shifts months later? Responsible decision-making requires periodic review, traceability and criteria for withdrawing a system when it causes harm.

How to document a case without creating new risks#

When an outcome appears unfair, the first step is a factual chronology: date, decision received, screen or message, explanation requested, answer and tangible effect. Separate observations from theories. «On Tuesday my allocation fell from four shifts to one after a punctuality alert» is more useful than «the algorithm hates me». Keeping contracts, policies, payslips and communications helps, provided business secrets and other people's personal information are respected.

Do not extract databases, make prohibited recordings or remove confidential information to prove a problem. Record what was observed and use lawful access or complaint channels. If a screenshot reveals information about colleagues or customers, avoid capturing it or seek advice. Documentation should not expose the worker to a separate breach.

Next, make a concrete request: correct a record, provide the principal criteria, obtain human review, pause a penalty during investigation or identify the appeal route. A short request often works better than a general manifesto. If several people experience the same pattern, a worker representative or union may analyse it collectively without leaving one individual alone against the system.

What an employer should do before deployment#

A responsible organisation begins by demonstrating necessity. It compares the tool with less intrusive alternatives and identifies decisions that will never be delegated. It then maps everybody affected, including rejected candidates who never become employees, temporary workers and people with accessibility needs. Assessment must cover more than average accuracy: severe errors, discrimination, privacy, mental load and safety also matter.

Early worker participation improves design. People doing the job know exceptions that a supplier misses: a pause may reflect an incident, a longer route may be safer, and a lengthy conversation may prevent a complaint. Incorporating that knowledge is not resistance to innovation; it tests whether the metric represents real work.

The employer should provide understandable information before use, train managers and reviewers, restrict access, log changes and establish a challenge mechanism. It also needs an incident plan: who stops decisions, how people are notified, how lost income is restored and how consequential decisions are corrected. Purchasing technology from a vendor does not transfer responsibility for workplace consequences.

A useful family conversation#

When a young worker says an app controls the working day, the family response should be neither «put up with it, every job is like that» nor «report them immediately». First listen to what happened and what outcome the person wants. They may need to understand a rota, recover a shift or prepare a conversation with a manager. Asking about the desired outcome restores agency.

Rehearse a sentence together: «I noticed that my score fell after this incident. Could you confirm the data used, explain how it affects shift allocation and arrange for a person to review the case?». A calm tone does not surrender rights. If the answer is vague, ask for it in writing. If there is retaliation, danger, discrimination or significant loss, seek specialist support early.

Pay attention to emotional effects too. Continuous scoring can create hypervigilance, shame and a sense of worthlessness. A workplace ranking does not measure a person's value. Sleep, conversation with a trusted person and separating measured performance from identity will not resolve the dispute, but they help somebody address it without being consumed by it.

A checklist before accepting an automated outcome#

  • What specific decision was made, and how does it affect employment, pay, shifts or reputation?
  • Did the system decide alone, make a recommendation or contribute one fact among several?
  • Which main data were used, and is there a verifiable error?
  • Who can review the outcome with genuine authority to change it?
  • How does an appeal work, what is the deadline and what happens during review?
  • Were workers informed and their representatives consulted where required?
  • Are there signs of unequal impact, unsafe pressure or disproportionate surveillance?
  • Has a lawful copy of the communication and response been retained?

The silent change stops being inevitable when the process becomes visible. This is not an argument against every algorithm: a good tool can distribute workload, detect hazards or improve accessibility. The dividing line is whether it enlarges human capability or conceals power behind a score. The future of work is not decided by the system alone; it is also shaped by the questions, safeguards and limits that people and institutions demand today.