A company may decide to buy software that screens applicants, allocates shifts or measures productivity. The difficult question comes next: which decisions about working life may legitimately be delegated, and under what conditions? The answer does not belong only to management, IT or the system vendor. Pay, health, privacy and opportunity for real people are at stake. Those who bear the risk, worker representatives and publicly defensible criteria must therefore be part of the conversation.
Consider Nora, twenty, who works for a retail chain. Software forecasts demand and publishes her shifts two days in advance. The company calls that efficiency; for Nora it means not knowing whether she can attend class or care for her brother. When she asks, her manager says that the algorithm uses thousands of data points and nobody can change it. This is not merely a technical problem. Somebody decided that cutting costs mattered more than predictable working time, even though that preference has disappeared inside a tool.
The invisible decision before the algorithm#
No system discovers by itself what good work means. Before calculation begins, an organisation selects an objective: sell more, shorten waiting times, prevent accidents, cover shifts or retain customers. It then chooses indicators that partly stand in for that objective. Calls handled may represent service; parcels per hour may represent productivity; instant availability may represent commitment. Every substitution loses context. A long conversation may resolve a difficult case, a pause may prevent an injury, and a person who does not answer at night may still fulfil the contract perfectly. What can be counted is not automatically what matters.
No system discovers by itself what good work means. Before calculation begins, an organisation selects an objective: sell more, shorten waiting times, prevent accidents, cover shifts or retain customers.
The right question is therefore not simply whether an algorithm is accurate on average. We must ask who defined success, which errors are regarded as acceptable and who carries their consequences. A system that improves scheduling by two per cent may look excellent in a presentation but be unfair if it concentrates unpredictable hours among people with caring duties. The business understands its operation, but it cannot understand every burden it creates by itself. It needs situated worker experience and must acknowledge that efficiency, dignity and safety can conflict.
Algorithmic management covers much more than a fully automated decision. A tool may recommend whom to interview, rank a list later viewed by a supervisor, allocate tasks, calculate incentives, detect supposed anomalies or generate a risk score. A human clicking accept does not guarantee meaningful review. If that person lacks time, information, authority or willingness to disagree, the person is merely a rubber stamp. The entire process matters: input data, inferences, recommendation, final decision, route for challenge and cumulative effects.
Four voices needed before deployment#
Management must explain the problem it wants to solve and retain final responsibility. The technical team must describe limitations, data and circumstances in which the model becomes unreliable. Workers contribute situations that rarely appear in a database: informal tasks, accessibility barriers, invisible cooperation, physical hazards and differences between a quiet shift and an overwhelmed one. Worker representatives can identify collective effects, compare departments and negotiate safeguards. When one of those voices is missing, the assessment remains incomplete however sophisticated the software may be.
The vendor must answer questions too, but cannot become an excuse. Saying that the model is a trade secret does not explain which variables the employer uses, how outcomes are tested or who handles a challenge. Nor is a generic hundred-page manual enough. Useful information enables a person to understand which behaviour affects an outcome, which data can be corrected and what concrete consequence may follow. If an organisation cannot obtain that explanation from its supplier, it may not be equipped to use the tool for consequential decisions.
In Spain, Article 64.4(d) of the Workers' Statute gives legal worker representatives a right to information about the parameters, rules and instructions underpinning algorithms or artificial-intelligence systems that influence working conditions and access to or continuation in employment, including profiling. The amendment can be read in the official publication from the Spanish state gazette. This does not mean every worker receives source code, but it contradicts the claim that everything must remain in an inaccessible black box.
A risk map that fits into one meeting#
Before deployment, a team can draw a simple table. One column lists every decision influenced by the tool: hiring, appraisal, scheduling, target setting, discipline or dismissal. Another identifies affected people and groups that might be disadvantaged. The team then records the data used, expected benefit, worst plausible harm, method of detecting that harm and person authorised to stop the process. This exercise replaces abstract phrases such as improving efficiency with observable effects that colleagues can question.
Ordinary and edge cases both need testing. What happens to someone who uses assistive technology, shares a device, has an unstable connection, changes their name, is pregnant, works in another language or needs an accommodation? How does the system interpret medical leave, an authorised break or time spent helping a colleague that reduces individual output? The point is not to imagine every possible life. It is to look deliberately for places where one uniform rule can produce unequal harm.
A red line must also be specified. A support system may propose a rota but should not publish it without checking rest periods, agreed availability and protected needs. It may flag a change in performance but should not translate that flag directly into discipline. It may rank applications for review but should not silently erase people whose career path is unusual. The greater the impact on income, continuity or health, the stronger the supervision, documentation and ability to suspend automation must be.
Questions a young worker can ask#
A person entering a first job does not need advanced mathematics to ask legitimate questions. They can ask: is any system used to assess my application or performance; which data does it take; where do they come from; how long are they kept; who sees the score; can it affect shifts, pay or continued employment; and which person has authority to review it? Asking calmly in writing is useful, as is retaining the job advertisement, contract, policies, screenshots and relevant messages.
When a harmful decision appears, describe facts instead of guessing motives. For example: on 4 May the app changed three shifts; the manager said the change resulted from a score; the score includes two absences that were authorised. Then request correction of the data, an explanation of the relevant criterion and review by someone able to change the result. A precise timeline is more effective than saying the algorithm hates me, and it helps representatives, a union or an adviser assess what happened.
Article 22 of the General Data Protection Regulation addresses decisions based solely on automated processing that produce legal or similarly significant effects. In specified circumstances its safeguards include human intervention, the ability to express one's point of view and the right to contest a decision; the official GDPR text is available through EUR-Lex. Application depends on the facts, and not every algorithmic recommendation falls within Article 22 automatically. Serious cases merit advice from a worker representative or suitable professional rather than a confident conclusion based on a short description.
Human review must be able to change something#
Review worthy of the name identifies an accountable person, shows that person the relevant data and reasoning, provides time to hear the affected worker and grants authority to alter the outcome without penalty. The reviewer should record what was checked and why the decision was confirmed or corrected. If the reviewer sees only a score, cannot discover how it arose or must accept it unless there is a software malfunction, the human involvement is decorative.
Making a challenge should not worsen the challenger's position. There should be an accessible channel, a reasonable response time and protection against retaliation. Larger organisations should study complaints in aggregate: ten similar corrections may reveal one design flaw, not ten exceptional workers. Internally reporting how many decisions were reviewed, how many changed and which improvements followed turns accountability into organisational learning.
What European regulation contributes#
The European Artificial Intelligence Act classifies certain employment uses as high risk, including recruitment and selection, decisions affecting terms of work, promotion or termination, task allocation based on traits or behaviour, and monitoring or evaluation of performance. Its precise scope and exceptions should be checked in Regulation (EU) 2024/1689. High risk does not mean automatically prohibited. It brings duties designed to control risk, document the system, monitor operation and preserve human oversight according to each actor's role.
Law is a floor rather than a complete conversation about the kind of work people want. A practice may satisfy formal requirements and still cause unbearable pace, isolation or loss of autonomy. Joint research by the International Labour Organization and the European Commission's Joint Research Centre on logistics and healthcare found operational gains alongside concerns about intrusive surveillance and deterioration in job quality. The report is available from the ILO's official research repository. Looking at both sides prevents technology from being framed as either inevitable salvation or inevitable threat.
A minimum agreement before saying yes#
A reasonable protocol can include a limited purpose; data inventory; a ban on reuse for new purposes without reassessment; tests for accuracy and unequal outcomes; prior consultation; training; meaningful human review; a correction channel; periodic audits; a deletion timetable; a change log and a suspension mechanism. It should name responsible people or roles, not an ambiguous appropriate department. Approval should also expire: an updated system or a new worker population requires fresh examination.
For a family, support does not mean taking over an adult child's complaint. It means helping to order dates, distinguish facts from suspicions, read documents and decide which outcome to request: correction of data, restored shifts, an explanation or a halted sanction. Health matters too. Constant monitoring can produce anxiety, permanent availability and fear of resting. Persistent symptoms, serious conflict or a risk of dismissal are sensible reasons to seek appropriate union, legal or healthcare support.
The question a company cannot answer alone is easy to state: how much risk are we entitled to impose on other people in exchange for this benefit? Answering it requires listening to those who understand the work from within, exposing hidden priorities and accepting limits. An algorithm can calculate a route or detect a pattern; it cannot decide by itself what treatment is dignified. That remains a human, collective and revisable responsibility.
