A nineteen-year-old courier receives fewer jobs for three weeks. He asks what is happening and hears a phrase that sounds final: “the system decides”. Nobody can identify a specific incident, explain which data mattered or name the person who can correct them. The official story promises efficiency, objectivity and opportunities allocated by merit; his daily reality is falling income with nobody to answer. The uncomfortable question is not whether the technology works on average. It is who chose what should be optimised, who receives the benefit when it works, and who answers when a person pays for an error.
This question matters to a family because a first job teaches rules that a young person may accept as natural for years. If they learn that a score cannot be challenged, they will confuse calculation with authority. If they learn to request facts, deadlines and human review with genuine power to change an outcome, they develop professional autonomy. The answer is not to distrust every tool. A system may usefully organise applications, forecast demand or identify hazards. The problem begins when an organisation uses a technical appearance to hide human choices about productivity, availability, risk and the distribution of power.
Neutrality does not come pre-installed#
Every management system begins with choices. Someone defines the desired outcome, selects variables, decides which historical cases are suitable for training or comparison, sets thresholds and determines what consequence follows each score. Even a simple rule —giving priority to workers who accept shifts at short notice— contains an idea of the ideal employee. It rewards somebody able to reorganise life immediately and disadvantages a person who studies, provides care, relies on public transport or needs notice because of a disability. The formula can be applied identically to everyone and nevertheless produce a perfectly foreseeable inequality.
Every management system begins with choices. Someone defines the desired outcome, selects variables, decides which historical cases are suitable for training or comparison, sets thresholds and determines what consequence follows each score.
That is why “the algorithm does not see names” does not settle the problem. A postcode, available hours, gaps in employment or type of device may operate as proxies for social conditions. Nor is it enough to say that the model is right most of the time. An overall success rate conceals who experiences the failures and what they cost. Mistaking one early delivery for a late one may affect a statistic; mistaking a suitable person for an unsuitable one may close access to work. Responsible evaluation separates groups, contexts and consequences, and asks which kind of error is tolerable before deployment.
The discussion becomes clearer when we distinguish four actors: whoever commissions the tool, whoever designs it, whoever uses it and whoever bears its effects. They may belong to different organisations, but that does not dissolve responsibility. An employer should not answer “that is the supplier’s issue” after using a recommendation to change a shift or reject an applicant. A supplier should not hide behind trade secrecy when basic controls are missing. Drawing this map supports directed questions: who can stop the system, who corrects data, who audits outcomes and who repairs a loss?
Questions a young worker can actually ask#
After an adverse decision, it is useful to replace allegations that cannot be demonstrated with requests for concrete information. “What decision was made and when did it take effect? Was an automated tool involved? Which categories of data were used? Is there a record I can verify? Who will review the case and by what date?” These questions do not presume bad faith, but they require a vague answer to become a testable process. If the conversation takes place orally, the young person can send a short email summarising what they understood and ask for confirmation. That trace helps if the problem recurs.
Documentation does not mean secretly recording people or collecting colleagues’ information. It is enough to retain the vacancy, contract, schedules, communications, permitted personal screenshots, evaluations and access changes. A timeline is useful: date, decision, explanation received, financial effect and person contacted. It can also record context the system may have missed, such as authorised leave, a logged equipment failure or an agreed adjustment. Evidence of context is stronger than a hundred impressions. If the matter affects pay, discrimination, health or continued employment, appropriate trade-union, professional or regulatory advice may be necessary.
When personal data are involved, an employer should explain processing in an understandable way; an endless policy that nobody can apply to the case is insufficient. The Spanish Data Protection Agency’s guide to data protection in employment relationships provides criteria concerning information, necessity, proportionality and rights. The practical question is whether each item of data is needed for a specific work purpose. A scheduling application should not obtain personal contacts by default; tracking location outside working hours requires far stronger justification than recording arrival at a workplace.
Human review is not clicking accept#
Many organisations say a person is always supervising. That statement has value only if the person understands the output, can see sufficient information, has time, may depart from the recommendation and is not rewarded for confirming it automatically. A manager reviewing five hundred alerts each hour is not exercising judgement; they are acting as a stamp. The employer should examine how many decisions are changed, why, how long correction takes and whether some groups must appeal more often. A zero reversal rate might signal perfection, but is more likely to reveal that the remedy does not work.
Effective review starts by identifying the exact decision rather than defending the system in the abstract. It then checks input data, hears contextual information and applies known criteria. The final response explains what is maintained or changed and why. Where possible, an irreversible consequence is paused during examination. If an application was rejected because of an inaccurate record, correcting the database after the vacancy has been filled does not make the case complete. Remedy addresses the effect: restoring an opportunity, returning hours, removing a penalty or reinstating access, as well as repairing the record.
Spain’s Workers’ Statute recognises worker representatives’ right to information about the parameters, rules and instructions behind algorithms or artificial-intelligence systems influencing decisions capable of affecting working conditions. This collective information complements an individual complaint. It can reveal a pattern one person cannot see: fewer shifts for workers who reject instant availability, worse evaluations after authorised leave or exclusions concentrated in particular career histories. A family can encourage a young worker to consult employee or union representatives without portraying that step as inevitable confrontation.
The benefit-and-risk test#
Before introducing a tool, an organisation should complete a one-page record. Problem to be solved; less intrusive alternative; people receiving the benefit; people taking new risks; necessary data; prohibited decisions; accountable owner; appeal channel; harm indicators; review date; and withdrawal condition. Requiring the phrases “who gains” and “who may lose” dismantles impersonal language. If easier planning depends on notifying shifts later, uncertainty has not been removed. It has been transferred from the employer to workers’ households, which must reorganise education, care, transport and income at short notice.
The same test belongs in a pilot. Accuracy and savings are not the only measures. Compare short-notice changes, interrupted rest, complaints, corrections, differences between groups, accidents, turnover and extra time spent feeding the system. Ask workers which invisible tasks it created: explaining breaks, correcting time records, competing for assignments or keeping an application active. A technically accurate outcome may degrade the craft if it encourages people to avoid difficult cases or conceal problems to protect a score. What an indicator causes matters as much as what it measures.
Certain artificial-intelligence uses in employment, recruitment and worker management have specific obligations under the European Union Artificial Intelligence Act. The law does not automatically make every output fair, and it does not replace employment and data-protection analysis. It reinforces an essential idea: risk depends on use and effects, not on how spectacular a technology appears. A spreadsheet containing opaque rules can cause more harm than an advanced model restricted to assistance. The organisation must govern the entire process, including suppliers, updates and newly proposed uses.
A practical case: the availability score#
Imagine a café that allocates more hours to workers accepting rapid substitutions. Management calls this flexibility; a student attending lessons experiences it as punishment. The first step is to specify the rule. Does each refusal remove points? For how long? Is previously declared unavailability treated differently? Does the score also affect contract renewal? The second step is checking data and effects. The third is considering alternatives: publishing shifts earlier, creating a voluntary reserve or distributing emergencies within limits. The discussion stops opposing “innovation” and “resistance” and begins comparing concrete designs.
A reasonable solution might separate normal from exceptional availability, prohibit penalties for rejecting unagreed changes and set a maximum number of emergencies per worker. The system would display the reason for each assignment, allow availability to be corrected and flag conflicts for a manager instead of resolving them alone. The team would review monthly whether hours are concentrated and whether education, caring responsibilities or disability create disadvantages. This design does not promise neutrality: it makes its values visible and supplies controls. Useful transparency does not teach code; it allows people to understand a rule, anticipate its consequence and challenge it with evidence.
Discussing it at home without teaching helplessness#
Families can rehearse three scenes: an unexplained application rejection, a sudden fall in shifts and a low-performance alert. In each scene the young person practises four moves: describe the fact without exaggeration, ask for the criterion, supply context and request a reviewable decision with a deadline. The adult avoids taking over unless there is serious risk or a minor needs legal support. The aim is to learn a transferable competence: speaking firmly without inventing certainty. It is also worth acknowledging that a satisfactory answer may not arrive and the matter may need to be escalated.
Some signals justify rapid help: a major income effect, indications of discrimination, monitoring outside working hours, use of particularly sensitive data, retaliation for complaining, a threat to health or a serious sanction. In those circumstances, do not improvise a battle on social media. Preserve evidence, consult appropriate worker representation or advice, and use formal channels. Publishing names, third-party screenshots or allegations can create new problems. Protection combines voice with prudence. Asking well does not guarantee success, but turns a confusing experience into a case another person can examine.
The answer the official story often avoids#
The uncomfortable question has a simple answer: the organisation choosing to use the system remains a decision-maker, even when complex suppliers and models are involved. It must therefore be able to explain the purpose, justify data and criteria, provide real oversight and repair harm. Responsibility does not disappear inside a technical box. Nor is it transferred to a young worker because they cannot understand an inaccessible calculation. Whoever gains the capacity to organise, select or monitor assumes a proportional duty to govern that capacity and open a route for disputing outcomes.
A dignified future of work does not require a choice between perfect machines and technological rejection. It requires institutions able to state what they want to optimise, what they refuse to sacrifice and who will answer when things fail. For a teenager, keeping that question alive is a form of citizenship: who decided, and with what authority? For an employer, answering it well is a test of maturity. When an organisation can name an owner, evidence, boundaries and remedy, technology stops being an excuse and becomes a tool again. A less comfortable but much more honest conversation starts there.
