When someone describes exhaustion, loneliness, emotional distress, or trouble sleeping, a responsible digital response should not begin by prescribing an app, a chat, or an exercise. It should begin with an open question: what is happening, what support is being sought, what context shapes the situation, and what would make the conversation safe? A tool can organise information or make resources easier to find, but it cannot independently understand the meaning of bereavement, a difficult relationship, financial insecurity, or immediate danger. Listening before recommending prevents a human need from being reduced to a feature list.
Digital wellbeing also needs explicit limits. A platform does not replace assessment by qualified professionals, should not diagnose or promise clinical outcomes, and is not an appropriate channel for an emergency. If there is immediate danger, thoughts of self-harm or harm to another person, violence, abuse, severe confusion, or an inability to stay safe, urgent help should be sought through local emergency services, a crisis line, or health services. In Spain, 112 is the emergency number and 024 is a suicide-behaviour support line. This is general information, not clinical advice.
The first question is safer than a quick recommendation#
Two people who say “I need to switch off” may mean very different things. One may want fewer notifications during a demanding week; another may feel isolated and want someone to talk to; a third may be at risk. It is therefore useful to ask what kind of help is wanted, whether there is a trusted person, what makes seeking support difficult, and whether technology has helped or worsened matters before. Listening is not a form for sorting people. It is a chance to make the response fit an expressed need.
Two people who say “I need to switch off” may mean very different things. One may want fewer notifications during a demanding week; another may feel isolated and want someone to talk to; a third may be at risk.
Material context matters too. Advice that assumes a personal phone, unlimited data, privacy at home, fluency in a language, or spare time can add pressure. Alternatives should include non-digital routes: primary care, school guidance, social services, local organisations, or a person chosen by the individual seeking help. WHO notes that mental health is shaped by social, economic, and environmental conditions. An interface cannot by itself solve housing insecurity, violence, discrimination, or an inadequate network of care.
Scenario one: a mood app that concealed a need for company#
A student downloads a mood-tracking app after seeing advertisements about streaks and graphs. For several days she records increasingly low answers, and the system encourages her to preserve the streak. Later, she explains that she has just moved to a new city, knows nobody, and is struggling to attend class. The problem was not a failure to enter data. She needed companionship and a practical way to reconnect with her surroundings.
A careful response does not turn a graph into a diagnosis or demand intimate details. It can ask whether she wants to identify a safe person to talk to, whether the school has guidance support, or whether community resources would be preferable. If she expresses risk or cannot stay safe, the appropriate next step is to help her reach urgent or professional support, not extend an automated conversation. The app may remain as an optional journal for noticing personal patterns, never as an authority on what is happening to her.
Limits must be understood before a screen is accepted#
A wellbeing tool should explain, in plain language, its purpose, limits, data collection, storage, access, and deletion. Consent is not meaningful if someone does not understand that a questionnaire may create alerts, send information to a provider, or affect access to another service. Institutions have heightened duties of information, proportionality, and protection when children are involved. Collecting sensitive data merely because it is technically easy to collect is poor practice.
Data-protection rights, including access, correction, deletion, and objection, should be usable rather than buried in a long help centre. Providers should document security measures, retention periods, incident handling, and the boundaries of automated systems. If they cannot explain those matters, they should not receive health or wellbeing information. Privacy is not a separate legal detail; it is a condition under which many people can safely ask for support.
Scenario two: a questionnaire that turned an alert into a label#
An organisation offers a weekly wellbeing questionnaire, and one answer triggers an alert visible to group managers. A person who slept badly while caring for a relative learns that several colleagues know about the alert and worries that they now see her as incapable. The design mistook a signal for conversation for a conclusion about a person. It had also failed to explain who would see the result or how long it would be retained.
A responsible redesign removes labels, collects the minimum necessary information, and makes the consequence of each answer clear. Any alert that is necessary and proportionate should reach a trained person bound by confidentiality and a reviewable protocol. There should be a way to seek help without completing the questionnaire and a way to correct or contextualise a record. Success is not the number of alerts produced. It is whether support is offered without exposing, surveilling, or penalising the person.
Responsible referral opens a door rather than closing a conversation#
Acknowledging limits does not mean abandoning someone who asks for help. A responsible referral explains why speaking to a professional may help, what type of resource exists, how to reach it, and what can be done while waiting. It may include practical help to locate primary care, mental-health services, educational guidance, violence-support services, or community organisations. It can also ask whether the person wants a trusted companion. Choice and pace should be respected whenever there is no urgent risk.
Referral should not be presented as a diagnosis. Saying that a situation merits a professional conversation is not the same as assigning a clinical condition. A digital system should not independently determine severity, decide treatment, or state that a practice will work for a particular person. General information from WHO or health services can help people recognise when to seek support, but individual assessment belongs to qualified services. Clear boundaries protect both the individual and the people running the service.
Scenario three: a chatbot available at night#
A worker uses a chatbot late at night because he does not want to worry his family. The bot responds warmly and suggests breathing, writing, and returning tomorrow. In a later message, he says he cannot carry on and has access to means of harming himself. At that point, conversational continuity is not enough. The design should visibly offer instructions for contacting emergency services, crisis lines, and a nearby trusted person, without pretending that the system can monitor or rescue him.
A responsible service prepares for these cases before launch. It reviews safety language with specialists, adapts resources to the country, tests that links work, and does not promise absolute confidentiality if escalation mechanisms exist. It also avoids language that blames someone or reduces a crisis to a self-care technique. The priority is to make urgent human contact easier. After immediate danger has passed, digital support may remember resources or preferences, but it cannot replace needed care.
Privacy, accessibility, and equity are part of care#
Privacy is not a legal detail separate from wellbeing. Someone may stop seeking help if they fear that a partner, employer, family member, school, or insurer will learn what they wrote. Notifications should be discreet, accounts secure without impossible barriers, and language understandable. Accessibility requires screen-reader compatibility, adequate time, low data use, translation where needed, and equivalent human channels. Not everyone can or wants to use the same technology.
Equity requires examining who receives worse support. A video-only service can exclude a person without private space; a text-based service can make participation harder for people with some disabilities or limited literacy; a service that recommends paid resources can reinforce inequality. Before deployment, it is wise to test with diverse groups and publish what changed. UNESCO links AI ethics to human rights, inclusion, and human oversight. Those principles become practical when they change a screen, a schedule, or a path to support.
Scenario four: a reminder on a shared screen#
A platform sends a notification titled “emotional follow-up pending.” The recipient shares a tablet with family, and the alert appears on the home screen. Although it reveals no diagnosis, it exposes a request for support that the person had not chosen to share. The solution is not simply changing a phrase. People should be able to choose the channel, disable previews, use neutral messages, and continue through a non-digital route.
Safety depends on ordinary circumstances. Design testing should include shared devices, insecure connections, language differences, and coercive control. People seeking help should retain control over notifications and data wherever possible. When a tool cannot provide that protection, it should say so prominently and offer a safer alternative. Honest limitation is more protective than a reassuring interface that hides its risks.
Governing support tools with accountability#
Institutions offering wellbeing technology need identifiable owners, training, periodic review, and a complaints process. They should record the need they intend to address, identified risks, indispensable data, available alternatives, and the date on which the system will be reconsidered or stopped. Affected people must be able to challenge a recommendation, report harm, and receive an understandable reply. NIST recommends continuous AI risk management; listening after launch is as important here as testing before launch.
The first question does not delay help. It prevents a quick response from becoming the wrong response. Asking, listening, and explaining limits makes it possible to guide without intruding, recommend without prescribing, and refer without disengaging. A well-situated tool can reduce barriers and bring resources closer. Responsibility begins when it recognises what it cannot do and always leaves a route to human, professional, and urgent support when that is needed.
Institutional and primary sources: World Health Organization, mental health; Spain’s Ministry of Health, 024 line; Spanish Data Protection Agency, data-protection rights; United Nations, Convention on the Rights of the Child; UNESCO, Recommendation on the Ethics of AI; NIST, AI Risk Management Framework.
