Experts Warn on Risks of AI Mental Health Chatbots for Children
Researchers warn AI mental health chatbots for children may harm development, worsen inequity, and require urgent regulation and developer accountability.
Children’s mental health experts warned this week that the rapid spread of AI mental health chatbots for children is outpacing safeguards and could produce unintended harms, particularly for younger users. A commentary by researchers at the University of Rochester Medical Center and SUNY Upstate argues these tools are mostly designed for adults and lack the contextual awareness and ethical design needed for pediatric care. The authors urged developers, clinicians, and regulators to work together to ensure safety, equity, and appropriate use.
Rising use driven by access gaps
AI mental health chatbots for children are emerging amid persistent shortages of pediatric mental health providers and uneven insurance coverage in the United States. Developers market apps, mood trackers and conversational agents as low-cost, on-demand alternatives that can reach families who otherwise face long waits or high expenses. That accessibility creates pressure to adopt these tools in schools, clinics, and homes before their limits are fully understood.
Experts highlight developmental and attachment risks
Researchers emphasize that children are not small adults and interact with technology differently than adults do. Evidence cited by the authors suggests young users may attribute moral qualities and inner life to robots and chatbots, raising the risk of unhealthy attachment and reduced motivation to build human relationships. Those developmental concerns are especially acute for younger children whose social and emotional skills are still forming.
Clinical context is missing from chatbot interactions
Pediatric mental health treatment typically involves families, schools and other caregivers because a child’s social environment is central to diagnosis and safety planning. AI chatbots lack the ability to observe family dynamics, verify home conditions, or coordinate with clinicians and social services when a child appears at risk. That blind spot means chatbots may miss warning signs of abuse, neglect, or imminent harm that a trained therapist would identify and escalate.
Bias in training data deepens inequities
The commentary warns that AI systems reflect the data used to build them, and unrepresentative datasets can produce biased outcomes for children from different racial, ethnic, geographic or economic backgrounds. Children who experience adverse childhood events—such as community violence, parental substance use, or family incarceration—often need more intensive care but face the greatest access barriers. If chatbots are trained on limited or skewed data, they are unlikely to recognize or respond effectively to the needs of marginalized children.
Regulatory gaps leave protections uncertain
Currently most AI therapy chatbots are not subject to formal regulatory oversight for pediatric use, the authors note, and only a single AI-based app has FDA clearance to treat major depression in adults. That regulatory gap creates no standardized requirements for safety testing, reporting of harms, or transparency about training data and clinical validation. Researchers argue that without clear rules, there is no reliable mechanism to prevent misuse or to ensure products are safe for children.
Researchers call for partnership with developers
The team behind the commentary is seeking collaboration with app developers to examine how safety, ethics and pediatric expertise are integrated into product design. They want to know whether companies consult pediatricians, child psychologists, parents and adolescents during development and whether models are tested on diverse populations. Rather than advocating a ban, the researchers call for thoughtful deployment, stronger evidence standards, and processes that prioritize children’s developmental needs.
The debate over AI mental health chatbots for children underscores a broader tension: the urgent need for scalable mental health support and the equally urgent need to protect vulnerable users from harm and inequity. As technology companies move quickly to meet demand, clinicians and ethicists warn that deliberate, well-governed steps are required to ensure that digital tools supplement—not supplant—human-centered pediatric mental health care.
