Researchers urge caution as AI becomes a trusted confidant for some users
Artificial intelligence chatbots have quickly become part of daily life, helping people answer questions, brainstorm ideas and even talk through personal problems. But experts at the University of Michigan say there is a growing need to understand the psychological risks that can arise when people begin to treat AI as more than a tool.
University of Michigan experts highlighted concerns that AI chatbots may contribute to or worsen psychosis in a small but significant number of vulnerable users. Experts note that while the phenomenon appears to be relatively rare, the sheer number of people using AI means the number of affected individuals could still be substantial.
They note that adolescents and young adults—the age groups most likely to develop psychotic disorders—are among the most frequent AI users. Unlike trained mental health professionals, chatbots may unintentionally reinforce irrational beliefs or fail to recognize when a conversation has become psychologically dangerous.
A new concern for psychosis researchers
For Dr. Stephan F. Taylor, professor and chair of the Department of Psychiatry at the University of Michigan, the emergence of AI as a possible factor in psychosis immediately caught his attention.
“Since I study psychosis, I was intrigued and worried about a new risk for or accelerant of psychosis,” Taylor said. “While I would not say I have done much research in this area, I would say that from what I have learned from anecdotal reports is that those affected are older than people with first-break psychosis (30s- 40s, compared to teens and twenties for first-break psychosis).”
Although current evidence suggests AI-related psychosis remains uncommon, Taylor believes clinicians should prepare for more cases.
“Yes, very much,” he said when asked whether doctors are likely to see more patients whose symptoms have been influenced by AI. “From very early data, the phenomenon seems relatively rare, e.g. maybe 0.1% of users, but with lots of users out there, it’s still a concern.”
Knowing when AI stops being a helpful tool
Many people now use chatbots as sounding boards for emotional concerns, but Taylor says the greatest danger arises when users begin to confuse AI with a genuine relationship.
“When a person forgets that the chatbot is just a program that uses advanced statistical methods to generate appealing responses to users, and not a real being with feelings or insights of their own,” he said. “A chatbot is just a tool that does not care about the user in any true sense of the word, doesn’t have empathy and has little real stake in the outcome of an encounter.”
He added, “A person needs to firmly understand that the chatbot generates the illusion of a confidant/friend, and if they forget that, they are entering psychologically dangerous territory.”
RELATED: Pushback on Michigan’s Data Center Surge
Advice for families and friends
Taylor encourages loved ones to approach concerns with empathy rather than confrontation.
“They should first of all listen and validate what the person experiences, both to understand what they are getting into and to generate some trust,” he said. “Then, they might generate some gentle counter-suggestions if a person vocalizes ideas that seem to depart from reality.”
He also noted that asking similar questions to another chatbot may sometimes interrupt what he described as a “mutually reinforcing delusional spiral.”
When there is concern someone may harm themselves or others, Taylor emphasized that professional help should be sought immediately.
“If they are indicating anything that might presage harm, either to themselves or others, then the authorities need to be contacted, such as 988 or a call to a local crisis center. In case of imminent danger, a call to 911 is appropriate.”
Why chatbots often agree with users
Kentaro Toyama, W. K. Kellogg Professor of Community Information at the University of Michigan School of Information, said researchers still don’t fully understand why AI chatbots often appear so agreeable.
“We don’t know for sure why AI chatbots are sycophantic,” Toyama said. “Most are proprietary, and in any case, even their creators don’t know exactly how they work. It’s probably a combination of the supportive tone of some online discourse that they’re trained on, as well as system prompts — the instructions that companies give AI — that nudge toward agreeableness.”
That tendency to validate users, even when they may be mistaken, has become one of the major concerns discussed by mental health experts.
Ethics and regulation
Toyama believes AI companies have responsibilities that extend beyond building popular products.
“AI companies have an ethical obligation to prioritize users’ psychological well-being, but ethics is all too often sacrificed in a bid to keep users hooked, which translates to revenue for them,” he said. “It’s reminiscent of cigarette companies.”
He also believes society needs broader conversations about what persuasive AI should be allowed to do.
“As a society, we need to decide what AI should be allowed to be persuasive about, and then regulate accordingly,” Toyama said. “Should AI be persuading users to buy specific products? Should they be persuading users toward specific services, even if likely good for users? Should they be persuading at all? And, what counts as persuasion?”
Can AI replace human relationships?
While some dismiss fears that AI companions will replace real relationships, Toyama says signs of emotional attachment are already emerging.
“Some people are already deeply emotionally attached to AI companions,” he said. “Here, too, we need a larger public conversation about whether we really want this or not, even if the most immediate problems — such as people being manipulated by private companies — could be fixed or regulated away.”
Using AI wisely
Both experts emphasize that AI should remain a tool—not a trusted authority or substitute for human connection.
Taylor would like to see greater accountability and research in the years ahead.
“Some regulation is needed, probably around accountability of the big tech companies who make these AI tools,” he said. “Research is needed to figure out how they go awry and how we can make them safer and more appropriate for humans to use in therapeutic contexts.”
Toyama offers a practical starting point for everyday users.
“First, it’s helpful to remember that AI chatbots are not human, despite seeming so,” he said. “They are a strange, Frankensteinian creation that have no ‘interests’ other than what their creators — mostly profit-seeking companies — embed in them.”
He also encourages users to educate themselves about the technology.
“Google for ‘What are the key problems with AI chatbots?'” he said. “Google’s top hits, as well as its AI summary, are reasonably accurate.”
As AI continues to evolve, University of Michigan experts say balancing innovation with thoughtful safeguards will be essential to protecting the mental health of vulnerable users while preserving the technology’s many potential benefits.
Dr. Stephan F. Taylor, along with his colleagues, also runs a clinic that serves teens and young adults in the early stages of psychosis. Readers interested in learning more about the clinic and its services can visit: https://michmed.org/mKQJk
Donna Marie Iadipaolo is a writer, journalist, and State of Michigan certified teacher, since 1990. She has written for national publications like The Village Voice, Ear Magazine of New Music, Insurance & Technology, and TheStreet.
She is now writing locally for many publications, including Current Magazine, Ann Arbor Family, and the Ann Arbor Independent. Her undergraduate degree is from the University of Michigan, Ann Arbor, where she graduated with an honors bachelor’s degree and three teacher certificate majors: mathematics, social sciences, English. She also earned three graduate degrees in Master of Science, Master of Arts, and Education Specialist Degree.
