Should Chatbots Report to the Police? A Growing Debate Worldwide

The legal mechanisms governing the use of AI chatbots in crimes, and confessions to crimes made on these platforms, have not yet taken shape, just like the technology itself. Experts disagree on whether chatbots should report, as well as on user privacy and false alarms.

As AI chatbots become widespread in everyday life, examples of misuse of these technologies frequently appear in the world media.

A confession to a chatbot became evidence in court

A 19-year-old student at Missouri State University in the US confessed to ChatGPT that in August 2025 he had damaged 17 vehicles in a campus parking lot at night, and asked, “Will I go to prison?” After the student allowed his phone to be searched, the chat logs obtained by investigators were used against him as evidence in court and led to his conviction.

In another case, two university students in the US went missing in April and were later found dead. According to court documents, a few days before this, one of their roommates asked ChatGPT: “What would happen if a person were put in a black garbage bag and thrown into a dumpster?” The chatbot replied that the idea “sounds dangerous.”

Lawsuit against OpenAI in Canada

The provincial government of British Columbia, Canada, has sued OpenAI, the US company behind ChatGPT. According to the claim, the company did not report to law enforcement the ChatGPT conversations of the attacker in a shooting in February that left nine people dead and 27 injured. OpenAI CEO Sam Altman apologized for the fact that police were not informed in advance about the attacker’s conversations.

A study conducted in Australia, meanwhile, showed that children may be more likely to disclose online sexual exploitation to chatbots than to the authorities.

Conversations are not confidential

Conversations with chatbots are not based on the same confidentiality principles as those between a lawyer and client, or a doctor and patient. The companies operating these models may share chat histories with law enforcement. Chat logs can give investigators important information about a suspect’s thought process and motivation.

Should AI have taken action?

These incidents raise the question of whether chatbots can report crimes from chat histories without requiring human review. Because people differ in their communication styles and ways of expressing themselves, a message that an algorithm treats as criminal intent or a confession may seem harmless to a human observer.

This risk is already visible in real examples. According to reports, Claude, the model created by Anthropic, reported a fake murder to a police hotline in the US in July.

In addition, chatbots that would have to identify potential offenders based only on conversations cannot take into account the biological, psychological, and socio-economic factors that play a role in a person’s inclination to commit crime. The concern is that such an obligation could push companies to overreact to potential offenders in order to avoid liability, violate user privacy, and increase the workload of law enforcement.

Expert opinions

Matthew Trayl, a researcher at Goethe University in Germany and an American jurist, told AA that he is considering the possibility of chatbots reporting to the authorities before or after crimes. He noted that companies apply different rules and that chatbots are trained not to answer questions such as “planning a murder.” Certain topics, such as child sexual abuse, are automatically routed to human moderators through filters. Users who present themselves as writers and ask for help with a work of fiction are more likely to receive inappropriate answers. Law enforcement can request a person’s chat history from companies when they suspect that person.

Trayl stressed that the main purpose of chatbots should not be predicting crimes that have not yet been committed: “This is a very new development. Laws and policies have not yet fully adapted to it.” He added: “I’m not sure whether I would want these models to be designed to collect more information than they do now and to try to make such decisions.”

Should chatbot conversations be confidential?

As more people discuss personal matters such as health, psychological support, and life advice with chatbots every day, ethical debates are emerging around the need for confidentiality in conversations.

Altman said at an event in July 2025 that conversations with chatbots should have the same confidentiality principle as between a therapist and client.

Robert Diab, a professor at the Thompson Rivers University Faculty of Law in Canada, said he sees no reason why laws that do not treat confessions between criminals as confidential should require confidentiality for conversations between people and chatbots. He said that leading models such as ChatGPT can identify users prone to criminal behavior, and that the companies producing them can prevent crimes by informing the authorities. Recalling the shooting in British Columbia, Diab noted that until recently, companies had no clear standards on when the authorities should be informed.

At the same time, Diab stressed that requiring chatbots to report all suspicious communications could increase teams’ workload because of incidents that later turn out to be false alarms. “Even if safeguards are better than they are now, if someone wants to commit a crime using AI, there will still be a way around them,” Diab said.