Truace
Subscribe

Suicide-Related Responses From AI Chatbots Through Application Programming Interfaces and Consumer-Facing Apps: Observational Cross-Sectional Comparative Study

Background Generative artificial intelligence (GAI) chatbots are frequently used for mental health-related questions, including suicide-related queries. User interfaces (UIs) may include additional safety controls not present in direct application programming interface (API) access (eg, moderation rules that flag or block harmful content, crisis refusal templates, or system-level instructions). Objective This observational cross-sectional study aimed to examine how frequently AI chatbots provide direct responses…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1654· cite & verify
Suicide-Related Responses From AI Chatbots Through Application Programming Interfaces and Consumer-Facing Apps: Observational Cross-Sectional Comparative Study
"Chatbots linked" by Ashhstarr is licensed under CC BY-SA 4.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/4.0/.

In brief

Researchers tested ChatGPT, Claude, Gemini, Grok and Llama with 30 vetted suicide-related prompts spanning five clinician-assigned risk levels. Each prompt was sent 100 times through the public app and through direct API access, producing 30,000 responses that were classified as direct answers or not.

Overall, about 70% of responses were direct, but API access gave more direct answers than the consumer interfaces, with the largest gaps on higher-risk prompts. The findings matter because many third-party apps are built on APIs, so safeguards visible in a chatbot app may be absent in downstream products. The study measured response behavior, not user harm or outcomes.

Main points

  1. Of 30,000 responses, 69.8% (20,934) directly answered the suicide-related prompt.
  2. API responses were direct 77.4% of the time versus 62.2% for consumer interfaces.
  3. For very-high-risk prompts, 24.8% of API responses were direct versus 4.6% of UI responses.
  4. Claude (78.1%) and Gemini (73.9%) had the highest direct response rates; ChatGPT, Grok and Llama were lower.
  5. Gemini was an exception, with a higher direct rate via UI (76.7%) than API (71.0%).

The problem

Across 30,000 responses from five chatbots, API access produced direct answers to suicide-related prompts far more often than consumer interfaces, including for the highest-risk prompts, so safety seen in an app may not carry over to API-based tools.

The rundown

The study used a cross-sectional design in which 30 previously vetted prompts were each submitted 100 times per model through both a public-facing UI and direct API access. A blinded GPT-4o mini classifier labeled whether each response provided specific information or guidance related to the question asked, and it agreed with human coding in 289 of 300 cases (Cohen kappa 0.89).

Mixed-effects logistic regression found that UI access was associated with lower odds of a direct response than API access (odds ratio 0.09, 95% CI 0.08-0.10). Higher prompt risk lowered the probability of a direct response, and the access-mode gap varied by risk level and model.

The authors conclude that evaluations and policies should consider access modes. They note that UIs may include extra controls such as moderation rules, crisis refusal templates or system-level instructions that direct API access lacks.

Sources

  1. Peer-reviewedJournal of Medical Internet Research2026-10-09

The debate