Truace
Subscribe

RAG chatbot for personalized PrEP support

Source article: A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation

Background Chatbots have the potential to reduce barriers to preexposure prophylaxis (PrEP), including lack of awareness, misconceptions, and stigma, by providing anonymous and continuous support. However, in the context of PrEP, chatbots are still nascent; they lack personalized informational expertise, peer experiential expertise, and human-like emotional support to facilitate future PrEP uptake. These personalized and relatable forms of support are crucial for increasing engagement, influencing health decisio…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1446· cite & verify
A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation
Kapitel 3 1 ChatBot internal by Robert Wagner. CC BY 4.0 · https://creativecommons.org/licenses/by/4.0
Trace impact readingContested
P 64The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 66The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

Researchers built a chatbot for people considering preexposure prophylaxis. It splits incoming queries, classifies whether they ask for information, emotional support, or context, and retrieves relevant social media documents before the Gemini-2.0-flash model writes a reply. After ten internal evaluation rounds, they ran a blinded comparison against a general LLM and real-world user responses.

The work matters because PrEP uptake is held back by lack of awareness, misconceptions, and stigma, and an anonymous, always-available tool could help. Yet the specialized chatbot did not beat a general LLM on most criteria, and expert and user feedback has not yet been gathered, so its real-world benefit is still uncertain.

Main points

  1. The system pairs a query preprocessor using Segment Any Text and a fine-tuned Gemma 2 support classifier with a retrieval-augmented generation module.
  2. Retrieval relied on SBERT embeddings with cosine similarity plus topic matching to select relevant documents.
  3. Internal rounds scored responses on 10 criteria, including accuracy, actionability, tone, and empathy.
  4. Prompt decomposition and segmenting instructions improved comprehensiveness and relevancy for complex, long queries.
  5. Prompt effectiveness varied with task complexity, likely due to LLM sensitivity to prompt structure and linguistic variability.

The gain

In a blinded comparison, a retrieval-augmented PrEP chatbot was preferred over real-world user responses and matched or beat a general LLM on empathy and tone.

The problem

The specialized RAG chatbot was rated below a general LLM on most evaluation criteria, so its added value for PrEP support remains unproven.

The rundown

The development followed two phases: prototype conceptualization, in which real-world PrEP needs were identified and a dialogue flow diagram designed, and iterative chatbot development. The aim was to deliver personalized information, peer experiential expertise, and human-like emotional support.

Evaluation combined ten internal rounds with a blinded comparative study analyzed by a linear mixed model. Both the general LLM and the RAG chatbot were preferred over real-world user responses, but the general LLM led on most criteria except empathy and tone.

The authors conclude the chatbot uses social media data to address PrEP misconceptions and promote self-efficacy, and say further analysis with expert and user feedback will be done to validate and improve it.

What this doesn’t fix

The chatbot's potential is not yet validated: expert and user feedback is still to come, and the general LLM outperformed it on most criteria.

Sources

  1. Peer-reviewedJMIR Research Protocols2026-10-01

The debate