Skip to content
Open access

CounselingQA: scaling professional mental-health guidance via dense retrieval and large language models

Jul 2026 · Social Network Analysis and Mining · 0 citations

TL;DR

Results show that dense retrieval provides strong candidates and that transformer/LLM-driven reranking further elevates relevant, on-topic advice to the top positions, indicating that retrieval-first pipelines can help scale access to professional guidance.

Abstract

In the digital era, the internet and social media have emerged as essential platforms for individuals facing mental health issues, often used for seeking information and community support. Despite the resources of informal advice available on social media, the complexity of these issues frequently exceeds non-expert knowledge. Specialized sites such as CounselChat and 7Cups offer professional guidance, yet many at-risk individuals still rely on unmoderated sources and general web search. We address this gap by investigating ranking strategies that match pre-existing expert advice to incoming mental-health questions. We introduce , a collection built from two specialized websites, pairing user questions with verified expert responses. We address the task as answer retrieval (AR): given a question, rank expert answers by relevance. We evaluate dense retrieval with SentenceBERT and MentalBERT, and propose a second stage that improves the initial ranking via transformer-based models and large language models (LLMs), used for filtering non-relevant candidates and for reordering. Beyond retrieval, we analyze linguistic style and affective attributes across topics, questions, and responses. Results show that dense retrieval provides strong candidates and that transformer/LLM-driven reranking further elevates relevant, on-topic advice to the top positions. We further conduct qualitative error analyses, including human evaluation to study the benefits and limitations of our approaches. Taken together, these findings indicate that retrieval-first pipelines can help scale access to professional guidance.

Read PDF

Similar papers

Conference Jul 2026

A Web-Based Mental Health Assessment Framework using Natural Language Processing Techniques

Mental illnesses like depression, anxiety, stress, and so on have become more widespread, and this has necessitated the availability of assessment tools that are readily available, scalable, and automated. The paper will offer a web-based mental health risk assessment system that utilizes the latest technology of Natural Language Processing (NLP) in real-time to analyze textual information provided by users. The offered system incorporates a hybrid deep learning framework with the relation of DeBERTa, BiLSTM, and XGBoost to promote the contextual comprehension, sequential emotional pattern identification, and effective classification of the performance. First, text input by the users is received with a secure web interface and processed with general NLP preprocessing methods, such as tokenization, lemmatization, and sentiment normalization. To extract deep semantic relationships in the text, DeBERTa is used to extract contextual embeddings. Such embeddings are also trained in the form of a Bidirectional Long Short-Term Memory (BiLSTM) network in order to capture emotional dynamics and linguistic reinforcing relations. Fused feature representation, sentiments, and linguistic indicators are input into an XGBoost classifier to predict mental health in multi-class. There is a weighted risk scoring system used to measure the level of severity and provide tailored feedback. It is experimentally tested on standard mental health text data sets that the proposed hybrid framework is more effective than the traditional machine learning and standalone transformer models in terms of accuracy, precision, recall, and F1-score. The architecture is scalable to the deployment of a web system, which is guaranteed to perform in real-time, secure data, and privacy of users. The suggested framework offers a solid and smart instrument to identify the risk of mental health early and help intervene in time and to promote the development of digital health care.

D. D, S. S, L. K et al. · 0 citations
Open access Aug 2026

An ontology-guided knowledge graph and retrieval-augmented LLM framework for interpretable mental health assessment

Mental health detection from social media has gained increasing attention due to the spontaneous expression of psychological states on online platforms. However, accurate interpretation remains challenging because social media language is highly contextual, ambiguous, and often includes sarcasm, metaphors, or non-clinical expressions, which can lead to unreliable predictions. Existing approaches based on machine learning, deep learning, transformers, and large language models (LLMs) primarily rely on text-driven patterns and often lack explicit ontology-guided grounding, structured knowledge validation, and interpretability. Although LLMs improve contextual reasoning, their predictions can be inconsistent and prone to hallucination when not supported by structured domain knowledge and a unified cross-component validation mechanism. To address these limitations, this paper proposes an ontology-guided framework that integrates a Knowledge Graph (KG), an Ontology-Informed Retrieval Classifier (ORC), and a Large Language Model for interpretable mental health detection from social media text. The framework leverages the Human Phenotype Ontology (HPO) to guide symptom extraction and Knowledge Graph construction, normalizing informal linguistic expressions into ontology-grounded representations. A key design contribution is an explicit KG–ORC cross-validation consistency gate, which requires both the Knowledge Graph and the ORC module to independently reach agreement before a prediction is committed reducing erroneous inferences from either component alone. The LLM is deliberately restricted to a post-validation role, generating human-readable explanations and supportive recommendations only from validated outputs, thereby reducing hallucination risks and preserving knowledge-consistency. The proposed framework is evaluated on the Dreaddit dataset, a large-scale, multi-domain Reddit corpus for stress and mental health analysis. Empirical evaluation on the HPO guided symptom extraction confirms differential detection rates of 56.04% in stress positive posts versus 31.11% in stress negative posts, validating the signal relevance of the ontology-driven extraction. End-to-end binary classification on the held-out test split achieves an accuracy of 74.82% with an F1-score of 75.64%. Crucially, on the 66.55% of instances where the KG and ORC modules reach consensus, the consistent subset accuracy rises to 78.84% (F1 = 78.28%), compared to 63.03% for the Knowledge Graph alone. The KG–ORC agreement rate across the full test set is 82.02%. These results demonstrate that the KG–ORC cross-validation gate measurably improves predictive reliability over single component baselines, and that ontology-guided, knowledge-grounded reasoning offers a principled path toward interpretable and knowledge-consistent mental health analysis from social media.

Amina Tahir, Ghulam Mustafa, Muhammad Tanvir Afzal et al. · 0 citations
Preprint Jul 2026

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

This work extends PubHealthBench, a question answering benchmark of 7,929 questions derived from UK Government public health guidance, into a retrieval-augmented setting and systematically evaluates retrieval and generation choices, and introduces a rubric-based LLM-as-a-judge covering faithfulness, completeness, clarity, and factual consistency.

Felix Feldman, Joshua Harris, Timothy Laurence et al. · 0 citations
Open access Jul 2026

Comparing Human and Large Language Model Responses to Patients Online Questions: Towards Multi-dimensional Patient-centered Support

Patients and caregivers seek informational and emotional support throughout medical care, especially when interpreting unfamiliar laboratory test results. Although resources such as patient portals and online health communities (OHCs) help address questions, gaps remain. The emergence of large language models (LLMs) offers the potential to be a complementary source of support to assist patients and caregivers in understanding and using their test results. The objective of our study is to empirically compare LLM responses to patients online questions containing their laboratory test results to responses written by peers in an OHC. We compared the 519 peer replies to 122 laboratory test-related posts from an OHC to 488 responses generated from four LLMs using mixed computational and qualitative methods. LLMs frequently provided clear explanations of medical terminology and structured interpretations of numeric results but were longer and less readable. Peers offered more personalized, context-specific emotional support. Overall, LLMs have the potential to complement peer responses in OHCs, but require greater emotional depth, reasoning transparency, and alignment with community norms.

M. Hussein, R. Doshi, L. He et al. · 0 citations
Jul 2026

Healthmate: An AI Chatbot for Symptom-Based Healthcare Consultation

HealthMate is presented, an intelligent, explainable AI chatbot framework designed for preliminary healthcare consultation that demonstrates rapid retrieval, robust natural language comprehension, and clear explainability without replacing professional medical diagnosis.

K. Jyothi, Shaik Khasim Basha · 0 citations