Experimental comparisons with GPT-4o vanilla across three roles, evaluated through an ablation study and a multi-evaluator panel combining LLM-based and human judges, consistently rank XBot as the best performing system, demonstrating superior empathy, role stability and conversational depth, while GPT-4o vanilla exhibits pervasive persona drift across all experimental scenarios.
Abstract
Large language models (LLMs) have demonstrated remarkable capabilities across different domains, yet two critical challenges limit their reliability: opaquely generated responses and role drift in extended interactions. Moreover, the possibility of a multi-layered dialogue, shaping LLM behavior both asynchronously across sessions and, optionally, in real time, remains largely unexplored. This paper presents XBot, a conversational agent designed to address these challenges. Built on the GPT-4o API, XBot requires no access to internal weights or activation space, making it portable across different LLM-based systems. It analyzes the user message and decomposes it into chunks, each annotated with topic and sentiment. It then selects a domain validated strategy from a curated expert-defined set, based on the assigned role, spanning multiple levels of granularity from general response types down to topic- and sentiment-specific guidance. Each response is therefore accompanied by explicit justifications accessible to non-technical users, supporting a continuous three-way dialogue among users, domain experts and XBot for ongoing validation and iterative refinement. Domain experts can inspect, modify and extend the strategy set at any time, ensuring alignment with professional standards and ethical requirements. Experimental comparisons with GPT-4o vanilla across three roles, evaluated through an ablation study and a multi-evaluator panel combining LLM-based and human judges, consistently rank XBot as the best performing system across all dimensions, demonstrating superior empathy, role stability and conversational depth, while GPT-4o vanilla exhibits pervasive persona drift across all experimental scenarios.
This study explores LLM-generated documents, their social impacts and biases, and their outcomes in research. Language, as a deeply intricate and nuanced system, continues to challenge our efforts in both understanding and generation. Language modeling has progressed from foundational statistical techniques to the sophistication of Large Language Models (LLMs) powered by deep learning. Trained on vast quantities of data, self-generated content, and self- and semi supervised learning, demonstrate remarkable capabilities for producing contextually relevant, human-like text and executing a broad spectrum of language tasks. Despite their promise, these systems are not without limitations, particularly concerning embedded biases stemming from training data and model architecture. Emerging scholarship delves into their symbolic reasoning, potential, and interdisciplinary applications, though many avenues remain insufficiently explored. As examples of generative artificial intelligence, ChatGPT increasingly contributes to education by fostering accessible learning, facilitating domain-specific assessments, and nurturing creativity and critical thinking among learners. It attempts to understand the LLM model by unraveling the facts in day-to-day practices. This article applies the qualitative methods used in comparing two AI-generated recommendation letters, indicating male and female (Ajit/ Ajita). Textual review as a major tool and technique of knowing LLMs and their biases, significance, social impacts, and the ways of interpreting biases with outcomes. It is based on sociotechnical system theory and algorism bias theory. This article claims that LLMs are the latest AI-generated product utilized in academic discourses, enriching the capacity of the user. The biases could be minimized by careful attention, capacity, tactful utilization, and knowledge of the user.
A. Sharma· Multi-Disciplinary Explorati...· 0 citations
The resulting prototype confirms that a cloud-hosted multimodal LLM, when combined with a minimal and well-structured web stack, can serve as a practical foundation for next-generation digital assistants suitable for customer support, education, and personal productivity applications.
G. Vamsi, Vinay Kumar Male· International Scientific Jou...· 0 citations
Despite their high conversational fluency, large language models (LLMs) tend to produce responses that are either contrived or factually inaccurate, a phenomenon known as hallucination. This constrains their dependability in areas where accuracy is crucial, like customer service. This study leverages a context-aware chatbot built on a Retrieval-Augmented Generation (RAG) pipeline to solve the problem. The system retrieves semantically relevant text from an external knowledge base by integrating neural information retrieval with LLMs to ensure factual correctness and enhanced contextual relevance. These externally retrieved documents are given as a reference during response generation. The LLM-as-a-judge approach is used to evaluate the system by comparing responses to a qualitative performance matrix using GPT-4o. Results show that the RAG-based chatbot increases context precision by a factor of 7.5, decreases hallucinations (measured through Faithfulness) by 73.20% and improves answer relevance by 6.97% when compared to a random retrieval baseline system. This study advances conversational AI by demonstrating how the retrieval method significantly enhances the usefulness and reliability of LLMs for enterprise-level customer service. The results show that the RAG architecture provides a scalable alternative for creating precise, contextually grounded conversational agents, thereby mitigating some of the main drawbacks of LLMs.
Rabia Shabbir, K. Talpur, Shakeel Ahmad· ICCK Transactions on Machine...· 0 citations
This work analyzes real chatbot failures to identify six recurring mechanisms and defines six controlled evaluation modes in Hy-MultiTurn, a Chinese benchmark for deep multi-turn dialogue understanding, which shows that Hy-MultiTurn is broadly challenging.
Eileen Ye, Ji-Hua Tao, Yao-Ming Li et al.· 0 citations
Large language models (LLMs) often exhibit sycophancy, optimizing for agreement over productive challenge, which severely limits their utility in domains like professional skills training, where growth requires pushback. We introduce, ConvoDojo, a novel conversational AI platform for practicing difficult workplace conversations, engineered not merely as a commercial training application but also as a flexible, instrumented research platform for evaluating conversational AI strategies. ConvoDojo repurposes LLMs as structured sparring partners to support skill development in difficult workplace conversations (e.g., performance feedback, conflict resolution), addressing the reported managerial tendency to avoid them. This paper showcases the platform and presents an evaluation of how key conversational user interface (CUI) design elements, namely, the addition of structured feedback and upfront instructional scaffolding, impact managers’ learning. Results show that ConvoDojo is highly engaging and promotes user reflection. We demonstrate how theory-informed dialogue and adaptive pushback can transform an LLM into an effective, measurable tool for complex communication skills development.
Everlyne Kimani Cross, Luiza A Santos, Laurent Denoue et al.· International Conference on...· 0 citations
