ChatGPT Versus MediSearch: A Comparative Analysis of Artificial Intelligence Platforms for Patient Education in Cardiac Surgery and Cardiac Anesthesia.

Objectives

To evaluate the reliability and readability of responses generated by two artificial intelligence (AI) platforms-ChatGPT and MediSearch-to common patient-focused questions about cardiac surgery and its anesthetic implications.

Design

Prospective, comparative, cross-sectional evaluation.

Setting

Single-institution academic study; no direct patient involvement. Questions were sourced from publicly available online data (Google "People Also Ask" feature).

Participants

One hundred unique patient-focused questions about cardiac surgery and anesthesia, categorized using the Rothwell classification system (80 fact, 14 policy, 6 value).

Interventions

Each question was submitted individually to the free versions of ChatGPT (version 4.0) and MediSearch. Responses were evaluated using three validated metrics: reliability, assessed using the modified DISCERN tool; readability, assessed using the Flesch-Kincaid grade level (FK) and Coleman-Liau index (CL); and clinical accuracy, assessed by a board-certified cardiothoracic anesthesiologist and a board-certified cardiothoracic surgeon using a 3-point Likert scale.

Measurements

AND MAIN

Results

MediSearch demonstrated significantly higher reliability (mean DISCERN score, 4.64 v 2.47; p < 0.001). ChatGPT generated significantly more readable responses (mean FK score, 10.33 v 11.34 [p < 0.01]; mean CL score, 15.72 v 17.30 [p < 0.001]) and significantly higher clinical accuracy (mean, 0.82 v 0.71; p < 0.01). Subgroup analysis by Rothwell category confirmed MediSearch's reliability advantage across all question types, while ChatGPT's readability advantage was most pronounced for fact-based questions.

Conclusions

This study reveals a critical trade-off between evidence-based reliability and linguistic accessibility in AI-generated cardiac surgery patient education content. MediSearch offers superior reliability, while ChatGPT provides more readable and accurate responses. Neither platform achieves the sixth-grade readability level recommended for broad public accessibility. These findings underscore the need for AI platforms to balance clinical accuracy with health literacy standards before being endorsed as patient education tools in high-stakes settings.