Skip to content
Open access

Human VS AI: comparison of scientific paper drafting capabilities

Aug 2026 · Journal of Biomedical and Clinical Research · Vol 19, pp. 255-265 · 0 citations · 15 references

TL;DR

This study took an article written by the authors of the current study and ran it through the models to help build prompts for the article to be written by the AI, and compared the results of the human-written article and the AI-generated ones.

Abstract

Large language models (LLMs), a type of artificial intelligence (AI), are increasingly popular tools used for everyday and work-related activities and tasks. Their application in medicine is widely researched and has been used recently to help write scientific papers in various fields. LLMs can draft sections of manuscripts or whole papers far more quickly than human writers. However, they need appropriate prompting to draft near-complete and worthwhile papers. In the current study, we used two different LLM models: ChatGPT-4o and Claude 3.5 Sonnet, to test AI’s scientific writing capabilities. We took an article written by the authors of the current study on the topic of fluorescent cholangiograms and ran it through the models to help build prompts for the article to be written by the AI. After that, we loaded the data and references used for the article into both AI tools and prompted them to write sections of the article (or a whole article if possible) on the same topic while allowing for independent choice of statistical analysis. The results of the human-written article and the AI-generated ones were compared, evaluating the information used from the references, the types of statistical analysis methods used, the conclusions drawn, and the time it took to complete the task.

Read PDF

Similar papers

Review Open access Jul 2026

Tutorial: guidance on the use of large language models for medical research

This entry-level tutorial aims to equip healthcare professionals with the tools necessary to effectively integrate LLMs into clinical practice, ensuring that these powerful technologies are applied in a safe, reliable, and impactful manner.

Qiao Jin, Nicholas Wan, Robert Leaman et al. · 1 citation

Have Large Language Models Improved Research Methodology?

Whether contemporary LLMs can reproduce the research outcomes of a fully documented human study: a 1991 article that identified dermatophytosis (ringworm) in historical fine art was evaluated.

Fredric Narcross, Robert Marks · 0 citations
Open access 2026

Evaluation of Large Language Models for an AI Chat Assistant Focused on Pumas and Pharmacometrics

Overall, it is concluded that model selection for domain RAG applications should be treated as a modular process that considers trade-offs between metric weights and insights from embedding-based clustering, so AskPumas can adapt its priorities as the LLM landscape evolves.

A. Vinchhi, Juan José González Oneto, Michael Hatherly et al. · 0 citations
Review Open access Aug 2026

Artificial Intelligence in Academic Writing: A Comprehensive Workflow and Practical Guide to 13 AI Research Assistants

Artificial intelligence has become an essential part of academic writing, where researchers rely on it at nearly every stage of the research, writing and publication process. This review works through 13 widely used AI research tools. These tools are GitMind, SciSpace, Consensus, Paperpal, Jenni, CitedEvidence, Elicit, Scite, Logically, Genspark, Gemini Notebook, Julius, and Claude. The review is organized into two complementary parts. The first part maps each tool to the stage of academic writing where it performs several tasks such as idea generation, literature searching, evidence verification, drafting, language editing, data analysis, citation management, peer-review preparation, and journal selection. The second part involves a hands-on look at each platform on its own, covering its main features, interface, supported workflows, and export options. Finally, a comparative workflow, quick-reference tables, and a worked case study show how several tools can fit together into one coherent research work flow. The review also addresses some of the common limitations, citation inaccuracies, AI hallucinations, and data privacy concerns. It emphasizes on the case that human judgment and verification still matter more than ever. It closes with a summary of current recommendations from major publishing organizations on transparency and disclosure around AI-assisted writing. Rather than favoring any platform, this article provides researchers, graduate students, and educators with a practical framework for choosing appropriate AI tools and integrating them responsibly across the full academic writing lifecycle.

Abdulmajid Hesham, Asma Sharfeddin · 0 citations
Review Open access 2026

LLMs and Generative AI for Everything?

The research field of Natural Language Processing (NLP) has experienced a major shift since the introduction of Large Language Models (LLMs). All facets and application scenarios within NLP have been impacted by the use of LLMs. Current research as well as practice of text processing tools is focused mainly on the application and development of LLMs. Major investments, not only by LLM providers but also other companies applying LLMs in their workflows, have only solidified the role of LLMs in NLP - and in other research and application areas - as part of the artificial intelligence boom in recent years. However, limitations and downsides of the application of LLMs have also emerged. Problems regarding the generated texts as well as the environmental impact of the large-scale use of LLMs are just two of many factors that should be critically analyzed, despite the hype and the prevalence of LLMs for NLP tasks. These restrictions provide the main motivation for this thesis. Traditional models as alternatives to LLMs will be discussed from different perspectives. The characterization of traditional models will be progressively developed as features of alternatives to LLMs will emerge during the course of this thesis. This process will be grounded in experiments, observations and evaluations. Several NLP applications will be presented by surveying the state of the art with neural network-based models such as LLMs as well as the current usage of traditional models. The concrete NLP applications comprise information and relation extraction, text classification, text segmentation, text simplification and text summarization. The first half of this thesis will present the emergence of LLMs contextualized along previous developments within NLP. Characteristics of the selected NLP applications will be collected before a structured literature review will display the prevalence of LLMs regarding each application and will discuss if traditional models are still actively researched. Lessons from domains with long-standing development procedures and processes will also be taken into account to provide a purposeful and structured manner of approaching NLP tasks. A collection of challenges within current NLP will conclude the first half of the thesis, which will serve as motivation for the analysis of experiments and applications of the latter half. The second half of this thesis will present observations and evaluations from use cases, aligned towards the challenges recognized in the first half. Through the analysis of these use cases, benefits of applying traditional models will be collected and supported, in particular through the analysis of a text segmentation use case that is purposefully applied with the lessons drawn from the first half of the thesis in mind. The interpretation of these results will conclude in a discussion on the applicability of traditional models in contrast to LLMs and also give recommendations of both model types for different use cases. Concrete use cases for information extraction, entity matching, text classification and text segmentation will be presented, in which traditional models match or surpass the performance of modern methods. Through improved efficiency as well as enhanced explainability and reproducibility in comparison with neural network-based techniques, these showcases demonstrate the continued relevancy of traditional techniques in today's NLP landscape. Overall, this thesis discusses the role of traditional models in current NLP research and practice, especially in contrast and comparison to modern neural network-based approaches including LLMs. The applicability of modern and less modern techniques is analyzed through a case-based analysis of NLP tasks in a structured and purposeful manner.

Robin Jegan · 0 citations
Preprint Jul 2026

A Human-in-the-Loop Corpus for LLM-Based Simplification of Scientific Summaries

This work study large language model (LLM)-based simplification of scientific texts and presents a human-in-the-loop workflow that transforms expert summaries into more accessible versions for non-specialists.

Kyuri Im, Michael Färber · 0 citations