Beyond Automation: How Generative AI is Reshaping Research Evaluation and Scholarly Communication

Abstract

The rapid proliferation of Generative AI (GenAI) tools—including large language models (LLMs) such as GPT-4, Claude, and Gemini—is fundamentally transforming the landscape of scholarly research, from hypothesis generation and literature synthesis to manuscript preparation, peer review, and research impact assessment. While these tools promise to democratize research productivity and accelerate scientific discovery, they simultaneously introduce profound challenges to the integrity, equity, and epistemological foundations of academic knowledge production. This paper presents a comprehensive mixed-methods investigation into the impact of GenAI on research evaluation and scholarly communication, combining a large-scale survey of 2,850 researchers, editors, and graduate students across 45 countries with a quantitative analysis of 18,000 manuscripts submitted to 12 peer-reviewed journals between 2022 and 2025. We find that GenAI adoption in research workflows has increased from 28% to 72% for literature review and from 18% to 54% for manuscript drafting over two years. AI-assisted manuscripts demonstrate 29 points higher clarity scores and 33 points higher structural coherence compared to unassisted manuscripts, but exhibit 8% lower citation accuracy due to hallucinated references. We further propose ScholarAI, a responsible GenAI framework for scholarly communication that integrates provenance tracking, citation verification, bias detection, and transparent authorship attribution. A controlled evaluation with 480 researchers demonstrates that ScholarAI improves manuscript quality by 46.8% while reducing ethical violations by 78.5% compared to unregulated GenAI use. We conclude with policy recommendations for institutions, publishers, and funding agencies navigating the GenAI transformation of scholarship