Human-in-the-Loop AI Agent for Venue-Specific Manuscript Readiness Assessment

Abstract

Background: Manuscript readiness is shaped before formal submission, especially when authors must align scholarly claims, venue scope, template rules, references, and presentation quality under deadline pressure. Objective: This study develops and evaluates a human-in-the-loop AI agent for venue-specific manuscript readiness assessment in AI-driven smart society and digital transformation research ecosystems. Methodology: The prototype was implemented as a Streamlit application connected to a local LLM through LM Studio. It integrates venue-profile configuration, manuscript parsing, scope checking, template checking, novelty and contribution assessment, methodology review, reference checking, language and readability checking, reviewer persona simulation, readiness scoring, author override, and audit logging. The revised evaluation involved eight author-facing sessions, and twelve assessments runs across four manuscript domains and three assessment conditions: unassisted checklist, generic LLM prompting, and the proposed venue-specific agent. Results: The venue-specific agent achieved higher average issue coverage (81.25%) and actionable suggestion rate (78.75%) than checklist-only review (52.25%, 61.25%) and generic prompting (65.75%, 68.50%), while reducing mean issue-identification time from 43.0 minutes to 18.5 minutes. Conclusion: The system does not predict acceptance; it supports authors by converting venue rules and reviewer-like criteria into traceable revision priorities before submission.