Skip to content

Domain-Sensitive Patterns in Student Writing: Comparing Human, AI-Revised, and AI-Generated Abstracts in Human Factors

Oct 2026 · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · 0 citations · 14 references
Academic Writing and Publishing

TL;DR

The preliminary findings motivate examining degrees of AI involvement and using participant-grouped validation in future studies to examine structural and lexical features, readability, Sentence-BERT embeddings, exploratory writing-condition classification, technology acceptance, and perceived workload.

Abstract

Generative AI is increasingly used in academic writing, although many studies compare only human-written and AI-generated text. This within-subjects study examined manual, AI-revised, and AI-generated extended abstracts in a graduate human factors course. Text analyses included 96 abstracts from 32 participants, and repeated-measures survey analyses included 30 participants. We compared structural and lexical features, readability, Sentence-BERT embeddings, exploratory writing-condition classification, technology acceptance, and perceived workload. Manual abstracts showed the greatest variability. AI-generated abstracts showed less structural variability, higher lexical density, and greater estimated reading difficulty. AI-revised abstracts showed metric-specific overlap with both other conditions. A three-class classifier yielded 68.75% accuracy under abstract-level leave-one-out cross-validation, although participant and topic overlap limits generalization. Participants reported greater acceptance and lower workload in the AI-supported conditions. These preliminary findings motivate examining degrees of AI involvement and using participant-grouped validation in future studies.

View source

Similar papers

#natural language process... Preprint Nov 2026

Detecting AI-Authored Text in Authentic Student Writing: Evidence from PISA 2025

AI-text detectors reach near-perfect scores in evaluations that contrast fully human-written texts with texts generated from researcher-designed prompts. In authentic student writing, however, AI use is voluntary and often partial. We study this setting with the PISA 2025 FLA dataset, in which 15-year-olds in Germany w...

Nils-Jonathan Schaller, Thorben Jansen, O. Köller et al. · 0 citations
Open access Oct 2026

Human or Machine? EFL Students’ Trust Calibration in AI-Generated English Content:A Study at the Faculty of Languages, Surman

Generative artificial intelligence (GenAI) is increasingly used by university EFL students for language learning and academic tasks, yet fluent output may be inaccurate. This study examined self-reported trust and verification practices and students’ judgments of selected AI-generated English content among undergraduat...

Asim Omar Salim Altarabulsi · 0 citations
Open access Nov 2026

Lexico-syntactic complexity in human and AI-generated outputs across interpreting and translation: a paired-corpus study

Abstract This study examines differences in lexico-syntactic complexity between human translation outputs and AI-generated texts under interpreting and translation conditions, with implications for computer-assisted language learning (CALL) and translator/interpreter education. The dataset comprises 60 source segments...

Yi-Li Dan · 0 citations
Open access Oct 2026

MACHINE LEARNING-BASED DETECTION OF AI-GENERATED CONTENT IN STUDENTS’ WRITTEN WORKS

The results support the use of stylometric indicators as an interpretable supplementary signal for educational text analysis, while also emphasizing that detector outputs should not be treated as definitive proof of AI use in individual student work.

A. Sultanali, G. Kazbekova · 0 citations
#large language models Open access Sep 2026

Assessing AI-generated academic writing in Spanish: a multivariate analysis of syntactic variation across ChatGPT, Gemini, and Claude

Findings indicate that the ChatGPT, Gemini, and Claude models generate academic Spanish through differentiated syntactic configurations, suggesting that dependency-based syntactic profiling can support the interpretation of AI-assisted academic writing, inform rubric refinement for academic Spanish, and provide transpa...

E. Vázquez-Cano · 0 citations

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

MIT News · Artificial Intelligence Nov 17, 2026

Putting a spotlight on AI, society, and humanity

Supported by the MIT Human Insight Collaborative, scholars are exploring how AI is impacting the human experience and reshaping higher education.

Microsoft Research Blog Nov 17, 2026

Can AI help build a healthier future?

What can AI enable in medicine, and how do we get there? Microsoft Research lead Doug Burger, Microsoft Principal Research Manager Ava Amini, Microsoft Chief Scientific Officer Eric Horvitz, and physician-scientist Eric Topol of Scripps Research explore these questions and more. The post Can AI help build a healthier future? appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.