Authentic assessment in an AI-integrated qualitative research methods course

Abstract

Authentic assessment seeks to evaluate students through work resembling professional practice, but complex professional tasks can exceed the procedural capabilities of undergraduate novices. Generative artificial intelligence (AI) may alter this constraint by providing on-demand procedural support while leaving judgment-dependent dimensions of performance available for assessment. This single-semester case study examines an AI-integrated undergraduate qualitative research methods course in which students were expected to use AI throughout development of a publishable research manuscript. The quantitative corpus comprised 33 final manuscripts produced under unrestricted AI access. Three rule-based, rubric-proximal indicators were used to estimate dispersion. The Methods Specificity Index showed strong upper-bound compression: the median was 6 on a 0–6 scale, 69.7% of manuscripts received the maximum score, and 97.0% scored either 5 or 6. Claim–Evidence Coupling remained substantially dispersed (range = 0.000–0.750, median = 0.333, Q1–Q3 = 0.273–0.542, IQR = 0.269), as did Theory–Interpretation Linkage (range = 0.000–0.917, median = 0.444, Q1–Q3 = 0.250–0.500, IQR = 0.250). Qualitative comparison of four contrasting student cases identified three recurring process differences associated with stronger outcomes: criteria-based orchestration, provision of irreducible project-specific inputs, and substantive multi-turn iteration. In this course, unrestricted AI access therefore did not eliminate meaningful variation in student products. Structural aspects of research writing clustered near competence, while dimensions requiring evidence integration and theoretical judgment remained diagnostic. The case suggests that authentic assessment can remain viable in AI-integrated undergraduate settings when assessment design distinguishes between work AI can scaffold reliably and forms of judgment students must still exercise.