Jul 2026· Annual Conference on Innovation and Technology in Computer Science Education· pp. 646-652· 0 citations· 22 references
Computer Science
TL;DR
Though retake opportunities allowed students to improve quiz performance, frequent retake attempts were associated with lower final exam outcomes, suggesting continued struggle on novel problems, illustrating how ML?inspired assessment can be incorporated into courses without a full course redesign.
Abstract
Students may learn at different paces due to differences in prior programming experience (PPE), physical or mental health challenges, and economic or lifestyle barriers (e.g., employment or caregiving responsibilities). Additionally, increasing use of AI tools for homework can contribute to inaccurate self-assessment and poor preparation for supervised tests. Motivated by these challenges, we introduced bi-weekly low-stakes checkpoint quizzes in a large (~500-student) introductory CS course. Inspired by alternative grading paradigms such as mastery learning (ML), each quiz could be attempted multiple times without penalty, offering students frequent feedback and opportunities for iterative improvement. Our mixed-methods study investigated the impact of these quizzes on performance, self-assessment, stress levels, and overall experience, using performance and survey data (N=456). Results showed that though retake opportunities allowed students to improve quiz performance, frequent retake attempts were associated with lower final exam outcomes, suggesting continued struggle on novel problems. Despite limited performance benefits, survey data revealed strong affective outcomes based on overwhelmingly positive student sentiment: students reported high Likert-scale ratings for learning/engagement and stress reduction value (though subgroup differences by gender, PPE, English fluency, and retake frequency suggest room to improve equity outcomes), and the majority of open-ended responses described the quizzes as helpful for improving self-assessment, reducing stress, and supporting meaningful learning. Overall, our implementation allowed students to experience some benefits of ML while retaining enough structure to prevent procrastination, illustrating how ML?inspired assessment can be incorporated into courses without a full course redesign.
AI support is increasingly embedded in online quizzes, yet instructors often lack clear, actionable signals about where students struggle during those assessments. We present a classroom-deployed quiz system that combines integrity-preserving hinting (TA-AI), trace summarization (Analytics-AI), and feedback-driven refinement (AI-Improver) to generate instructor diagnostics from routine interaction logs. The system was used in a graduate assembly programming course over five quiz weeks (N = 18). We report deployment evidence focused on reliability and instructional usefulness for monitoring: promptintent coding reached substantial agreement (Cohen’s kappa [κ] = 0.81); fixed-effects models (with student and item controls) showed a negative association for one-hint interactions (odds ratio [OR] = 0.231, indicating approximately 77% lower odds of a correct response for single-hint interactions relative to 0-hint interactions); and item-level demand spikes were operationalized via a demand × success prioritization process for weekly review. Rather than producing automated judgments or claims of causal learning gains, the analytics are designed as practical prioritization cues that direct instructor attention toward high-need items during AI-assisted quizzes.
M. P. Lin, Daniel H. Chang, V. Janarthanan et al.· International Journal of Eme...· 0 citations
It is found that there is no significant differential effect of GenAI availability on grades overall or among previously lower-performing students, and the findings temper concerns that GenAI inflates grades and reduces students's satisfaction.
J. Dumlao, Meng Wang, Zhonghan Xie et al.· 0 citations
Skills-based evaluation is an alternative evaluation model that is a variation of both mastery and specifications grading. Students are evaluated on the set of skills they have demonstrably acquired over the duration of a course and the level at which they are able to demonstrate those skills. This is in opposition to traditional models which evaluate performance at fixed time points. A growing body of research suggests that such alternative evaluation models are more equitable, motivating, and efficacious for students. However, adoption remains limited in part due to concerns about student acceptance, perceived lack of rigour, and the potential increase in workload due to repeated assessments. In this study, we assess perceptions of skills-based evaluation in an introductory (CS0) course for non majors. A total of 248 students and 7 teaching assistants responded to survey prompts addressing their experience of the efficacy, psychological impact, and workload when compared to traditional grading. Participants also provided open-ended feedback to provide additional context. The students showed broad support for the skills-based evaluation, expressing strong preferences across all recorded metrics. Results were consistent across gender lines, and supported by sentiment analysis of open ended responses. Although the small number of responses prevent any strong conclusions, teaching assistants were generally positive towards the methodology's impact on students, though opinions were largely split on the impact on workload.
Brian Harrington, Katherine Lambert, Leon Lee et al.· Annual Conference on Innovat...· 0 citations
Science and Engineering subjects require high levels of student efforts and self-motivation for understanding complex concepts, often abstract and counter intuitive. Different student backgrounds and prior experience can provide important challenges for the learning experience and student learning outcome gains in HE settings, particularly in the context of large classes and scientific disciplines. Students coming from non-traditional academic backgrounds may experience significant difficulties, which can cause low performance and loss of motivation in continuing their studies. In light of this, a Peer Assisted Student Mentor (PASS) scheme has been conceptualised and implemented in the last few academic years within the School of Engineering & Built Environment, with the intent to further improve the student outcomes, enhancing continuation and retention rates. The project aims to provide our students with an additional and effective support opportunity route for challenging modules, e.g. highly maths-based or other modules requiring high problem-solving skills, across all levels (from 4 to 7). This consists in scheduled weekly student-mentor sessions, where a group of students from higher levels of the course help newer students struggling with their modules. Support may include reviewing theory, specific teaching material on BB, resolution methods of specific numerical exercises, preparation for typical exam questions, etc. The scheme has provided significant improvements, not only in terms of student outcomes in traditional low-performing modules, but also for building a stronger students’ sense of community, where mutual support and peer-to-peer learning are at the basis of a growth in sense of belonging, self-confidence, and motivation.
Adriano Cerminara, Stephen A. Agha, Victoria Mellon· Journal of Scholarship of Te...· 0 citations
While the computer-generated feedback was broadly considered useful by students, student engagement patterns were markedly different in the solo setting, with students demonstrating reluctance to use the interface's built-in help features and tending to internalize failure in unproductive ways counter to the intention of a formative learning environment.
J. C. Meyer, S. Pollock, Bethany R. Wilcox et al.· 0 citations
This study intentionally delayed the introduction of an LMS-integrated LLM tool, CodeHelp, until after early-semester assessments and substantial measures of student effort had already been collected, suggesting that engagement with LLM-based tools reflects underlying self-regulatory behaviors and that the tool functions as a form of technological scaffolding primarily activated by already-engaged learners.
Laura M. Cruz Castro, Maryam K. Multani, Gabriel Castelblanco et al.· IEEE Access· 0 citations