Two controlled experiments report two controlled experiments crossing all three factors on one held, contamination-free synthetic corpus, evaluated across five models, finding which format is preferable where accuracy spread is genuine.
Abstract
Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade. We report two controlled experiments crossing all three factors on one held, contamination-free synthetic corpus (the"Book of Veyra,"8,780 uniquely-named entities, deterministically regenerable from a fixed seed), evaluated across five models. Experiment 1 (960 calls/model) measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement. Perfect-response rate collapses to zero by N=80 for every model, format, and placement. Placement produces effects at least as large as format at N=160 in most models, but the direction is model-specific. No model shows a reliable markdown advantage; one 35B model favors plain text instead. Experiment 2 (5,520 calls/model) measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder in the same four formats. Recall stays near ceiling through 64-128k tokens, then degrades sharply and format-dependently: one model's accuracy spread reaches 48 points at 128k tokens. Fabrication never occurs (0/5,760 probes), and sycophancy stays negligible (<=8.3%). What rises sharply near each model's context ceiling is outright refusal to answer (0% to 79-90%), distinct from sycophancy or fabrication. Neither pre-registered format ordering holds, and token overhead (+22% to +37% over plain text) further changes which format is preferable where accuracy spread is genuine. We release the full harness, corpus generator, and raw results (VeyraBench): https://github.com/iNetanel/veyrabench
The study identifies prompt anchoring as a source of methodological variation in LLM-assisted content analysis, indicating that anchoring strategies should be explicitly specified, justified, and reported as part of the study methodology.
This single-case experimental study examined the effectiveness of the colour-coded paragraph-writing intervention (CCPWI) with three fifth-grade students attending a German lower-secondary school that provides inclusive special education. All participants demonstrated adequate spelling and grammar but persistent difficulties in organizing written text. A multiple-baseline-across-participants design was employed, featuring concurrent baselines and staggered intervention onsets. Instruction followed the gradual-release model (“I do, we do, you do”) and utilized a colour-coded graphic organizer to support topic, detail, explanation, and conclusion sentences. Paragraph quality served as the dependent variable. Visual analysis revealed consistent phase changes across all three cases. Effect-size indices indicated strong improvements, with nonoverlap of all pairs (NAP) values of 94%, 89%, and 94% and Tau-U coefficients of .62, .63, and .87 for the three participants, respectively; Hedges’ g ranged from 1.20 to 1.53. Hierarchical piecewise regression showed a significant positive intervention slope, suggesting progressive growth rather than an abrupt level shift. End-of-study, one-on-one interviews reflected high social validity. Overall, findings support CCPWI as a feasible, acceptable, and effective approach for enhancing expository-paragraph structure among at-risk learners who, despite adequate foundational skills, exhibit pronounced and persistent difficulties in higher-order writing.
Matthias Grünke, Kathy B. Ewoldt, Anne Barwasser et al.· Exceptionality Education Int...· 0 citations
Generative AI tutors have become a common tool for independent learning, yet their capacity to support self-regulated learning (SRL) is poorly understood. This simulation-based textual analysis of prompt design evaluates a frontier large language model (Claude Sonnet 4.6) as a tutor across 60 scripted sessions on a single topic (density), crossing three levels of SRL-informed system prompting (Minimal, Moderate, Extensive) with four learner-behavior variants (Standard, Misconception, Disengagement, Overconfidence). Tutoring transcripts were scored on a 14-dimension framework spanning SRL phases, SRL developmental stages, self-determination theory principles, and Merrill’s First Principles of Instruction, applied via an LLM judge. Adding SRL context to the system prompt raised total tutoring scores, but only at the Extensive SRL support level. Minimal and Moderate prompting produced the same performance, near 36 on a 70-point scale, and Extensive prompting raised it to 40, a statistically significant effect (partial η2 = 0.24). The learner’s behavior in the session had a larger effect than the prompt did (partial η2 = 0.37), with disengaged learners scoring lowest. The threshold pattern held under an independent judge from a different developer than the tutor model. The findings support a method for evaluating GenAI tutors empirically and point to dynamic, dialogue-aware prompting alongside explicit SRL scaffolding.
Kendall Hartley, Fabiola Sáez-Delgado, Javier Mella-Norambuena· Future Internet· 0 citations
Large language models (LLMs) are common tools for learning, research, and productivity, yet their dense outputs disproportionately burden users who experience attention, working memory, and information-processing challenges. We present Adapt AI, a system that restructures LLM responses through content condensation, visual hierarchy, and an original/adapted toggle, intervening at the presentation layer rather than the model itself. Through semi-structured interviews with ten participants screened for self-reported cognitive-processing difficulties, all ten reported the adapted view was easier to read, and seven of ten spontaneously described it as making AI-generated content more accessible. We treat these preference results as exploratory: the study used no comparison group and is subject to demand characteristics. Our findings show that the value of Adapt AI lies not in shortening responses but in reorganizing them to support scanning, comprehension, and user agency, and that the appropriate level of restructuring depends on task context. We discuss how designing the presentation layer of AI outputs offers an underexplored pathway to making LLM-based tools more inclusive.
This study examines the reliability and validity of generative AI in summative assessment, emphasizing how prompt design influences grading when applying a common rubric to complex student work. Fifteen business plans from a master’s-level course were evaluated by GPT-5 through Microsoft Copilot using three prompts of increasing rigor (basic, intermediate, rigorous). Each plan was scored in five independent runs per prompt, producing 225 AI evaluations. Analyses included intraclass correlation for consistency, severity contrasts, and convergence with instructor scores using correlation, error metrics, and Bland–Altman limits of agreement.
Prompt design significantly shaped score distribution and strictness. The most rigorous prompt reduced inflated scores and aligned more closely with instructor judgments, yet it also underestimated performance and showed the greatest inconsistency. Single AI runs were unreliable, but averaging multiple evaluations improved stability. At the criterion level, AI struggled to match instructor ratings on commercial and economic viability, even under stricter prompts.
Findings highlight the pedagogical implications of prompt sensitivity in AI-assisted grading. Reliability and fairness depend not only on rubric quality but also on evaluative instructions. Results support multi-run aggregation, bias-aware calibration, and hybrid human–AI models to ensure rigor and equity in technology-enhanced assessment. These findings inform the design of AI-enhanced assessment practices that support fair, transparent, and pedagogically aligned learning environments.
Large Language Models generate text autoregressively, but open-ended generation is prone to verbatim looping, in which models repeat spans already present in context. Standard defenses such as repetition, presence, and frequency penalties and n-gram blocking act on token recurrence rather than the sequential structure of a loop, and often suppress looping only at strengths that also degrade formatting or fluency. We propose Don't Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context. Sequence breakers protect chat templates and formatting tokens. Across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study, DRY reduces suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no comparable reduction, identifying suffix matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8K performance, whereas standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its practical impact on text generation.
Philippe Weidmann, Allen G. Roush, Judah Goldfeder et al.· 0 citations