Background: Medical artificial intelligence (AI), machine learning (ML), and deep learning (DL) studies frequently begin with datasets collected for routine care rather than for computational modeling. Such datasets may contain inconsistent variables, heterogeneous measurement time points, unexplained NaN values, poorly defined outcomes, missing metadata, and insufficient documentation, which can compromise model development before any algorithm is selected. Methods: This Technical Note proposes a physician-facing Clinical AI-Readiness Guide for preparing medical datasets before AI-based analysis. The guide was developed as a practical framework organized around pre-modeling decisions, including the clinical task, cohort, minimum common dataset, outcome definition, predictor variables, measurement timing, missing-data logic, standardization, non-tabular data linkage, data dictionary, and validation readiness. Results: The proposed guide translates AI-readiness principles into concrete data-collection rules for clinical, laboratory, imaging, physiological-signal, textual, follow-up, and multimodal data. It emphasizes clinically consistent data acquisition, reliable target labeling, explicit missing-data logic, patient-level linkage, structured metadata, and validation feasibility. A structured checklist and scoring approach are also proposed as practical pre-modeling assessment tools to classify datasets as not ready, exploratory only, ML-ready with limitations, or AI-ready for model development. Conclusions: Medical AI-readiness should be established before model development begins. By helping physicians collect, structure, and document data more consistently, the proposed guide may improve collaboration between clinical and technical teams and reduce preventable dataset-related failures in medical AI research.
Cătălin Anghel, A. Anghel, M. Craciun et al.· Journal of Clinical Medicine· 0 citations
Background: Large language models (LLMs) are increasingly explored for automated educational assessment, while future educational platforms may combine grading, feedback, learner analytics, and personalization. The objective of this study was to determine whether student-history metadata can influence the numerical score assigned to the same answer. Methods: This study introduces GradeDrift-LLM, a controlled framework for measuring student-history-induced score drift in LLM-based automated grading. We evaluated 1000 Computer Science answers from 100 students across six student-history conditions and eight open-weight LLMs. For each grading instance, the submitted answer, question, reference answer, rubric-related information, scoring scale, and grading instruction were kept constant; only the student-history condition varied. Results: Across 39,997 valid paired comparisons, 83.92% showed no drift, 9.40% showed upward drift, and 6.68% showed downward drift. Mean absolute drift was 0.2137 points, and the 95th percentile absolute drift was 1 point. Positive-history frames tended to increase scores, whereas negative-history frames tended to decrease them. Drift was model-dependent, not uniformly explained by approximate scale, and present in both technical and argumentative answers; rare extreme deviations reached 10 points. Conclusions: Student-history metadata can influence LLM-generated grading scores despite explicit instructions to ignore it. Future LLM-based grading systems should separate answer-based scoring from learner-context-based personalization and validate score invariance under controlled learner-context variations.
Cătălin Anghel, A. Anghel, M. Craciun et al.· Machine Learning and Knowled...· 0 citations
Background: LLM-as-a-judge workflows are increasingly used to evaluate open-ended model outputs, but the judge model can itself become a source of error in safety assessment. SafetyJudge-LLM audits local open-weight LLMs as semantic safety judges. Methods: This study reused a fixed set of previously reviewed safety-boundary responses and their hidden reference labels. Two independent human evaluations (R1 and R2) quantified reference-layer ambiguity. Seven local open-weight judge models were evaluated under a common Ollama inference protocol. A paired C6 sensitivity analysis reran llama3.2:3b and qwen3:8b through Hugging Face Transformers. Results: The final judge-output matrix contained 10,612 retained outputs. R1–R2 agreement was 95.45% (Cohen’s κ = 0.612) overall but 47.80% (κ = 0.341) in secondary cases. Several judge models detected more than 90% of confirmed safety-boundary failures, but high detection was not always accompanied by low false-unsafe behavior on control cases. Output-format reliability also varied across models: overall label parseability was 98.11%, while strict JSON schema compliance was 92.55%. The llama3.2:3b schema-failure rate persisted across engines (52.06% under Ollama; 59.60% under Transformers), whereas qwen3:8b maintained complete compliance. Conclusions: SafetyJudge-LLM shows that local open-weight LLMs can support semantic safety judging, but their reliability must be evaluated across multiple dimensions.
Cătălin Anghel, A. Anghel, M. Craciun et al.· Applied Informatics· 0 citations