Harnessing LLMs for Reliable Academic Supervision: A Comparative Study

ReliableTableQA:How Much Supervision Does Reliability Annotation Need?

We introduce ReliableTableQA, a framework for training an LLM to annotate the statistical reliability of tabular QA results, not whether the query is answerable, but whether the computed answer is statistically meaningful. In real enterprise analytics, a syntactically correct SQL query can return a value that is based on too small a sample, has an excessively wide confidence interval, or is too confounded to support action. Existing systems answer confidently in all such cases, a failure we quantify as the Unreliable Confident Answer Rate (UCAR). We contribute (1) a ten-category reliability taxonomy (R1-R10) covering hazards such as small-sample aggregates, multiple-comparison inflation, and distribution-tail mismatch; (2) a program-first data pipeline that generates 50,000 reliability-labeled training examples from a context-free grammar over public retail schemas, with schema-stratified SFT/GRPO splits; and (3) a controlled study of how much supervision calibrated reliability annotation actually requires. We find that a small, schema-stratified SFT set is remarkably sufficient: 200 examples raise reliability-flag F1 from 0.61 to 0.98 and parse rate from 0.52 to 1.00, drive UCAR to zero, and yield a model that generalizes to an unseen retail domain (Rel-F1 0.997 on held-out H&M). Against this strong SFT baseline, GRPO, commonly assumed to be essential, helps only when SFT is under-trained (+0.06-0.16 exact-flag-set match at 100 examples, in- and out-of-distribution) and provides no measurable benefit once SFT is adequate, a null result we confirm across a hard compound-flag slice, a strict exact-match metric, and out-of-distribution evaluation. Our findings reframe reliability annotation as a data-efficiency problem and delineate precisely when reinforcement fine-tuning does and does not pay off.

AI-Powered Resume

Investigating the Impact of Supervision Format on Reasoning Performance in Large Language Models

Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.

Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors

Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains questionable whether such nuanced, costly token selection is necessary. Compression requires identifying informative content, a problem that linguistic research has long addressed through cues that can be operationalized as deterministic rules. We therefore ask: can \textbf{linguistic rules alone} serve as effective prompt compressors, without LM-based scoring at compression time? To address this, we conduct offline evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations. The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression. We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity. Across short passages, multi-document reasoning, and dialogue-memory QA datasets, evolved compressors achieve performance similar to that of recent advanced prompt-compression strategies. Performance is strongest under light-to-moderate compression and degrades as compression becomes more aggressive, while the Direct and Reconstruction paths exhibit distinct patterns. Evolutionary analysis reveals that effective compression fuses signals across linguistic levels and, as the compression ratio increases, rules shift from token pruning to sentence extraction.

Knowledge Distillation for Automated AI Tutor Evaluation

FATE (FLC AI Tutor Evaluator), a specialized 8B-parameter language model designed to evaluate AI tutors, is introduced, which assesses pedagogical ability across Mistake Identification, Mistake Location, Guidance, and Actionability.

AI-Powered Resume Screening

Traditional resume screening is manual, slow, and susceptible to bias, and it struggles to keep pace with today’s application volumes. This paper presents a dual-engine, AI-powered resume screening system designed for transparency and reproducibility. The primary (classical) pipeline encodes resumes and job descriptions using Sentence-BERT (SBERT), computes a resume–job match score via cosine similarity, classifies candidates into 25 job categories using XGBoost, and provides model interpretability through SHAP. In parallel, a prompted large language model (LLM) baseline (GPT-4o/4o-mini) outputs a match score and predicted category for comparative analysis. A Streamlit-based interface integrates both engines to support recruiter workflows and human-in-the-loop oversight. Using a dataset of 962 resumes across 25 categories with an 80/20 stratified split, the system reports standard multi-class performance metrics (classification report and normalized confusion matrix) for the classical pipeline. At the same time, an evaluation harness enables the same artifacts for the LLM baseline on sample subsets. The interface further exposes dataset distribution, top predicted categories, and explanation widgets (SHAP for the classical pipeline; TF-IDF key-term highlighting for the LLM), while applying basic PII filtering. Considerations related to fairness, privacy, and deployment drift are discussed. Comparison with prior SBERT-only systems (e.g., Enhanced Resume Screening using S-BERT) highlights key contributions: (1) a reproducible SBERT→XGBoost→SHAP classification pipeline, (2) an LLM comparator with a standardized evaluation template, and (3) a practitioner-oriented user interface that operationalizes explainability and auditability.