small language model

Fine-Tuning and Evaluation of LLMs

This chapter explains why fine-tuning is essential for adapting large language models (LLMs) to specialized and high-stakes domains, where general-purpose prompting may be unreliable. This chapter introduces fine-tuning as the process of updating a pretrained model with task-specific data and reviews evidence from healthcare, cybersecurity, ontology engineering, and transportation showing that fine-tuned models achieve higher accuracy, domain compliance, and practical usefulness. This chapter emphasizes that specialization must be paired with rigorous evaluation, highlighting current gaps in reporting on safety, fairness, and bias, and motivating the development of multidimensional assessment frameworks. It then presents a step-by-step data preparation pipeline for collection, cleaning, formatting, splitting, and privacy/bias checks, illustrated using a small synthetic university frequently asked question (FAQ) dataset. Next, it compares major fine-tuning strategies, including full-model fine-tuning, parameter-efficient methods such as low-rank adaptation (LoRA), and instruction tuning. A hands-on walkthrough demonstrates fine-tuning DistilGPT2 with LoRA and interpreting trainable-parameter efficiency. Finally, this chapter introduces intrinsic metrics (BLEU, ROUGE, METEOR, perplexity) and extrinsic metrics (accuracy, precision, recall, F 1) to assess robustness and real-world readiness.

Social Contract as a Generative Relational Process: Agency, Rule Formation, and Effective Freedom

Social-contract arguments often assign capacities, interpretations, options, and reasons to parties at a selected justificatory or bargaining stage. This analytical practice can be appropriate for its declared purpose. It can also omit two dependencies that matter for explanation and, under further premises, for normative assessment. Prior relations of care, education, language, recognition, material support, and institutional access can help form contract-capable agency. A selected rule can then reshape the legibility, cost, viability, and transition structure of later action. The parties and the rule environment consequently participate in a recursive process. This position paper proposes a bounded generative reconstruction. Its target is the class of contract representations that hold target-relevant agent capacities, interpretations, or effective action spaces invariant at the stage under analysis. The reconstruction distinguishes historical genesis, causal dependence, justificatory validity, authority, legitimacy, obligation, and justice. It also recognizes generative resources already present in the tradition. Rousseau connects association with a new collective body and moral liberty; Kant treats original contract as an idea of reason; Rawls uses an idealized justificatory representation within a wider theory of moral powers, social bases, and institutional formation. Relational-autonomy, dependency, capability, relational-sociology, and exclusion literatures supply further antecedents and constraints. The formal model represents participant development, a rule-environment transformation, and horizon-indexed viable trajectories. Effective freedom is reported through reachable trajectories, security, control, revision, and exit. Finite countermodels establish three separations within the proposed model class: a smaller nominal action set can accompany broader effective reach; perfect transition predictability can accompany unequal participant capability; and identical observed agreement can accompany different generative histories and later revision capacities. A higher-order interaction model separates constitutive contribution from legislative influence. Finally, a target-relative factorization criterion states when a static contract representation remains an adequate quotient of the generative model. The paper's contribution is methodological and conditional. A generative description can reveal omitted dependencies, while legitimacy and obligation continue to require independent normative premises. Empirical validation, approximate-adequacy metrics, and institutional applications remain open parts of the research programme.

When to Escalate: A Cost-Aware Belief Policy for Conversational Agents Under Hidden Intent

A conversational agent replying to inbound sales leads must choose an action before it knows the lead’s intent. It can answer, ask a qualifying question, hold, or escalate to a human, but the true buying-readiness and whether a human is actually needed are hidden. The two ways of being wrong are not equally costly. A missed escalation, where a lead who needed a person did not get one, is far more expensive than a needless one, and some wrong answers are not a cost to trade off at all but a line the agent must not cross. We build a small agent that makes this trade-off explicit. It holds a belief over the hidden state, split into readiness (hot, warm, cold) and a separate probability that the case needs a human, and at each message it picks the action with the lowest expected cost under a cost matrix that encodes the asymmetry. Escalation is split into notify, where a human is told while the conversation continues, and pause, where the agent stops, because the two carry different costs. The hardest wrong answers, such as false legal or land claims, are enforced as a hard constraint rather than a priced term. We test on 100 synthetic cases with cached language-model beliefs, comparing the cost-aware policy against the same policy with a uniform cost matrix and against three fixed-action baselines. Two of those baselines give opposite answers, so both are named wherever the cost result appears. Against the uniform-cost version of the same rule — same belief, same feasible set, only the asymmetry removed — the cost-aware policy reaches 1.72 against 2.58 over all 100 cases, a reduction of a third; that gap is what cost-awareness buys, and it is against that baseline and no other. Against always-notify, which escalates every message, there is no cost win: 1.72 against 1.74 over the same 100 cases is a near-tie, a total of 172 against 174, so the cost-aware policy is not the cheaper policy and is not claimed to be. What it buys there is human load rather than cost — over all 100 cases it reaches the same cost while escalating 43 times rather than 100, at an escalation precision of 0.605 against 0.420. The uniform-baseline gap is also not all machinery: that baseline selects the same action as a plain 0.5 threshold on all 100 cases, and reweighting the case set toward the design’s own readiness prior moves the gap from 0.86 to 1.07 rather than closing it, so the direction is robust to reweighting even as the magnitude shifts. Every missed escalation traces to one cause: the belief under-estimated the needs-human probability rather than misreading readiness. Expected calibration error on that marginal is 0.142 (bootstrap 95% CI [0.100, 0.249]), concentrated in the bins adjacent to the escalation threshold rather than the bin containing it; recalibrating that marginal on the same 100 cases is an in-sample ceiling, and the body reports it as one. Held out, the result is different in kind. Re-eliciting bh from the model’s digit logprobs, fitting a map on 50 development cases and scoring the other 50, improves all three calibration measures named in advance — on those 50 test cases expected calibration error falls from 0.1526 to 0.0696, cross-entropy from 0.8546 to 0.8136 bits, and Brier from 0.2063 to 0.1962. Better calibration does not buy a better decision. The fitted map is isotonic, and the lowest block it pools sets a reachable-score floor of 6/23 ≈ 0.2609, above the 3/13 ≈ 0.2308 a belief must fall below for answering to be cheapest, so the threshold sits inside an interval the map cannot emit: on those same 50 cases the policy stops choosing answer at all, escalation precision falls from 0.667 to 0.463, and recall rises from 0.667 to 0.905. The cause is the map’s range rather than the scores it produced, and a calibration-quality gain that the range prevents from becoming a decision gain is the finding. We also report a failure the test set cannot produce by construction, a high-cost message dropped when it arrived batched with a routine one, observed in a live run. Code and data are available.

DDA: a traceable multi-agent framework for automated structure-based drug design

Structure-Based Drug Design (SBDD) is a key computational paradigm that uses protein structural information to design and optimize small molecules with desired binding properties. Existing SBDD methods mainly focus on molecular design and often lack the capability to independently conduct downstream evaluation and optimize workflows. Meanwhile, directly applying large language models (LLMs) to molecular design faces challenges such as fragmented execution, limited tool coordination, and insufficient traceability of intermediate decision-making processes. To address these limitations, we propose a Drug Discovery Agent (DDA), a traceable and auditable multi-agent biomedical informatics framework for automating SBDD. DDA uses a role-specific multi-agent architecture to transform natural-language drug-discovery goals into actionable scientific workflows. Through a unified tool-calling protocol, this framework seamlessly integrates bioinformatics, molecular modeling, and molecular docking modules, enabling autonomous workflow execution from target preparation and molecular generation to multi-objective evaluation, candidate prioritization, and trajectory tracking. We systematically evaluated DDA on the CrossDocked2020 benchmark. Under the closed-loop delivery protocol, the framework produced 2,000 final candidate records, with a joint screen-pass rate 20 percentage points higher than that of the strongest specialized baseline. These results indicate that DDA provides a scalable, executable, and traceable computational framework that reduces manual coordination in structure-based automated drug discovery and delivers prioritized candidate sets with minimal human intervention.

Policy-TD: Teacher-Distilled Runtime Control for Frozen Small Language Models

A survey of edge–cloud collaborative large language models for discrete manufacturing: Architectures, technologies, and applications

HIEF: An Interpretable Evidence-Fusion Framework for Phishing Email Detection with Decomposable Decision Uncertainty and a Preliminary English–Spanish Evaluation

(1) Background: Phishing remains a pervasive and economically damaging cyberthreat. The dominant detection paradigm has moved toward deep neural and transformer-based classifiers, a literature that reports high accuracy and that does not, in general, expose a per-decision justification, whereas interpretability and auditability are increasingly required in regulated environments; no comparison against transformer-scale detectors is made in this paper. This work asks how far a fully interpretable detector can close the accuracy gap to an opaque text classifier while preserving per-decision explanations, and what such a detector returns that accuracy alone does not measure. (2) Methods: HIEF, an interpretable evidence-fusion framework, is presented. Each email is represented by eighteen human-readable signals: fourteen structural and linguistic cues and four lexical aggregates derived from a published sparse log-odds lexicon. The signals are fused by three transparent layers, namely an L1-regularized logistic model, a shallow interaction-rule tree, and a calibrated Dempster–Shafer stage that reports belief, disbelief and ignorance masses together with an order-invariant global conflict coefficient derived in closed form. A logistic meta-learner fitted on out-of-fold component scores integrates the three layers. The evidential layer uses a type-aware calibration in which discrete signals are calibrated on their attainable values and continuous signals by isotonic regression. Evaluation uses 38,908 public emails, 38,512 of them after exact-duplicate removal, with near-duplicate control, group-aware partitioning, ten repeated splits, a source-held-out protocol, a two-class cross-source test set, a component ablation and a human audit of 100 messages annotated independently by two evaluators. (3) Results: Under group-aware partitioning, HIEF attains an F1 of 0.855 and the strongest term frequency–inverse document frequency (TF–IDF) baseline 0.954; a compact character n-gram neural reference model, evaluated over the same ten partitions, attains 0.973. The linear layer alone attains 0.872, so the two fusion layers do not improve accuracy over it, and the paired difference of 0.017 excludes zero. Type-aware calibration raises the evidential layer from 0.771 to 0.780 and more than halves its partition-to-partition standard deviation, but does not make it competitive; the weakness, therefore, lies in the fusion formulation rather than in the binning. What the evidential layer does supply is a decomposable account of decision uncertainty: the ignorance mass separates errors from correct decisions, 0.265 against 0.175. The human audit reaches an inter-annotator Cohen’s kappa of 0.950 over the five categories before adjudication, and shows that the permissive corpus label agrees with human phishing judgment at a Cohen’s kappa between 0.18 and 0.21, against 0.70 to 0.77 for the automatic strict rule; the audited block is annotated by two of the authors and its human positives are confined to the advance-fee family, so the audit is a bounded comparison of label assignments and not an independent annotation study. (4) Conclusions: HIEF is positioned as an uncertainty and explanation framework rather than as an accuracy-improving fusion method, since the measured accuracy cost of the fusion layers is not compensated by an accuracy gain. Quantifying how much of the performance reported on these widely used corpora is attributable to template leakage and to label permissiveness is a contribution independent of the detector itself. Cross-source operation has not been demonstrated: specificity falls to 0.041 on an unseen collection, so all evaluation reported here is proof-of-concept and no operational deployment claim is made. The Spanish-language evaluation rests on a small and entirely positive subset and is reported as preliminary.

IOLM-DB: Instance-optimized large language model execution for column-scale semantic operators

Large language models (LLMs) can implement row-level semantic transformations over tabular data, but a general-purpose model reserves the same accelerator resources even when every row uses one fixed prompt. IOLM-DB compiles that recurring prompt–column pair into an operator sized to the work it actually does. It samples the target column, constructs calibration sequences that represent both prompt and output behavior, evaluates specialized candidates, and selects an implementation under explicit memory and quality constraints. Across heterogeneous datasets and operator types, column-calibrated quantization reduces the resident footprint of the reference model by 1.8–2.8 × while the 8-bit profile remains a near-lossless replacement and the 4-bit profile preserves behavior on categorical label-output operators. At fixed precision, column calibration consistently improves fidelity over generic GPTQ calibration, and output-aware calibration provides an additional benefit for generative operators. Deployment experiments reveal two complementary outcomes: weight-only compression does not raise per-row throughput in a compute-rich regime, where it primarily releases capacity, but the smaller weight stream also improves throughput when memory bandwidth becomes limiting. The compact artifacts enable deployment under a memory budget where the full-precision operator cannot start and allow three specialized operators to occupy roughly the footprint of one full-precision model. IOLM-DB therefore turns a recurring semantic transformation from an immutable call to a general-purpose backend into a compact, measurable, and hardware-aware operator.

When to Escalate: A Cost-Aware Belief Policy for Conversational Agents Under Hidden Intent

A conversational agent replying to inbound sales leads must choose an action before it knows the lead’s intent. It can answer, ask a qualifying question, hold, or escalate to a human, but the true buying-readiness and whether a human is actually needed are hidden. The two ways of being wrong are not equally costly. A missed escalation, where a lead who needed a person did not get one, is far more expensive than a needless one, and some wrong answers are not a cost to trade off at all but a line the agent must not cross. We build a small agent that makes this trade-off explicit. It holds a belief over the hidden state, split into readiness (hot, warm, cold) and a separate probability that the case needs a human, and at each message it picks the action with the lowest expected cost under a cost matrix that encodes the asymmetry. Escalation is split into notify, where a human is told while the conversation continues, and pause, where the agent stops, because the two carry different costs. The hardest wrong answers, such as false legal or land claims, are enforced as a hard constraint rather than a priced term. We test on 100 synthetic cases with cached language-model beliefs, comparing the cost-aware policy against the same policy with a uniform cost matrix and against three fixed-action baselines. Two of those baselines give opposite answers, so both are named wherever the cost result appears. Against the uniform-cost version of the same rule — same belief, same feasible set, only the asymmetry removed — the cost-aware policy reaches 1.72 against 2.58 over all 100 cases, a reduction of a third; that gap is what cost-awareness buys, and it is against that baseline and no other. Against always-notify, which escalates every message, there is no cost win: 1.72 against 1.74 over the same 100 cases is a near-tie, a total of 172 against 174, so the cost-aware policy is not the cheaper policy and is not claimed to be. What it buys there is human load rather than cost — over all 100 cases it reaches the same cost while escalating 43 times rather than 100, at an escalation precision of 0.605 against 0.420. The uniform-baseline gap is also not all machinery: that baseline selects the same action as a plain 0.5 threshold on all 100 cases, and reweighting the case set toward the design’s own readiness prior moves the gap from 0.86 to 1.07 rather than closing it, so the direction is robust to reweighting even as the magnitude shifts. Every missed escalation traces to one cause: the belief under-estimated the needs-human probability rather than misreading readiness. Expected calibration error on that marginal is 0.142 (bootstrap 95% CI [0.100, 0.249]), concentrated in the bins adjacent to the escalation threshold rather than the bin containing it; recalibrating that marginal on the same 100 cases is an in-sample ceiling, and the body reports it as one. Held out, the result is different in kind. Re-eliciting bh from the model’s digit logprobs, fitting a map on 50 development cases and scoring the other 50, improves all three calibration measures named in advance — on those 50 test cases expected calibration error falls from 0.1526 to 0.0696, cross-entropy from 0.8546 to 0.8136 bits, and Brier from 0.2063 to 0.1962. Better calibration does not buy a better decision. The fitted map is isotonic, and the lowest block it pools sets a reachable-score floor of 6/23 ≈ 0.2609, above the 3/13 ≈ 0.2308 a belief must fall below for answering to be cheapest, so the threshold sits inside an interval the map cannot emit: on those same 50 cases the policy stops choosing answer at all, escalation precision falls from 0.667 to 0.463, and recall rises from 0.667 to 0.905. The cause is the map’s range rather than the scores it produced, and a calibration-quality gain that the range prevents from becoming a decision gain is the finding. We also report a failure the test set cannot produce by construction, a high-cost message dropped when it arrived batched with a routine one, observed in a live run. Code and data are available.

Semantic anchoring with concise ideal answers outperforms unstructured full course materials as context for multi-LLM automated grading of open-ended questions

Abstract Large language models (LLMs) are increasingly used to grade open-ended student responses, yet the role of contextual input in this process remains poorly understood. This study compares three context conditions for multi-LLM automated grading: no context, full course materials, and instructor-defined ideal answers as semantic anchors. Using a dataset of 3,041 student responses (3,011 after common-support exclusions) to 50 open-ended questions from an undergraduate computer science course, we evaluated three base LLMs (DeepSeek, Qwen, Gemini) against grades derived from two independent blind instructor assessments. A factorial analysis based on the Aligned Rank Transform revealed significant main effects of model and condition, with a significant interaction. Ideal-answer anchoring significantly outperformed both alternatives in absolute grading error, while providing full course materials significantly worsened accuracy relative to the no-context baseline. The anchored condition achieved the lowest mean absolute error (1.268), the highest correlation with instructor grades ( r = 0.801), and the lowest inter-model disagreement (median SD = 0.864), at a per-response cost comparable to the no-context baseline (EUR 0.00119 vs. 0.00113) and 4.2 times cheaper than the full-materials condition (EUR 0.00501). The absolute accuracy gain over the no-context baseline is small (≈ 0.08 points on a 0–10 scale; marginal R 2 = 0.013); its practical value lies in the convergence of accuracy, inter-model agreement, feedback-quality and cost improvements and in avoiding the accuracy loss caused by unstructured full course materials. A complementary analysis of 27,099 feedback instances, validated against a human gold standard (κ = 0.898), showed that out-of-scope feedback decreased by 25.4% under semantic anchoring, with a logistic regression revealing that this benefit was concentrated in two of the three evaluators. Evidence derives from a single course, institution, language (Spanish) and academic year, and from convergent-answer theoretical assessment; within this setting, concise instructor-defined ideal answers (rather than large volumes of unfiltered course material) yielded the most reliable grading and feedback.

Social Contract as a Generative Relational Process: Agency, Rule Formation, and Effective Freedom

Social-contract arguments often assign capacities, interpretations, options, and reasons to parties at a selected justificatory or bargaining stage. This analytical practice can be appropriate for its declared purpose. It can also omit two dependencies that matter for explanation and, under further premises, for normative assessment. Prior relations of care, education, language, recognition, material support, and institutional access can help form contract-capable agency. A selected rule can then reshape the legibility, cost, viability, and transition structure of later action. The parties and the rule environment consequently participate in a recursive process. This position paper proposes a bounded generative reconstruction. Its target is the class of contract representations that hold target-relevant agent capacities, interpretations, or effective action spaces invariant at the stage under analysis. The reconstruction distinguishes historical genesis, causal dependence, justificatory validity, authority, legitimacy, obligation, and justice. It also recognizes generative resources already present in the tradition. Rousseau connects association with a new collective body and moral liberty; Kant treats original contract as an idea of reason; Rawls uses an idealized justificatory representation within a wider theory of moral powers, social bases, and institutional formation. Relational-autonomy, dependency, capability, relational-sociology, and exclusion literatures supply further antecedents and constraints. The formal model represents participant development, a rule-environment transformation, and horizon-indexed viable trajectories. Effective freedom is reported through reachable trajectories, security, control, revision, and exit. Finite countermodels establish three separations within the proposed model class: a smaller nominal action set can accompany broader effective reach; perfect transition predictability can accompany unequal participant capability; and identical observed agreement can accompany different generative histories and later revision capacities. A higher-order interaction model separates constitutive contribution from legislative influence. Finally, a target-relative factorization criterion states when a static contract representation remains an adequate quotient of the generative model. The paper's contribution is methodological and conditional. A generative description can reveal omitted dependencies, while legitimacy and obligation continue to require independent normative premises. Empirical validation, approximate-adequacy metrics, and institutional applications remain open parts of the research programme.

A Longitudinal Exploratory Case-Series Comparing Morphological Trajectories in Four Early-Implanted Greek-Cypriot Children and Eight Typically Hearing Peers

Objectives: This paper explores whether deaf Greek-Cypriot (GC) children with cochlear implants (CIs) begin using the various forms of the definite article (DA) of the Greek language similarly to their typically hearing peers in terms of order of appearance and age. Furthermore, it explores if their MLUm growth correlates with that of typically developing peers. Methods: This is a longitudinal study with quantitative and qualitative analyses. The sample comprises four prelingually deaf GC children with CIs and eight typically hearing (TH) children with no developmental delays. Age and hearing status (CIs vs. TH) were used to explain variability in results. The effect of random effects was also examined. Presence of random effects was examined using Mixed Linear Models. Growth trajectories were plotted for in-depth result description and interpretation. Results: Both groups demonstrated a significant linear increase in their accurate usage of definite articles with age, whereas hearing status showed no statistically significant impact on any outcome measure. Individual variance among children accounted for approximately 33% to 50% of the variance in correct article production. Additionally, Mean Length of Utterance in morphemes (MLUm) values between the two groups remained correlated across all age levels. Discussion: While random individual effects influence the rate of correct production, early morphosyntactic trajectories for CI children seem to be similar to those of typically developing controls. Due to the small sample size, studies with bigger samples are needed to replicate, confirm and validate further the results of the current study.