quant_eval v7.21: Per-Case Evaluation Results and Run Provenance for Mistral-Nemo-Instruct-2407

Abstract

Complete per-case evaluation output and run provenance for the two quant_eval version 7.21 runs that underlie the whitepaper "quant_eval: A Behavioral Evaluation Harness for Full-Weight and Quantized Large Language Models." Every pass rate, bucket score and latency figure reported in that paper can be recomputed from these files, and every individual failure inspected at the row level. CONTENTS 126 scored cases of Mistral-Nemo-Instruct-2407 across three inference backends, with 72 recorded signals per case: - Run 20260211_022944, a side-by-side comparison of FP16 on Hugging Face Transformers against Q4_K_M on llama.cpp. 84 rows.- Run 20260629_184555, a W4A16 AWQ build evaluated on vLLM. 42 rows. Each run's provenance manifest is included alongside its results. The manifests record the harness version, the fixture set and its SHA-256 digest, the random seed, the runner configuration, and, for the GGUF pair, the byte counts and SHA-256 digests of the evaluated model artifacts. Both runs executed the identical fixture set, golden_oracle_fixtures_v7_21 (SHA-256 prefix 6d71a0b9147c), at seed 42, across eight test families: json, json_multistep, stateful_followup, mixed_brief_json, mcq, toolcall, toolcall_only, and a 20-case property-based fuzz regression suite. SCOPE AND LIMITATIONS This is not part of the quant_eval public corpus. That corpus (datasets D0-D7) reports production evaluations under quant_eval v7.22 and is deposited separately under concept DOI 10.5281/zenodo.22009419. The two should not be merged or compared. The methodology used here was retired. Both runs used the fast_gate screening profile, which evaluates a small number of cases per family to triage models quickly rather than to establish statistical confidence. The harness marks every family verdict in these runs provisional, and four of the eight families carry five or fewer cases per runner. fast_gate was retired immediately after the whitepaper was written. The three runners used three different inference backends, so the wall-clock timings recorded here combine a backend change with a precision change and are not a measurement of quantization speedup. The full-weight lane of run 20260211_022944 reused results cached by an earlier run rather than re-executing; the manifest records this and the README explains how it is visible in the data. The run manifests have been sanitized to remove machine-local filesystem paths. No run identifier, timestamp, seed, fixture hash, artifact digest or cache-reuse field was altered. The CSVs are deposited exactly as the harness wrote them. README.md documents the file layout, the scoring signals, the complete pass-rate tables, and a runnable verification snippet. No model weights are redistributed. The evaluated models remain under their own licences. quant_eval itself is proprietary software of PBH Applied Systems, LLC and is not covered by this deposit's licence.