GIS-IR: Execution-Verified Multi-Step Spatial Analysis Workflows with Small Language Models — Code, Corpus and Evaluation Records (v1.0)
Abstract
This archive seals the v1.0 experimental state of the GIS-IR study onexecution-verified multi-step spatial analysis workflows with smalllanguage models. It accompanies a manuscript in preparation: "ExecuteFirst, Verify Always: Verifiable Multi-Step GIS Workflows from a 0.6BLanguage Model". Contents (two bundles; SHA-256 listed below): 1. gisir-code-v1.0.zip - core package: declarative workflow IR schema, static checker, deterministic executor, spatial assertion and invariant library, L1-L4 judge; training/evaluation/audit scripts; tests and examples; the frozen v4.1 training corpus (83 examples) and the dosage/ablation corpora; pinned environment. MIT licensed. 2. gisir-reports-v1.0.zip - sealed main results table; machine L3/L4 judge records; 27 capacity-experiment report JSONs including three-seed true-L2 re-measurements; best-of-N records; verbatim run logs; the two-factor figure; and the complete artifacts of the task-08 spatial-correctness worked example. CC BY 4.0. Model weights (full SFT of Qwen3-0.6B on the frozen corpus, and thecapacity-control LoRA adapter for Qwen3-8B) are NOT included in v1.0 andwill be added in a subsequent version. Both are derivatives of Qwen3(Apache 2.0). Headline results verifiable from the records: 0.6B and 8B tie on full-chainL2 (4/17 vs 4/17) under a preregistered criterion despite a ~13x capacitydifference; a three-seed corpus-quality ablation with no reversal;verifier-selected best-of-8 raises 11% to 50%; task-08 IoU 1.000 (IR) vs0.711 (direct-tool baseline). The archive also contains the report-levelevidence for an audit finding that GeoAnalystBench's reported metrics lackexecution gating. All training and evaluation were performed on a single 8 GB consumer GPU.See README.md, REPRODUCIBILITY.md and ARCHIVE_MANIFEST.md inside the codebundle for the claim-to-evidence map, exclusions (third-party benchmark dataand raw course materials are not redistributed; upstream links given), andhonesty notices. SHA-256:17D76DDC402CF1CAA3CE0F0392E8838E443DD9FD9277C5F8DCA4641D1B4FAFAF gisir-code-v1.0.zip79FED235D3964DE1997BEA6E82B9CDC29CE567D9FE3E825AED78265A3BE2F25C gisir-reports-v1.0.zip
