农科(蔬菜学·茶学)学术英语语料库 v1.2 · 文档与代码包 / Agricultural English Corpus v1.2 (Horticulture & Tea Science) · Documentation and Scripts
Abstract
本记录为该语料库 v1.2 的文档与代码包;语料数据包以受限访问发布,见关联记录(概念 DOI 10.5281/zenodo.23066607)。 内容:字段字典、Data Statements、句级表说明、标注库说明、语步标注覆盖率/合并/全量重跑审计报告与未匹配清单、CN-1280 缺陷复核与处置建议、方案 B 语料复核报告、数据来源与生成方式声明、机器翻译痕迹检测方案与结果、结构化摘要分层报告、语料残留标记排查报告、v1.2 冻结清单(逐文件 SHA256 与包级哈希)、打包校验报告、基础库期刊×年份×学科聚合统计、处理与核查脚本及脚本清单。本记录为文档与脚本,不含语料正文。 规模(v1.2):中文期刊论文 2,765 篇、中英句对 13,028 对、英文单语 1,783 篇、中文单语 56 篇、文献主表 4,604 条;句级表 53,303 行;语步标注 53,303 行(覆盖率 99.9%)。 本版(v1.2)修订——自 v1.0.3 起累计两批改动,均在此如实披露: 一、v1.1(语料层,本版一并发布) ① 剔除 EN-1601(Horticulture Research 2023,DOI 10.1093/hr/uhad149):该篇源件的 abstract 字段实为整篇通信(Letter)正文 10,629 字符(开头 Dear Editor,无 Abstract 段),按本库"语料单元=摘要"的口径整篇剔除并留档;全库复扫此类声明块标记仅此 1 篇。⇒ 文献主表 4,605→4,604、摘要表 7,370→7,369、英文单语 1,784→1,783、英文句数 40,016→39,935;中英句对 13,028 不变。② 正式并入"句级全量表"(一个句子一行、覆盖全部摘要):53,304 行(中文 13,369+英文 39,935)。句级统计一律以它为分母;句对表是"配对视图"(只收配对得上的句子),不等于全库句数。 二、v1.2(本版新增) ① 修复 CN-1280 的"中英摘要粘连"(《中国蔬菜》2025):该篇源件的英文字段实为"中文摘要 432 字+英文摘要 1,477 字"粘连成一行(1,910 字符、汉字占比 0.1272),被当作英文摘要入库,并在句级表里切出一句中文的"英文句序1",使该篇 5 个句对整体错位一格。已剥离为纯英文(1,477 字符,SHA256 前 16 位 604a77c7f4665a96),并以差量核对证明全库只有该篇英文侧变动。⇒ 句级表 53,304→53,303(英文 39,935→39,934)、该篇英文句数 8→7;句对表 13,028 不变、摘要表 7,369 与主表 4,604 不变。② 新增"语步标注表"(立项书"标注库"的语步层):用 SciAIEngine(move_recognition_cn/_en)对全部摘要做全量重跑(中文 2,821 篇+英文 4,547 篇=148 次调用,未沿用任何旧批标注),53,303 行、覆盖率 99.9%;未匹配 61 行(0.11%)留空并列明清单。四级匹配口径(精确→继承(引擎合并)→继承(引擎拆分)/多标签-取最长→近似≥0.6)会造成标签一对多,方法节须披露。 三、本版规模:文献主表 4,604 条/中英句对 13,028 /摘要表 7,369 行/句级表 53,303 行/语步标注 53,303 行。 说明:v1.0.3 与 v1.2 之间的 v1.1 未单独发布到 Zenodo,其改动已并入本版;上一版(v1.0.3)及其之前版本仍可访问,用于追溯修订过程。 English This record is the documentation and scripts package for version v1.2 of the corpus. The corpus data package is released with restricted access — see the related record (concept DOI 10.5281/zenodo.23066607). Contents: field dictionary; Data Statements; sentence-table note; annotation-library note; move-annotation coverage, merge and audit reports with the unmatched-rows list; CN-1280 defect review; corpus verification report; data-source and generation statement; machine-translation-trace detection report; structured-abstract stratification report; residue-marker audit report; v1.2 freeze list (per-file SHA256 and package-level hash); packaging verification report; aggregated basic-library statistics; processing and verification scripts with an inventory. Documentation and code only, not the corpus text. Changes in v1.2 — two cumulative batches of changes since v1.0.3, both disclosed here: (1) v1.1 (corpus layer, released together with this version) (1a) Removal of EN-1601 (Horticulture Research 2023, DOI 10.1093/hr/uhad149): its source `abstract` field was in fact the full text of a Letter (10,629 characters, beginning with `Dear Editor`, with no abstract paragraph); the record was removed entirely and archived, in line with this corpus's unit-of-analysis definition (abstract, not full text). A corpus-wide rescan found this class of marker in this one record only. ⇒ master table 4,605→4,604, abstract table 7,370→7,369, English-only records 1,784→1,783, English sentences 40,016→39,935; aligned pairs unchanged at 13,028.(1b) The sentence-level full table was formally added (one sentence per row, covering all abstracts): 53,304 rows (13,369 Chinese + 39,935 English). Sentence-level statistics use this table as denominator; the pair table is a paired view only and does not represent all sentences. (2) v1.2 (added in this version) (2a) Repair of a Chinese–English abstract concatenation in CN-1280 (《China Vegetables》2025): the English field actually contained the Chinese abstract (432 characters) concatenated with the English abstract (1,477 characters) in a single string (1,910 characters, 12.7% Chinese characters). It had been indexed as an English abstract and produced a Chinese "English sentence #1", shifting all five sentence pairs of that record by one. The field was split to pure English (1,477 characters; SHA256 prefix 604a77c7f4665a96); a differential audit confirms that no other record changed. ⇒ sentence table 53,304→53,303 (English 39,935→39,934), that record's English sentence count 8→7; pairs unchanged at 13,028; abstract table (7,369) and master table (4,604) unchanged.(2b) Move-annotation table added (the move layer of the annotation library): a complete re-run with SciAIEngine (`move_recognition_cn` / `_en`) over all abstracts (2,821 Chinese + 4,547 English = 148 API calls; no labels carried over from earlier batches): 53,303 rows, 99.9% coverage; 61 rows (0.11%) could not be matched reliably, are left empty and are listed in the unmatched-rows file. The four-level matching scheme (exact → inherited from a merged engine segment → inherited from split segments / longest label → approximate ≥0.6) produces one-to-many labels, which must be disclosed in the methods section. (3) Size of this version: master table 4,604 records / aligned pairs 13,028 / abstract table 7,369 rows / sentence table 53,303 rows / move annotations 53,303 rows. Note: the intermediate v1.1 was not released separately on Zenodo; its changes are included here. Previous versions remain accessible for provenance.
