Skip to content

Author

Zachary Shinnick

We have 4 of 7 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#machine learning Preprint Nov 2026

Language Models Can Benefit from Domain-Specific Inductive Biases

This work trains small language models from scratch on representative subsets of pretraining data and introduces variants of transformers in which standard activation functions are replaced with learned splines tuned on specific datasets, providing a simple way to modify their inductive biases.

Damien Teney, Liang-Ze Jiang, Zachary Shinnick et al. · 0 citations
#artificial intelligence Preprint Jun 2026

A Language Model from 1913: Pretraining on Historical Text

This work constructs TypewriterCorpus, a 54B-token historical corpus with extensive temporal filtering, proposes lexically grounded instruction tuning that constrains all responses to vocabulary from historical source documents, and introduces History-Event, a benchmark of 2,344 events for evaluating both competence an...

Xiaoxi Luo, Zachary Shinnick, Niclas Griesshaber et al. · 4 citations

Lasting Effects of Abstract Pretraining Beyond Perplexity

The results show that early abstract training can reliably shape the capabilities language models later acquire, and in small language models, such a warm-up improves specific capabilities that are not reflected in language-modeling perplexity.

Zachary Shinnick, Hemanth Saratchandran, Damien Teney et al. · 1 citation

Procedural Pretraining for Molecular Property Prediction

It is found that procedural pretraining can improve molecular property prediction even after subsequent molecular pretraining, and procedural data can provide transferable structure for molecular learning and offer a complementary route to improving performance when labeled molecular data are limited.

M. Friedemann, Zachary Shinnick, P. Torr et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.