Skip to content

Author

Zachary Shinnick

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#machine learning Preprint Sep 2026

Lasting Effects of Abstract Pretraining Beyond Perplexity

Language models are typically pretrained from random initialization. Recent work challenges this convention, showing that a brief warm-up on abstract, algorithmically generated data can provide a better starting point for subsequent learning of natural language. In this paper, we show that in small language models, suc...

Zachary Shinnick, Hemanth Saratchandran, Damien Teney et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Procedural Pretraining for Molecular Property Prediction

Molecular property prediction is often limited by the small size of labeled downstream datasets, motivating pretraining on large corpora of unlabeled molecules. In this work, we ask whether useful inductive biases can instead be learned from abstract, procedurally generated data before a model sees any molecular data....

M. Friedemann, Zachary Shinnick, Philip H. S. Torr et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.