Aug 2026· IEEE Symposium on Artificial Life· 0 citations
TL;DR
It is demonstrated that transformer derived surprisal systematically distinguishes functionally constrained sequences from unconstrained ones across digital and biological substrates, establishing transformer surprisal as a potential measure of functional information in evolving systems.
Abstract
To resist thermodynamic disorder, living systems maintain low entropy genomic configurations through selection. We propose pseudo-perplexity, the per position surprisal of a masked language model, as an estimator of this biological functional information. Using a unified framework, we demonstrate that transformer derived surprisal systematically distinguishes functionally constrained sequences from unconstrained ones across digital and biological substrates. In evolved populations of 64-byte BFF replicators, a fine-tuned GPT-2 assigns significantly lower pseudo-perplexity to conserved opcode positions than to variable regions (Mann-Whitney U, r = 0.99). A fine-tuned Nucleotide Transformer reproduces this pattern for bacterial 16S rRNA sequences (p = 1 × 10−16, r = 0.2), assigns lower pseudo-perplexity to essential than non-essential genes (p = 6.1 × 1020, r = 0.14), and reveals a negative power law relationship between genomic pseudo-perplexity and essential gene fraction across 12 organisms (Spearman Ï = −0.58, p = 0.048). Finally, pseudo-perplexity of variable regions increases with phylogenetic distance while conserved regions remain relatively stable, recapitulating a diffusion-like drift in both digital and biological lineages. These results establish transformer surprisal as a potential measure of functional information in evolving systems.
Data/Code available at: https://github.com/tydymy/bff-functional-information
StabilityArc is presented, which maps frozen ESMC-600M residue representations through a shared RoPE transformer to an Lx20 matrix of substitution effects; a symmetric, contact-aware residual aids in predicting epistasis in simultaneous substitutions.
Aaron L. Feller, Andrew D. Ellington, Claus O. Wilke· 0 citations
PyScale is an algorithmic toolkit that curates nucleotide propensity scales and provides reproducible software to transform DNA or RNA sequences into time-series-like profiles for statistical analysis and can be seamlessly adapted for protein sequence analysis.
A. Chattopadhyay, Yeganeh Abdollahinejad, D. Flower· IEEE Access· 0 citations
A thermodynamic decomposition of the human mutation field is given: most mutation bias is organized by a local equilibrium-like energy landscape aligned with genome composition, while the residual circulation points to specific directional mutational mechanisms.
Isabella Caranzano, D. M. Busiello, Stefano Priorelli et al.· 0 citations
The discordance persists: pLDDT correlates positively with PUNCH2 disorder in random and de novo proteins and negatively with β-strand fraction, opposite to the conserved and disordered baselines, a concrete failure mode that protein designers and other working on sequences remote in sequence space should be aware of w...
Lars A. Eicholt, Lasse Middendorf· bioRxiv· 0 citations
PHACTn (Phylogeny-Aware Computing of Tolerance for nucleotide variants), a training-free, parameter-minimal method that infers nucleotide variant tolerability by traversing the mammalian phylogenetic tree and explicitly modeling the evolutionary independence of observed substitutions and their distance from the query s...
Ceren Yıldırım, Nurdan Kuru, Ogun Adebali· bioRxiv· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.