It is shown that single-sequence PLMs can perform in-context peptide learning without gradient updates, task-specific retraining, or architectural modification, and MPEP conditioning is established as a lightweight strategy for low-data peptide classification.
Abstract
Protein language models (PLMs) are trained primarily on individual protein sequences, yet many peptide-discovery problems require inference from only a small number of labeled examples. Here, we show that single-sequence PLMs can perform in-context peptide learning without gradient updates, task-specific retraining, or architectural modification. We introduce multi-peptide example prompts (MPEPs), in which demonstration peptides are concatenated with glycine spacers and used as context for scoring query peptides by their prompted probability. We evaluate this approach across a synthetic pattern-completion task, secondary-structure classification, and MHC-II binder prediction using both encoder-only ESM-2 models and decoder-only ProGen2 models. Across tasks, performance improves with the number of peptide examples and with model scale, indicating that PLMs can extract shared sequence-level properties from prompted examples. We further introduce a difference score that contrasts positive-example and negative-example MPEPs, reducing compositional biases in raw PLM probabilities and substantially improving classification. On MHC-II binder prediction, MPEP-based classification with larger ESM-2 models matches or exceeds low-data classifiers trained on frozen ESM-2 embeddings, while requiring no training. These results reveal an unexpected in-context inference capability in single-sequence PLMs and establish MPEP conditioning as a lightweight strategy for low-data peptide classification.
Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology. These models, trained on large corpora of protein sequence data, are widely used to translate amino acid sequences into latent-space embeddings, ready for use in diverse downstream tasks (DTs). By a common consensus, embeddings from the model's last layer are used, and the model's internal behavior remains poorly understood. We analyzed 13 PLMs across 15 DTs from 11 datasets to investigate the informativeness of embeddings created in intermediate PLM layers. We trained probe models on embeddings from each layer, compared their performance, and computed characteristics of the latent spaces they span to estimate the information they contain, and found that the last layers of PLMs rarely contained embeddings that led to the best results on downstream tasks. Furthermore, we identified a connection between DTs and the distribution across PLMs'layers of the relevant information to predict that task. For example, similarity between the pre-training objective and the objective of predicting properties of individual residues leads to a steady increase in understanding of such tasks across the layers of PLMs. On the other hand, for whole-protein tasks, we observe that the dataset, rather than the task itself, defines PLMs'ability to perform well on a DT. Embeddings from shallow layers of PLMs perform better for datasets that contain deep mutational scan (DMS) data, while datasets containing diverse natural proteins find most useful embeddings in the models'deeper layers. Additionally, we discover that the performance of PLMs drops significantly when tasks are introduced for artificial proteins.
R. Joeres, Ilya S. Senatorov, A. Kolchina et al.· 0 citations
Large language models can learn new tasks through in-context learning (ICL), yet this ability remains underexplored for biological sequence classification. We evaluate ICL across 20 large language models on three antibody tasks: species-origin, antibody specificity, and isotype class classification. Few-shot prompting improves over zero-shot performance, but matching the performance of protein language model classifiers requires sequence-similar demonstrations. Building on this observation, we introduce a sequence similarity-based strategy for ICL in antibody sequence classification, Sim-ICL. Using 32-shot prompting, Sim-ICL achieves competitive performance on two of three tasks. Its simplicity makes few-shot ICL promising for antibody characterization, especially for researchers with limited coding expertise.
Sin-Hang Fung, Zhenghao Zhang, Ran Wang et al.· Genome Biology· 0 citations
General-purpose frontier language models are being increasingly utilized for protein-design work, yet their ability to understand and evaluate variant effects remains unclear. Here, we introduce PG-LLM, a benchmark comprising 276 protein-variant prioritization tasks: 217 from ProteinGym and a temporally held-out set of 59 from recently published studies. Each task follows the same format: a language model is asked to rank a list of variant sequences given only the wild-type protein sequence and an assay description with no access to tools, multiple-sequence alignments, or protein structures. We evaluate thirteen language models and 95 published protein predictors on the same variants with the same evaluation metric. Claude Opus 5 (Max) and GPT 5.6 Sol (Max) are the best performing LLMs with Spearman correlations of ρ = 0.406 and 0.402 respectively. Opus 5 outperforms 49 of 95 published protein predictors, including 41 of 46 sequence-only methods, and approaches ESM2-650M at ρ = 0.411, but remains below the leading predictor VenusREM at ρ = 0.523. We observe that variant-ranking performance scales with test-time compute across GPT, Claude, and Gemini models, but gains taper before closing the gap to specialist protein predictors. To address contamination risk, we create a held-out evaluation set with 59 DMS assays from 19 studies whose scores first became public after January 2026. On this set, we observe performance and test time compute scaling trends similar to those on the 217 tasks derived from ProteinGym. PG-LLM shows that tool-free language models capture substantial protein-variant signal, outperforming many sequence-based predictors while remaining below the strongest specialized models.
Rohit Arora, L. Chen, Melissa Du et al.· bioRxiv· 0 citations
Therapeutic peptides exert pivotal effects in diverse biological processes, and have attracted significant interest in the field of biomedicine in recent years. However, most existing methods often fail to adequately capture the intricate interactions among amino acid residues and the contextual dependencies within peptide sequences, which hampers the extraction of deep semantic representations and ultimately restricts predictive performance. Moreover, the task of multi-functional therapeutic peptide prediction is inherently constrained by the challenge of imbalanced multi-label classification resulting from long-tailed distribution patterns. In this study, we propose a two-stage hierarchical deep learning framework, named TPpred-PepPA, for the prediction of multi-functional therapeutic peptides based on pragmatic analysis. Specifically, ProtT5 is employed to extract deep semantic representations that capture residue-level contextual information. In the first stage, a transformer-based network is utilized to perform shared representation learning, wherein the encoder model captures the intricate inter-residue interaction to characterize the contextual semantics of peptide sequences. In the second stage, the framework is fine-tuned by incorporating task-specific classifiers and optimizing the classification decision with Asymmetric Loss. Then the dynamic thresholding strategy is utilized to address the long-tail distribution problem, enabling more accurate prediction performance of multi-functional therapeutic peptide. Moreover, we adopted the SHAP analysis and motif identification to interpret feature contributions and identify key functional peptide fragments, respectively. Our experimental results indicate that TPpred-PepPA significantly outperforms all current baseline methods in identifying multi-functional therapeutic peptides and exhibits robust performance in recognizing rare functional categories. We developed TPpred-PepPA, a two-stage hierarchical deep learning framework based on the ProtT5 pre-trained large language model. Compared with existing methods, TPpred-PepPA achieves state-of-the-art predictive performance and provides valuable interpretability for the discovery of multi-functional therapeutic peptides. Finally, a web server has been established and is accessible at http://bliulab.net/TPpred-PepPA.
Ke Yan, Siyang Lu, Shutao Chen et al.· BMC Biology· 0 citations
Applications to thioredoxins, visual opsins, and Tara Oceans environmental diatom cold-shock proteins show that PLMView can move from interpretable residue-level determinants in well-studied protein families to large-scale environmental functional discovery, linking molecular specialization to ecological distribution and transcriptional deployment across the global ocean.
Multimodal protein language models (pLMs) learn joint protein sequence-structure distributions, and their generation performance should also depend critically on inference-time sampling strategies. Yet prior work has focused more on model training than on how inference-time strategies behave. In this paper, we establish a three-stage investigation framework to empirically study the inference design space of multimodal pLMs across three representative pLMs and four fundamental tasks. We evaluate vanilla sampling, task-specific classifier-free guidance, and reward-guided beam search on multimodal pLMs, corresponding to controls over sampling distributions, per-step logits, and parallel trajectories. Throughout the complementary advancements centered on exploration-exploitation trade-off, we (1) reveal the suboptimality of default inference protocols and identify task-oriented sampling preferences; (2) observe substantial quantitative gains across tasks, consistently boosting the upper bound performance of multimodal pLMs without updating model parameters; (3) derive conclusions about base models that differ from prior consensus.
Yi Zhou, Qiping Wang, Yunqing Liu et al.· 0 citations