A contrastive pretraining framework that learns cell representations through complementary transcriptomic views by constructing two complementary views of each cell and introducing a competence-aware controller to determine how the contrastive objective is applied is established.
Abstract
The rapid growth of single-cell transcriptomic data has enabled the development of foundation models pretrained primarily by reconstructing masked expression values. This objective encourages these models to learn gene dependencies but does not directly optimize whole-cell representations, which are essential for many downstream tasks. To bridge this gap, we propose a contrastive pretraining framework that learns cell representations through complementary transcriptomic views. Since standard contrastive learning is not readily applicable to single-cell pretraining, we introduce specific adaptations along three dimensions --- co-expression-guided gene partitioning, expression-aware contrast-set construction, and competence-gated contrastive onset. Specifically, we first construct two complementary views of each cell by partitioning its genes according to their co-expression structure. Then, to prevent the model from using gene-set identity as a shortcut, we construct hard negatives by permuting expression values while keeping gene identities unchanged. Finally, we introduce a competence-aware controller to determine how the contrastive objective is applied. Experiments on cell-type annotation and gene regulatory network inference demonstrate competitive transfer under the evaluated protocols. In the six-network GRN evaluation, our method records the highest mean AUROC and AUPRC point estimates among the compared variants, while the highest-scoring variant differs across individual networks. These results establish complementary-view contrastive learning as an effective direction for single-cell pretraining beyond gene reconstruction.
Single-cell foundation models have shown strong potential for learning transferable representations from large-scale transcriptomic data. However, many existing approaches rely on reconstructing masked gene expression values, creating a potential mismatch between observation-space reconstruction and the goal of learnin...
Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation...
Han-Qing Zhang, Jie Bao, Mei Ma et al.· 0 citations
Single-cell transcriptomic analysis predominantly derives cell identity from gene expression analysis, while alternative splicing is processed separately despite its fundamental role for cell homeostasis. To overcome the limits of separate investigations, we developed a probabilistic deep learning framework, Crecerelle...
Friedrich-Maximilian Weberling, Ioakeim Ampartzidis, I. Mohorianu et al.· bioRxiv· 0 citations
Single-cell transcriptomics enables profiling of cellular states at unprecedented resolution, but its high dimensionality, sparsity, and technical batch effects pose significant challenges for representation learning. Existing single-cell foundation models typically encode each cell independently or only model cells fr...
Su-Yuan Zhao, Ming-Hao Liu, Yi-Zhen Luo et al.· arXiv.org· 0 citations
Spatial transcriptomics captures molecular states within cells and their organisation in tissue. However, integrating fine-grained gene information with spatial context at scale remains challenging for existing foundation models. Here we present NexuST, a hierarchical foundation model that repeatedly interleaves gene-l...
Hai-Ping Liu, Qian Zhao, Li-Jing Lin et al.· bioRxiv· 0 citations
StFormer, a transformer model tailored for ST data, which employs the cross-attention module to incorporate spatial ligand genes, is introduced, which enables the model to do learning with single-cell resolution on whole-transcriptome but low-resolution Visium data.