Skip to content
Preprint

Beyond Gene Reconstruction: Learning Cell Representations through Complementary Transcriptomic Views

Aug 2026 · 0 citations · 20 references
Computer Science Biology

TL;DR

A contrastive pretraining framework that learns cell representations through complementary transcriptomic views by constructing two complementary views of each cell and introducing a competence-aware controller to determine how the contrastive objective is applied is established.

Abstract

The rapid growth of single-cell transcriptomic data has enabled the development of foundation models pretrained primarily by reconstructing masked expression values. This objective encourages these models to learn gene dependencies but does not directly optimize whole-cell representations, which are essential for many downstream tasks. To bridge this gap, we propose a contrastive pretraining framework that learns cell representations through complementary transcriptomic views. Since standard contrastive learning is not readily applicable to single-cell pretraining, we introduce specific adaptations along three dimensions --- co-expression-guided gene partitioning, expression-aware contrast-set construction, and competence-gated contrastive onset. Specifically, we first construct two complementary views of each cell by partitioning its genes according to their co-expression structure. Then, to prevent the model from using gene-set identity as a shortcut, we construct hard negatives by permuting expression values while keeping gene identities unchanged. Finally, we introduce a competence-aware controller to determine how the contrastive objective is applied. Experiments on cell-type annotation and gene regulatory network inference demonstrate competitive transfer under the evaluated protocols. In the six-network GRN evaluation, our method records the highest mean AUROC and AUPRC point estimates among the compared variants, while the highest-scoring variant differs across individual networks. These results establish complementary-view contrastive learning as an effective direction for single-cell pretraining beyond gene reconstruction.

View source

Similar papers

Open access Sep 2026

scRep: A Latent-Space Self-Distilled Foundation Model for Single-Cell Representation Learning

Single-cell foundation models have shown strong potential for learning transferable representations from large-scale transcriptomic data. However, many existing approaches rely on reconstructing masked gene expression values, creating a potential mismatch between observation-space reconstruction and the goal of learnin...

Sheng-Jie Wang, Zong-Yong Hu, Yunlong Bie et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Towards a knowledge-enhanced single-cell foundation model

Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation...

Han-Qing Zhang, Jie Bao, Mei Ma et al. · 0 citations
Open access Oct 2026

Deep generative embeddings of gene expression and splicing reposition the interpretation of single-cell transcriptomic signatures

Single-cell transcriptomic analysis predominantly derives cell identity from gene expression analysis, while alternative splicing is processed separately despite its fundamental role for cell homeostasis. To overcome the limits of separate investigations, we developed a probabilistic deep learning framework, Crecerelle...

Friedrich-Maximilian Weberling, Ioakeim Ampartzidis, I. Mohorianu et al. · 0 citations

CellMSA: Context Modeling for Single-Cell Representation Learning

Single-cell transcriptomics enables profiling of cellular states at unprecedented resolution, but its high dimensionality, sparsity, and technical batch effects pose significant challenges for representation learning. Existing single-cell foundation models typically encode each cell independently or only model cells fr...

Su-Yuan Zhao, Ming-Hao Liu, Yi-Zhen Luo et al. · 0 citations
Open access Sep 2026

NexuST: A Hierarchical Foundation Model for Spatial Transcriptomics

Spatial transcriptomics captures molecular states within cells and their organisation in tissue. However, integrating fine-grained gene information with spatial context at scale remains challenging for existing foundation models. Here we present NexuST, a hierarchical foundation model that repeatedly interleaves gene-l...

Hai-Ping Liu, Qian Zhao, Li-Jing Lin et al. · 0 citations
Open access Sep 2026

stFormer integrates spatial ligand signaling into a foundation model for spatial transcriptomics.

StFormer, a transformer model tailored for ST data, which employs the cross-attention module to incorporate spatial ligand genes, is introduced, which enables the model to do learning with single-cell resolution on whole-transcriptome but low-resolution Visium data.

Sheng-Hao Cao, Kai-Yuan Yang, Jia-Bei Cheng et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.