DNAmBERT: a transformer-based model for non-invasive cancer diagnosis using DNA sequence and methylation data

Abstract

Abstract DNA methylation alterations are early and stable hallmarks of cancer and represent promising biomarkers for non-invasive detection using circulating cell-free DNA (cfDNA). However, current computational approaches often model DNA sequence and methylation features separately and struggle to capture complex read-level methylation architecture in heterogeneous, low-signal liquid biopsy data. Here, we present DNAmBERT, a Transformer-based deep learning framework designed to jointly model DNA sequence context and read-level methylation haplotype structure from cfDNA methylation sequencing data. DNAmBERT integrates k-mer–encoded DNA sequences with methylation haplotype tokens using a unified representation and masked language modelling objective, enabling context-aware learning of sequence–epigenetic dependencies through self-attention. We evaluated DNAmBERT across multiple cfDNA methylation platforms (RRBS, cfRRBS, and cfMethyl-seq) and cancer types, including colorectal cancer, lung adenocarcinoma and hepatocellular carcinoma. In binary classification tasks, the model achieved high performance across platforms (AUC up to 0.99–1.00) and outperformed conventional machine learning and existing deep learning approaches. Aggregation of read-level predictions enabled quantitative tumour probability estimation at the sample level. Beyond binary detection, DNAmBERT supported multi-cancer and stage-aware classification, including early-stage disease, with multiclass AUC values up to 0.99. The framework further demonstrated effective cross-cancer transfer learning, maintaining robust performance under limited data availability. These results indicate that integrated sequence–haplotype representation learning provides an accurate and scalable approach for cfDNA-based multi-cancer detection.