DILI Prediction Using Molecular Fingerprints and ChemBERTa

Abstract

Drug-induced liver injury (DILI) is a major cause of drug development failure and post-marketing withdrawal. Accurate computational prediction of hepatotoxicity is hindered by complex biological mechanisms and scarce labeled toxicity data. Although pretrained molecular language models like ChemBERTa perform well in molecular property prediction, their generalization ability for small-sample DILI prediction remains underexplored. Here, we systematically compared traditional molecular fingerprint-based machine learning methods and ChemBERTa-based models for DILI classification on the DILIst dataset. Canonical SMILES from PubChem were used to generate Morgan fingerprints and ChemBERTa embeddings. We evaluated Random Forest, XGBoost, full fine-tuning, frozen encoder transfer learning, and embedding-based classifiers under both random and scaffold data splits. Results showed that Morgan fingerprints combined with Random Forest achieved the best performance, with ROC-AUC of 0.783 and PR-AUC of 0.849 under random split. Scaffold split markedly degraded the performance of all models, indicating poor generalization to unseen chemical scaffolds. ChemBERTa embedding-based classifiers outperformed end-to-end fine-tuning, suggesting that pretrained representations are better used as fixed feature extractors under limited labeled DILI data. Further SHAP analysis detected key toxicity-related molecular fragments, and t-SNE showed insufficient latent-space separation between DILI-positive and negative compounds. Our results confirm that traditional fingerprint-based machine learning remains highly competitive for small-sample hepatotoxicity prediction, and this work provides a reliable computational framework for early drug safety assessment.