Results show that string representation meaningfully influences transformer fragment recovery and that representation-aware selection outperforms either representation alone.
Abstract
Fragment-based drug discovery (FBDD) uses small molecular fragments as starting points for drug development, and machine-learning models that operate on molecular string representations are increasingly applied to fragment-related tasks. SMILES is the dominant such representation, but its paired ring closures and balanced parentheses introduce syntactic complexity that can affect model behavior. We present a controlled study of how molecular string representation influences transformer-based fragment recovery, using an FBDD-motivated label pipeline: reference fragments are derived from known drugs via RECAP fragmentation and docking-based ranking, and the model is scored on recovering them. We introduce DeepBERTa, a ChemBERTa-derived transformer pretrained on DeepSMILES, and evaluate it against a matched SMILES baseline under identical architecture, data splits, and optimization (approximately 34,000 drug–fragment pairs). DeepSMILES produces syntactically valid predictions more often than SMILES (54.2% vs. 43.5%) and a higher full test-set mean Tanimoto similarity (0.36 vs. 0.29). A per-sample selection between the two representations raises mean Tanimoto to 0.43, and a deployable variant that selects on model confidence rather than the reference recovers most of this gain. Among molecules where one representation strictly wins, DeepSMILES wins more often than SMILES (27.8% vs. 16.3% of test molecules). These results show that string representation meaningfully influences transformer fragment recovery and that representation-aware selection outperforms either representation alone.
FragBERTa is introduced, a molecular fragment-aware transformer-based representation language model pretrained using masked language modeling on Sequential Attachment-based Fragment Embedding (SAFE) representations, suggesting that fragment-based string representations offer advantages over atom-level representations for scaffold-sensitive and interaction-driven tasks.
Quantitative prediction of inhibitor potency can accelerate early-stage drug discovery. Recently, data-driven approaches have gained widespread interest in drug discovery, as evidenced by a growing number of benchmarking challenges and open competitions. In this context, we developed a machine learning-based methodology that can find the most effective way of predicting IC50 values against ASK1 from SMILES, for "Jump AI(.py) 2025: 3rd AI Drug Discovery Competition", hosted by the Korea Pharmaceutical and Bio-Pharma Manufacturers Association (KPBMA) on the Dacon platform. Applying our methodology achieved the highest overall predictive performance among all participating teams. Beyond this competition setting, we present a compact SMILES-based modeling workflow comprising (i) a pre-trained encoder, (ii) regression models, (iii) data augmentation, and (iv) hyperparameter tuning. We systematically compared molecular representations from sequence- and graph-based models, including ChemBERTa-2 and MolCLR. Across encoder-regressor combinations, ChemBERTa-77 M-MLM embeddings paired with support vector regression (SVR) yielded the strongest predictive performance. Embedding-level mix-up augmentation and SVR hyperparameter tuning further improved predictive performance. Our findings highlight that careful SMILES preprocessing and encoder selection have a critical influence on IC50 values and provide a reproducible benchmark for single-target bioactivity prediction, thus contributing to a more efficient drug discovery process. Scientific Contribution In this study, we propose a machine learning methodology for predicting the IC50 values of ASK1 inhibitors from SMILES representations, with a systematic comparison of molecular encoders and regression models. Our results show that the use of suitable encoder-regressor pairs together with embedding-level mix-up augmentation improves model generalizability without requiring SMILES-level augmentation. This strategy would be particularly useful for settings with imbalanced labels or limited data, and could be applied more broadly to IC50 prediction for other kinase inhibitors.
Ju Hyung Lee, S. Choi, Utku Ozbulak et al.· Journal of Cheminformatics· 0 citations
A novel Dual-Attention Multimodal framework for Graphs and Sequence-based representations, so-called DAM-GS, which provides a promising solution for molecular property prediction with broad applications in drug discovery and computational molecular science.
Bay Van Nguyen, Vinh Truong, Ha Duong Thi Hong et al.· Journal of Chemical Informat...· 0 citations
The DRL-DSP is proposed, a novel dual representation learning framework designed to enhance drug synergy prediction by integrating molecular-level features from SMILES sequences with graph-level relational information from reconstructed molecular networks.
Juanzi Zhou, Xiaoliang Yang, Yin Zhang et al.· Intelligent Data Analysis· 0 citations
This is the first method to expose GNN-derived attributions to an LLM as evidence for property prediction, and achieves the best overall results among generalist models and narrows the gap to specialist models tuned for each task.
Junwoo Park, Minyoung Shin, C. Lee et al.· 0 citations
Drug repurposing offers a time-efficient strategy for identifying therapeutics against emerging pathogens such as SARS-CoV-2. In this study, we apply MPS2IT-DTI (Molecule and Protein Sequence to Image Transformer for Drug-Target Interaction), a deep learning framework that represents molecular (SMILES) and protein (FASTA) sequences as images using k-mer frequency encoding, enabling convolutional neural networks to capture spatial compositional patterns associated with biochemical interactions. A curated dataset (BindingDB-FDA) containing 83,165 binding interactions from 1640 FDA-approved ligands and 3270 targets was constructed from BindingDB, with binding scores derived from the KIBA scoring system. An enhanced variant, MPS2IT+MN, incorporating max-norm regularization, was introduced to improve generalization. The model was applied to predict binding affinities between 33 FDA-approved antiviral drugs and six key SARS-CoV-2 non-structural proteins. Results consistently identified five antivirals - MK-5172 (Grazoprevir), Simeprevir, Lopinavir, Etravirine, and Atazanavir - as top-ranked candidates across all targets. Comparative analysis with the MT-DTI model demonstrated competitive and, in several cases, superior ranking performance despite a simpler architecture. Importantly, these predictions are supported by independent experimental and clinical evidence, highlighting the potential of image-based representations as a computationally efficient and biologically meaningful approach for drug-target interaction prediction and drug repurposing.
Jackson G. de Souza, Marcelo A. C. Fernandes, Raquel de Melo Barbosa· Computational biology and ch...· 0 citations