Director of Machine Learning Insights [Part 3: Finance Edition]
More from the blog
Transfer learning for genomic prediction in underrepresented populations
General Science
Walter Torous named executive director of MIT Center for Real Estate
The senior lecturer, already director of the degree program, will now oversee all aspects of the center’s activities and operations.
Mapping global methane emissions from space with deep learning
Climate & Sustainability
Looking beyond natural sequences
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
Related papers
Enhanced Sampling in the Age of Machine Learning: Algorithms and Applications
Molecular dynamics simulations hold great promise for providing insight into the microscopic behavior of complex molecular systems. However, their effectiveness is often constrained by long timescales associated with rare events. Enhanced sampling methods have been developed to address these challenges, and recent years have seen a growing integration with machine learning techniques. This Review provides a comprehensive overview of how they are reshaping the field, with a particular focus on the data-driven construction of collective variables. Furthermore, these techniques have also improved biasing schemes and unlocked novel strategies via reinforcement learning and generative approaches. In addition to methodological advances, we highlight applications spanning different areas, such as biomolecular processes, ligand binding, catalytic reactions, and phase transitions. We conclude by outlining future directions aimed at enabling more automated strategies for rare-event sampling.
A Simultaneous, Multidisciplinary Development and Design Journey - Reflections on Prototyping
This paper proposes a wayfaring approach for the early concept creation stage of development projects that have a very high degree of intended innovation and thus uncertainty. The method is supported by a concrete game design example involving the development of a tangible programming interface for virtual car racing games. We focus onto projects that not only have high degrees of freedom, for example in terms of reframing the problem or iterating the final project vision, but are also complex in nature. For example, these can be projects that allow for the exploration and exploitation of unknown unknowns and serendipity findings. Process wise we are primarily focusing onto the early stage that precedes the requirement fixation, which we see as more dynamic and evolutionary in nature. The core conceptual elements that we have derived from the development experiences are: simultaneous prototyping in multiple disciplines (such as computer science, electronics and mechanics and engineering in general, abductive learning based on the outcome of rapid cycles of designing, building and testing prototypes (probing), and the importance of includingall the involved disciplines (knowledge domains) from the beginning of the project on.
Token-Mol 1.0: tokenized drug design with large language models
The integration of large language models (LLMs) into drug design is gaining momentum; however, existing approaches often struggle to effectively incorporate three-dimensional molecular structures. Here, we present Token-Mol, a token-only 3D drug design model that encodes both 2D and 3D structural information, along with molecular properties, into discrete tokens. Built on a transformer decoder and trained with causal masking, Token-Mol introduces a Gaussian cross-entropy loss function tailored for regression tasks, enabling superior performance across multiple downstream applications. The model surpasses existing methods, improving molecular conformation generation by over 10% and 20% across two datasets, while outperforming token-only models by 30% in property prediction. In pocket-based molecular generation, it enhances drug-likeness and synthetic accessibility by approximately 11% and 14%, respectively. Notably, Token-Mol operates 35 times faster than expert diffusion models. In real-world validation, it improves success rates and, when combined with reinforcement learning, further optimizes affinity and drug-likeness, advancing AI-driven drug discovery. In this work the authors present Token-Mol, a token-only 3D drug design model, which deploys the Gaussian cross-entropy (GCE) loss function for regression tasks. It exhibits superior performance in molecular conformation generation, property prediction, and pocket-based generation, thus opening up new avenues for drug design.
RSGPT: a generative transformer model for retrosynthesis planning pre-trained on ten billion datapoints
Retrosynthesis planning is a crucial task in organic synthesis, and deep-learning methods have enhanced and accelerated this process. With the advancement of the emergence of large language models, the demand for data is rapidly increasing. However, available retrosynthesis data are limited to only millions. Therefore, we pioneer the utilization of the template-based algorithm to generate chemical reaction data, resulting in the production of over 10 billion reaction datapoints. A generative pretrained transformer model is subsequently developed for template-free retrosynthesis planning by pre-training on 10 billion generated data. Inspired by the strategies of large language models, we introduce reinforcement learning to capture the relationships among products, reactants, and templates more accurately. Experiments demonstrate that our model achieves state-of-the-art performance on the benchmark, with a Top-1 accuracy of 63.4%, substantially outperforming previous models. Computer-aided synthesis-planning methods have significantly assisted synthesis planning. In this work, the authors present RSGPT, a generative model pre-trained on ten billion data points, achieving state-of-the-art performance for synthesis planning