Reinforcement learning (RL) is increasingly applied to Protein Language Models (PLMs), yet its effectiveness varies across tasks, and standard metrics such as pass@k can rise even when the model's solvable problem set is shrinking. We introduce two capability-level diagnostics. The Expansion-Shrinkage Ratio (ESR) measures how RL shifts the set of problems a PLM can solve, separating genuine gain from probability redistribution. Dual-Reward ESR reports ESR under both the training reward and an orthogonal evaluator; the gap ΔESR quantifies reward hacking as a single observable number. Applied across four protein design tasks, three RL algorithms (DPO, PPO, GRPO), and two PLM architectures, the diagnostics reveal that RL on PLMs is governed by two reward properties: verifiability, whether the reward is a fixed environment or a learned surrogate vulnerable to distribution shift, and coverage, the fraction of sequence space giving an informative gradient. The two axes produce three regimes with distinct ΔESR signatures: well-covered verifiable rewards yield genuine expansion; sparse verifiable rewards induce a coverage bottleneck; predicted rewards induce reward hacking. Controlled analyses isolate these two factors as operative, letting practitioners predict an RL run's outcome before committing to costly fine-tuning.
Hanqun Cao, Hongrui Zhang, Junde Xu et al.· Proceedings of the 32nd ACM...· 0 citations
This tutorial presents OneEHR, an open-source toolkit that defines a unified experiment contract for modern EHR modeling and enables head-to-head comparison among conventional, neural, LLM-based, and agentic methods through a single configuration-driven interface.
Yinghao Zhu, Zixiang Wang, Lei Gu et al.· Proceedings of the 32nd ACM...· 0 citations
Machine learning (ML) has seen promising developments in materials science, yet its efficacy largely depends on detailed crystal structural data, which are often complex and hard to obtain, limiting their applicability in real-world material synthesis processes. An alternative, using compositional descriptors, offers a simpler approach by indicating the elemental ratios of compounds without detailed structural insights. However, accurately representing materials solely with compositional descriptors presents challenges due to polymorphism, where a single composition can correspond to various structural arrangements, creating ambiguities in its representation. To this end, we introduce PCRL, a novel approach that employs probabilistic modeling of composition to capture the diverse polymorphs from available structural information. Extensive evaluations on sixteen datasets demonstrate the effectiveness of PCRL in learning compositional representation, and analysis on model uncertainty highlights its potential applicability of PCRL in material discovery.
Namkyeong Lee, Heewoong Noh, Gyoung S. Na et al.· Proceedings of the 32nd ACM...· 0 citations
