data science

Embedded Conditional Independence Tests for Large Language Model Generated Text with an Application to German Parliament Speeches

Conditional independence tests (CITs) test for conditional dependence between two random objects $X$ and $Y$ given a third random object $Z$. Existing CITs have limited applicability to high-dimensional data, especially multimodal data like text. However, we show that such tests are of interest for large language model (LLM) outputs, where we test whether an output $X$ generated from a source text $Z$ carries information about an attribute $Y$ beyond $Z$ itself. For this purpose, we propose embedded CITs (eCITs), which embed $X$ and $Z$ and apply an existing CIT to the resulting representations and to $Y$. We show that, provided the embedding of $Z$ is sufficient, i.e. retains the information $Z$ carries about either $Y$ or the representation of $X$, the null hypothesis transfers from $X$ and $Z$ to their representations, so that a CIT valid for the embedded hypothesis is valid for the original one. We further give conditions for equivalence of the two hypotheses, and show that sufficiency weakens to mean sufficiency when the embedded test targets conditional mean independence. We propose a semi-synthetic simulation design to assess type I error (T1E) control and power of the eCITs for given embedding maps on a specific dataset and task, and use it to evaluate them on our application. Applying the eCITs to German Parliament speeches, we find for all combinations of embedding maps considered that the summaries of two LLMs contain information about the speaker's faction and gender beyond the speech they were generated from.

Provably Efficient Federated Reinforcement Learning with Linear Function Approximation and Logarithmic Communication Cost

We study federated online reinforcement learning with linear function approximation. While recent multi-agent reinforcement learning algorithms achieve strong regret guarantees, they typically require sharing raw trajectories. This reliance incurs a communication cost that scales linearly with the number of episodes and violates the privacy constraints of federated settings. To address these limitations, we propose Fed-LSVI, the first provably efficient federated algorithm for online reinforcement learning with linear function approximation in episodic Markov decision processes. By integrating a determinant-based event-triggered synchronization with a stepwise backward update mechanism, Fed-LSVI enables agents to collaboratively learn an optimal policy by exchanging only compressed sufficient statistics. We prove that Fed-LSVI achieves a regret bound of $\widetilde{\mathcal O}(\sqrt{Md^3H^4T})$, where $d$ is the feature dimension, $H$ is the horizon length, $M$ is the number of agents, and $T$ is the number of episodes per agent, matching the best-known regret for multi-agent online reinforcement learning with linear function approximation. Moreover, by following the stringent communication and privacy constraints of the federated setting, Fed-LSVI reduces the communication cost to only logarithmic dependence on $T$, representing a significant improvement over prior methods.

Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length

The proliferation of Large Language Models (LLMs) necessitates valid evaluation methods to provide guidance for both downstream applications and actionable future improvements. The Item Response Theory (IRT) model with Computerized Adaptive Testing has recently emerged as a promising framework for evaluating LLMs via their response accuracy. Beyond simple response accuracy, LLMs' chain of thought (CoT) lengths serve as a vital indicator of their reasoning ability. To leverage the CoT length information to assist LLM evaluation, we propose the \textbf{La}tency-\textbf{R}esponse \textbf{T}heory (LaRT) model, which jointly models both the response accuracy and CoT length by introducing a key correlation parameter between the latent ability and the latent speed. We derive an efficient stochastic approximation Expectation-Maximization algorithm for parameter estimation. We establish rigorous identifiability results for the latent ability and latent speed parameters to ensure the statistical validity of their estimation. Through both theoretical asymptotic analyses and simulation studies, we demonstrate LaRT's advantages over IRT in terms of superior estimation accuracy and shorter confidence intervals for latent trait estimation. To evaluate LaRT in real data, we collect responses from diverse LLMs on popular benchmark datasets. We find that LaRT yields different LLM rankings than IRT and outperforms IRT across multiple key evaluation metrics including predictive power, item efficiency, ranking validity, and LLM evaluation efficiency. Code and data are available at https://github.com/Toby-X/Latency-Response-Theory-Model

First-Order Efficiency for Probabilistic Value Estimation via A Statistical Viewpoint

Probabilistic values, including Shapley values and semivalues, provide a model-agnostic framework to attribute the behavior of a black-box model to data points or features, with a wide range of applications including explainable artificial intelligence and data valuation. However, their exact computation requires utility evaluations over exponentially many coalitions, making Monte Carlo approximation essential in modern machine learning applications. Existing estimators are often developed through different representation strategies, including weighted averages, self-normalized weighting, regression adjustment, and weighted least squares. Our key observation is that these seemingly distinct constructions share a common first-order expansion, in which the leading term is determined by the sampling law and a working surrogate function. This first-order representation yields an explicit expression for the leading mean squared error (MSE), which characterizes how the sampling law and the surrogate jointly determine statistical efficiency. Guided by this criterion, we propose an Efficiency-Aware Surrogate-adjusted Estimator (EASE) that directly chooses the sampling law and surrogate to minimize the first-order MSE. We demonstrate that EASE consistently outperforms existing estimators for various probabilistic values.

Scale-Consistent Posterior Dynamics for Diffusion Inverse Problems

Posterior sampling with a pretrained diffusion prior is governed by a conditional score whose intermediate likelihood component is generally intractable. We begin from an ideal one-parameter posterior SDE family in which a stochasticity parameter controls probability-flow transport and stochastic exploration without changing the posterior marginals. To obtain a tractable model, we express the likelihood in a rescaled clean-image coordinate and use log-SNR to organize the resulting posterior proxies. Projecting the diffusion uncertainty through the forward operator then yields a noise-conditioned covariance path whose targets approach the clean posterior. Because endpoint consistency of these targets does not ensure that a surrogate transport follows them, we interleave the transport with a frozen-target Langevin corrector, producing a continuous surrogate SDE. We discretize this model with an outer Lie--Trotter splitting and a variance-matched split-step IMEX predictor that treats the learned prior explicitly, the linear likelihood implicitly, and the stochastic innovation after the implicit solve. We prove marginal invariance of the ideal family, posterior convergence of the continuous surrogate under mixing and transport-defect conditions, and a first-order weak error bound for the discrete algorithm. Experiments on FFHQ and ImageNet with 100 score evaluations demonstrate competitive reconstruction fidelity for super-resolution and deblurring. A controlled 100-image ablation separates scale consistency from the finite-step effects of stochastic-increment placement, continuation, and corrector allocation. A separate noiseless box-inpainting study shows that large exploration reaches a performance plateau only when the matched innovation is injected after the stiff likelihood solve.

LumiCharge: Spherical Harmonic Convolutional Networks for Atomic Charge Prediction in Drug Discovery.

Atomic charge is crucial in drug design for analyzing reactive sites and interactions between ligands and targets. While quantum mechanical methods offer high accuracy, they are generally computationally costly. Conversely, empirical approaches, while computationally efficient, frequently suffer from lack of precision and generalizability. Recent a number of machine learning-based models have been developed for atomic charge predictions, but they struggle with accurately representing molecular structures and capturing the chemical environments affecting atomic charges, thus limiting their generalization and accuracy. To overcome these limitations, we propose LumiCharge, a novel atomic charge prediction framework that incorporates high-order spherical harmonics convolutions and explicitly models multibody interactions. In constructing this model, we employ a strategy that integrates both high- and low-order information, enhancing its geometric spatial perception capability, which is currently underexplored in the field. Benchmark evaluations demonstrate that LumiCharge outperforms state-of-the-art (SOTA) models by 30%-60% across diverse data sets. Additionally, in cross-scale experiments, LumiCharge demonstrates exceptional extrapolation capability and robustness across molecules of varying sizes, effectively overcoming the limitations imposed by molecular sizes. On an external halogen-containing test set, LumiCharge achieves an RMSE of 0.055e, meeting practical application requirements. Finally, a case study of virtual screening for the androgen receptor (AR) target further validates its outstanding accuracy compared to the OPLS3e force field and other deep learning (DL)-based baseline models, highlighting its exceptional generalization capacity and practical utility in real-world scenarios.

PepBAN: A Deep Learning Framework with Bilinear Attention and Adversarial Learning for Peptide-Protein Interaction Prediction

Accurate prediction of the peptide-protein interaction (PepPI) is crucial for developing peptide-based therapeutics and vaccines. However, this computational task has traditionally faced significant challenges, such as the scarcity of structure data along with the corresponding label of the binding affinity for bound complexes. To address these challenges, we introduce PepBAN, a deep learning framework for modeling PepPI predictions. PepBAN incorporates two technical advancements: (1) adopting the protein language model ESM-2 to characterize proteins and ESM-2 or a graph-based foundation model for peptides without structure data and (2) leveraging the conditional domain adversarial learning to enhance generalization across a broad range of protein targets, especially when there are limited binding data. At the core of PepBAN is a bilinear attention network (BAN) that effectively learns the pattern of pairwise local interactions, enables the identification of key residues participating in the peptide-protein interactions, and offers an intuitive approach to interpret the underlying mechanisms of PepPIs via analyzing attention weights. Our numerical experiments demonstrated that PepBAN outperformed the previous state-of-the-art models across several well-established benchmark studies. Furthermore, we evaluated PepBAN's applicability in predicting cyclic peptide-protein interactions, a task that poses significant challenges due to the presence of noncanonical amino acids. These nonstandard residues require specialized handling, which most existing sequence-based PepPI prediction models did not adequately address, and we adopt an atom-resolved molecular graph approach to process cyclic peptides. Despite this complexity, PepBAN demonstrated a clear advantage by achieving a superior prediction performance and offering a distinct edge in tackling the emerging chemical space of cyclic peptides, which has great potential for novel therapeutic development. In summary, PepBAN serves as a valuable tool for advancing peptide-based drug and therapeutic development.

SynGFN: learning across chemical space with generative flow-based molecular discovery

Allosteric Cooperativity Mechanism Investigation of Orthosteric and Allosteric Ligands in Modulating AR Activity: A Molecular Dynamics Study

The androgen receptor (AR) represents a pivotal therapeutic target for prostate cancer. However, existing orthosteric ligand-binding pocket (LBP) antagonists [e.g., enzalutamide (ENZ)] encounter significant obstacles due to resistance-conferring mutations in the LBP. Allosteric antagonists targeting the BF3 site exhibit great potential in overcoming such resistance but have low inhibitory efficacy. In our study, we employed an integrated computational modeling strategy, including Gaussian-accelerated molecular dynamics (GaMD), MM/GBSA free-energy calculations, and elastic network model (ENM)-based signaling communication pathway analyses. This approach is used to probe the cooperativity of allosteric BF3 antagonists [e.g., VPC-13808 (VPC)] with diverse orthosteric LBP ligands [e.g., ENZ and testosterone (TES)] in suppressing AR activity. Herein, four types of AR systems were examined: AR bound to LBP agonist (AR·TES), LBP antagonists (e.g., AR·ENZ), and combinations of LBP agonist/antagonist with BF3 antagonist (e.g., AR·TES·VPC and AR·ENZ·VPC). Results indicate that BF3 antagonists can synergize with the LBP antagonist to amplify conformational flexibility in H12 and induce anticorrelated dynamics of H12 with H3 and H4. This induces the downward movement of H12 and its displacement away from H3/H4, triggering the wide opening of the AF2 binding cleft and substantially reducing the coactivator recruitment. Furthermore, the BF3 antagonist can interact with specific residues (e.g., F673, F826, L830, and Y834) and cooperate with the LBP agonist or antagonist to allosterically perturb the AF2 conformation. Multiple short- and/or long-range BF3→AF2 and LBP→AF2 signaling transition pathways are involved, such as F673→Y834→L722→L812→L744→V746→L873→ENZ→L880/V889/V891. These mechanistic insights establish the foundation for developing novel AR BF3 antagonist and LBP-BF3 combination therapies, suggesting a promising avenue for enhancing the efficacy and overcoming the resistance in castration-resistant prostate cancer treatment.

ChargeNet: E(3) Equivariant Graph Attention Network for Atomic Charge Prediction

Atomic charge is a fundamental quantum chemical property essential for advancing drug design and discovery. Although quantum mechanics (QM) methods offer the highest level of accuracy, their computational demands scale quadratically with the number of atoms, limiting their practicality for large-scale applications. In light of this, empirical and semiempirical methods have been introduced to improve computational efficiency, albeit often at the expense of accuracy. The advent of artificial intelligence has witnessed a growing application of machine learning (ML) techniques to accelerate atomic charge predictions. However, existing ML models often suffer from low accuracy and limited generalization capabilities. To address these challenges, we introduce an advanced equivariant graph attention neural network specifically engineered to model long-range atomic electrostatic interactions with high precision. This model introduces a sophisticated global graph attention mechanism, enabling it to capture charge contributions across multiple scales. By utilizing a combination of structural symmetry-preserving transformations and multiscale attention, our approach not only preserves the inherent symmetries of molecular structures but also substantially improves the model's accuracy, generalization, and robustness in complex scenarios. Our empirical analyses demonstrate that, compared to leading baseline models, the proposed model improves charge prediction accuracy by over 40% on average across various charge-calculation schemes. Remarkably, the model achieves superior performance on the external RESP (restrained electrostatic potential) test data sets, with a 54.6% improvement over the baseline. Additionally, we evaluated our charge model under the setting of virtual screening, where it outperforms both the OPLS3 charges and baseline deep learning models across all evaluation metrics, highlighting its extensive potential for scientific discovery.

PepPCBench is a Comprehensive Benchmarking Framework for Protein-Peptide Complex Structure Prediction

Accurate modeling of protein-peptide interactions is essential for understanding fundamental biological processes and designing peptide-based drugs. However, predicting the complex structures of these interactions remains challenging, primarily due to the high conformational flexibility of peptides. To support a fair and systematic evaluation of recent deep learning (DL) approaches, we introduce PepPCBench, a benchmarking framework tailored to assess protein folding neural networks (PFNNs) in protein-peptide complex prediction. As part of this framework, we curated PepPCSet, a data set of 261 experimentally resolved complexes with peptides ranging from 5 to 30 residues. We benchmark five full-atom PFNNs, including AlphaFold3 (AF3), AlphaFold-Multimer (AFM), Chai-1, HelixFold3 (HF3), and RoseTTAFold-All-Atom (RFAA), using comprehensive evaluation metrics. Our benchmarking reveals meaningful performance differences among these methods and highlights the influence of peptide length, conformational flexibility, and training set similarity on prediction accuracy. While AF3 shows strong performance in structure prediction, further analysis indicates that confidence metrics correlate poorly with experimental binding affinities, underscoring the need for improved scoring strategies and generalizability. By providing a reproducible and extensible framework, PepPCBench enables a robust evaluation of PFNN-based methods and supports their continued development for peptide-protein structure prediction.

CarsiDock-Cov: A deep learning-guided approach for automated covalent docking and screening

The interest in covalent drugs has resurged in recent decades, spurring the development of numerous specialized computational docking tools to facilitate covalent ligand design and screening. Herein, we present CarsiDock-Cov, a new paradigm distinguishing itself as the first deep learning (DL)-guided approach for covalent docking. CarsiDock-Cov retains the core components of its non-covalent predecessor, leveraging a DL model pretrained on millions of docking complexes to predict protein–ligand distance matrices, along with a dedicated-designed geometric optimization procedure to convert these distances into refined binding poses. Additionally, it incorporates several key enhancements specifically tailored to optimize the protocol for covalent docking applications. Our approach has been extensively validated on multiple public datasets regarding the docking and screening of covalent ligands, and the results indicate that our approach not only achieves comparably improved applicability compared to its non-covalent predecessor, but also exhibits competitive performance against various state-of-the-art covalent docking tools. Collectively, our approach represents a significant advance in covalent docking methodology, offering an automated and efficient solution that shows considerable promise for accelerating covalent drug discovery and design.