Robust geo-localization under changing environmental and operational conditions is critical for long-term aerial autonomy. Aerial visual place recognition (VPR) commonly uses pre-acquired remote-sensing imagery of the intended operating area, so the geographic label space can remain fixed while successive airborne missions introduce substantial visual distribution shifts. Continual adaptation to these shifts can cause catastrophic forgetting. We therefore formulate aerial VPR as a mission-based domain-incremental learning (DIL) problem and develop a heterogeneous memory framework. Before sequential adaptation, the satellite reference dataset is used once to train the initial model and construct a static satellite exemplar memory; a bounded replay buffer then retains selected airborne observations across missions. For replay management, we compare loss- and diversity-based selection criteria and introduce DBS-Hybrid, which combines prototype-based diversity trimming with representative-first feature-space coverage. Experiments on 21 visible and infrared UAV missions evaluate generalization to held-out missions, immediate adaptation, and knowledge retention. Under the primary Forward mission order, DBS-Hybrid achieves the highest mean final average accuracy, generalization, and knowledge retention among the evaluated methods, improving over the Random baseline by $5.06$, $5.32$, and $6.33$ percentage points, respectively, and improving backward transfer from -6.41% to 1.07%. Across five additional random mission orders, DBS-Hybrid ranks second in mean final average accuracy, backward transfer, generalization, and knowledge retention. Overall, heterogeneous memory and diversity-aware replay provide an effective basis for continual aerial VPR in mapped operating areas.
Xingyu Shao, Zhiqiang Yan, Liangzheng Sun et al.· 0 citations
GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.
Nina Wiedemann, Quentin Leboutet, Michael Paulitsch et al.· 0 citations
We propose a streamlined spectral algorithm for community detection in the two-community stochastic block model (SBM) under constant edge density assumptions. By reducing algorithmic complexity through the elimination of non-essential preprocessing steps, our method directly leverages the spectral properties of the adjacency matrix. We demonstrate that our algorithm exploits specific characteristics of the second eigenvector to achieve improved error bounds that approach information-theoretic limits, representing a significant improvement over existing methods. Theoretical analysis establishes that our error rates are tighter than previously reported bounds in the literature. Comprehensive experimental validation confirms our theoretical findings and demonstrates the practical effectiveness of the simplified approach. Our results suggest that algorithmic simplification, rather than increasing complexity, can lead to both computational efficiency and enhanced performance in spectral community detection.
Reliable video understanding requires high-quality video datasets that can provide both precise semantic labels and temporally consistent annotations. Detecting annotation errors in densely labeled videos is challenging because errors may arise from semantic **mislabeling**, where labels disagree with visual content, or temporal **disordering**, where otherwise plausible labels violate procedural progression. Training dynamics have been used to identify mislabeled training examples primarily for static samples. We investigate checkpoint loss dynamics for **out-of-sample auditing** of temporally annotated videos. We compute **Cumulative Sample Loss (CSL)** as the mean annotation-conditioned loss of an audit frame across checkpoints trained on a *disjoint* reference set. CSL acts as a dynamic fingerprint and captures the persistent disagreement between its annotation and learned visual-temporal structure. High-CSL frames are then flagged as likely candidates for potential annotation errors, including semantic mislabeling or temporal disordering. Experiments on EgoPER and Cholec80 show that CSL substantially outperforms final-checkpoint loss and achieves up to a **4.2-point AUC improvement** over prior baselines on EgoPER and **92.0/78.5 AUC** for mislabeling/disordering on Cholec80. These results demonstrate checkpoint loss dynamics as an effective diagnostic for temporal annotation auditing.
Praditha Alwis, Soumyadeep Chandra, Deepak Ravikumar et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Understanding how the structure of language can be learned from sentences alone is a central question in both cognitive science and machine learning. Studies of the internal representations of Large Language Models (LLMs) support their ability to parse text when predicting the next word, while representing semantic notions independently of surface form. Yet, which data statistics make these feats possible, and how much data is required, remain largely unknown. Probabilistic context-free grammars (PCFGs) provide a tractable testbed for studying these questions. However, prior work has focused either on the post-hoc characterization of the parsing-like algorithms used by trained networks; or on the learnability of PCFGs with fixed syntax, where parsing is unnecessary. Here, we (i) introduce a tunable class of PCFGs in which both the degree of ambiguity and the correlation structure across scales can be controlled; (ii) provide a learning mechanism -- an inference algorithm inspired by the structure of deep convolutional networks -- that links learnability and sample complexity to specific language statistics; and (iii) validate our predictions empirically across deep convolutional and transformer-based architectures. Overall, we propose a unifying framework where correlations at different scales lift local ambiguities, enabling the emergence of hierarchical representations of the data.
Jack T. Parley, Francesco Cagnetta, Matthieu Wyart· 0 citations
Functional bilevel optimization (FBO) provides a powerful framework for hierarchical learning in function spaces, yet current methods are limited to static offline settings and perform suboptimally in online, non-stationary scenarios. We propose SmoothFBO, the first algorithm for non-stationary FBO with both theoretical guarantees and practical scalability. SmoothFBO introduces a time-smoothed stochastic hypergradient estimator that reduces variance through a window parameter, enabling stable outer-loop updates with sublinear regret. Importantly, the classical parametric bilevel case is a special reduction of our framework, making SmoothFBO a natural extension to online, non-stationary settings. Empirically, SmoothFBO consistently outperforms existing FBO methods in non-stationary hyperparameter optimization and model-based reinforcement learning, demonstrating its practical effectiveness. Together, these results establish SmoothFBO as a general, theoretically grounded, and practically viable foundation for bilevel optimization in online, non-stationary scenarios.
Jason Bohne, Ieva Petrulionyte, Michael Arbel et al.· 0 citations
This work develops an active learning framework to intelligently enrich data-driven reduced-order models (ROMs) of parametric dynamical systems, which can serve as the foundation of virtual assets in a digital twin. Data-driven ROMs are explainable, computationally efficient scientific machine learning models that aim to preserve the underlying physics of complex dynamical simulations. Since the quality of data-driven ROMs is sensitive to the quality of the limited training data, we seek to identify training parameters for which using the associated training data results in the best possible parametric ROM. Our approach uses the operator inference methodology, a regression-based strategy which can be tailored to particular parametric structure for a large class of problems. We establish a probabilistic version of parametric operator inference, casting the learning problem as a Bayesian linear regression. Prediction uncertainties stemming from the resulting probabilistic ROM solutions are used to design a sequential adaptive sampling scheme to select new training parameter vectors that promote ROM stability and accuracy globally in the parameter domain. We conduct numerical experiments for several nonlinear parametric systems of partial differential equations and compare the results to ROMs trained on random parameter samples. The results demonstrate that the proposed adaptive sampling strategy consistently yields more stable and accurate ROMs than random sampling does under the same computational budget.
Shane A. McQuarrie, Mengwu Guo, Anirban Chaudhuri· 0 citations
We study the problem of selecting the best heterogeneous treatment effect (HTE) estimator from a collection of candidates in settings where the treatment effect is fundamentally unobserved. We cast estimator selection as a multiple testing problem and introduce a ground-truth-free procedure based on a cross-fitted, exponentially weighted test statistic. A key component of our method is a two-way sample splitting scheme that decouples nuisance estimation from weight learning and ensures the stability required for valid inference. Leveraging a stability-based central limit theorem, we establish asymptotic familywise error rate control under mild regularity conditions. Empirically, our procedure provides reliable error control while substantially reducing false selections compared with commonly used methods across ACIC 2016, IHDP, and Twins benchmarks, demonstrating that our method is feasible and powerful even without ground-truth treatment effects.
Neural networks with ReLU activations are a widely used model in machine learning. It is thus important to have a profound understanding of the properties of the functions computed by such networks. Recently, there has been increasing interest in the (parameterized) computational complexity of determining these properties. In this work, we close several gaps and resolve an open problem posed by Froese et al. [COLT '25] regarding the parameterized complexity of various problems related to network verification. In particular, we prove that, for all $\ell\ge 2$, deciding positivity (and thus surjectivity) of a function $f:\mathbb{R}^d\to\mathbb{R}$ computed by an $\ell$-layer ReLU network is W[$\ell-1$]-hard when parameterized by the input dimension $d$. The case $\ell=2$ implies that zonotope non-containment (a problem that is of independent interest in computational geometry, control theory, and robotics) is W[1]-hard with respect to the ambient dimension $d$. Moreover, we show that approximating the maximum within any multiplicative factor and computing the $L_p$-Lipschitz constant for $p\in(0,\infty]$ in $\ell$-layer networks is NP-hard and W[$\ell-1$]-hard with respect to $d$. For $\ell\ge 3$, approximating the $L_p$-Lipschitz constant is NP- and W[$\ell-2$]-hard. We further show that the above problems are NP- and W[$t$]-hard (for all $t\ge 1$) with respect to $\ell$ for constant $d$. Notably, our hardness results imply that the naive enumeration-based methods for these fundamental problems running in $n^{(\ell-1) d}\cdot\operatorname{poly}(N)$ time are all essentially optimal under the Exponential Time Hypothesis.
Vincent Froese, Moritz Grillo, Christoph Hertrich et al.· 0 citations
Hamiltonian dynamics describe a wide range of physical systems. As such, data-driven simulations of Hamiltonian systems are important for many scientific and engineering problems. In this work, we propose kernel-based methods for identifying and forecasting Hamiltonian systems directly from trajectory data. We present two approaches: a 2-step method that reconstructs trajectories before learning the Hamiltonian, and a 1-step method that jointly infers both. Across several benchmark systems, including mass-spring dynamics, a nonlinear pendulum, and the Henon-Heiles system, we demonstrate that our framework achieves accurate, data-efficient predictions and outperforms 2-step kernel-based baselines, particularly in scarce-data regimes, while preserving the Hamiltonian structure. Moreover, we prove a priori error estimates, ensuring reliability of the learned models. We also provide a more general, problem-agnostic numerical framework that goes beyond Hamiltonian systems and can be used for data-driven learning of arbitrary dynamical systems.
Yasamin Jalalian, Mostafa Samir, Boumediene Hamzi et al.· 0 citations
We study counterfactual regression, which maps features to outcomes under hypothetical scenarios that differ from those observed in the data. This problem is central to decision-making under distribution shift, where treatment patterns may change at deployment. We develop a semiparametric framework for counterfactual regression along a prespecified incremental-intervention path. The target is a finite-dimensional constrained projection of counterfactual risk, estimated using cross-fitted influence-function representations of the program components. For smooth programs with fixed constraints and finite-dimensional programs with estimated linear constraints, we establish consistency and local stability of the optimizer under class-specific conditions, and derive pointwise and uniform first-order expansions. These results yield asymptotically valid inference, including simultaneous confidence bands for the counterfactual regression path. Simulations and an application to SMS reminders illustrate the finite-sample performance and practical applicability of the proposed approach.
Purpose: Uncertainty Quantification (UQ) has gained traction in an attempt to improve the interpretability and robustness of machine learning predictions. Specifically (medical) biosignals such as electroencephalography (EEG), electrocardiography (ECG), electrooculography (EOG), and electromyography (EMG) could benefit from good UQ, since these suffer from a poor signal-to-noise ratio, and good human interpretability is pivotal for medical applications. To determine how uncertainty estimation can be used for biosignal tasks, we investigate current methods, use cases, applications, evaluations, and uncertainty measures.
Methods: In this paper, we systematically review the state of the art of applying Uncertainty Quantification to Machine Learning tasks in the biosignal domain. All works from Web of Science, Scopus, IEEE XPlore and PsycINFO that discuss uncertainty in Machine Learning on one of the aforementioned biosignals is included.
Results: We present various methods, shortcomings, uncertainty measures and theoretical frameworks that currently exist in this application domain based on the 53 reviewed papers and related literature. We address misconceptions in the field, provide recommendations for future work, and discuss gaps in the literature in relation to diagnostic implementations as well as control for prostheses or brain-computer interfaces.
Conclusion: Overall it can be concluded that promising UQ methods are available, but that research is needed on how people and systems may interact with an uncertainty-model in a (clinical) environment.
Ivo Pascal de Jong, Andreea Ioana Sburlea, Matias Valdenegro-Toro· 0 citations
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
