When you ask a vision-language model (VLM) a question with an image, it answers your question based on the image. But to act means to choose: saying one thing means choosing to respond not say another. This becomes an issue when the image contains multiple features that equally relate to the question at hand: how would...
Nurbüke Teker, Rui Xiao, Zeynep Akata et al.· 0 citations
A central challenge in continual learning is to acquire new knowledge without forgetting what the model has already learned. This challenge appears in language model training when training data comes from various domain-, user-, or task-specific distributions that are encountered unevenly over time. In such settings, s...
Anton Baumann, Jonas Hübotter, Zeynep Akata et al.· 0 citations
Text-to-image (T2I) models are judged by benchmarks that measure whether requested content appears, but these benchmarks largely overlook the complementary ability to satisfy negated constraints, for example, generating"a non-red cup."Measuring negation raises challenges not faced by affirmation-based benchmarks and re...
Omar Elfatairy, M. A. Bravo, Jessica Bader et al.· 0 citations
Agent evaluations increasingly go beyond a single success rate, reporting metrics such as cost, consistency, and robustness. Yet they typically treat the agent itself as fixed. In practice, an agent is a configurable system: users decide what to tell it, how long to let it run, and which model to use, and each of these...
Luis Wiedmann, Leander Girrbach, Cordelia Schmid et al.· 0 citations
Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time, outdated knowledge can pose safety risks. Existing evaluations of knowledge integration focus on factual recall, offering limited insight into whether newly integrated...
Lukas Thede, Yash Kumar Atri, David Chen et al.· 0 citations
Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus efficient on standard hardware. Existing methods, however, choose the subspace to remove from each weight matrix with local closed-form crit...
Massimo Bini, Anders Christensen, S. Alaniz et al.· 0 citations
Large language models (LLMs) implicitly infer attributes of their users and adapt their behavior accordingly, yet these beliefs remain difficult to inspect and causally manipulate. We introduce Belief Self-Distillation (BSD), a unified read-write framework that bridges linear and causal probing by learning a compact us...
Ali Holmov, Yi-Ran Huang, Kirill Bykov et al.· 0 citations
This work introduces a semi-supervised setting in which pretrained unimodal encoders are aligned using a small number of image-text pairs together with large amounts of unpaired data and proposes SOTAlign, a two-stage framework that first recovers a coarse shared geometry from limited paired data using a linear teacher...
Simon Roschmann, Paul Krzakala, Sonia Mazelet et al.· arXiv.org· 3 citations· ⚡1
This work proposes Time Vision Transformer (TiViT), the first framework that successfully unlocks the representational power of frozen Vision Transformers pretrained on large-scale image datasets for time series classification.
Simon Roschmann, Quentin Bouniot, Vasilii Feofanov et al.· 7 citations· ⚡2
VisLens (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head, and which matches or exceeds prior baselines while delivering a substantial latency advantage.
Jingfeng He, Sanghwan Kim, Zeynep Akata· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
