DisenTE, a sequence-conditioned neural model that combines separate sequence and context branches with a sparse low-rank pattern–context channel, is presented, a sequence-conditioned neural model that combines separate sequence and context branches with a sparse low-rank pattern–context channel.
Abstract
Partially observed object-by-context matrices arise across data-rich science, where dominant object effects can obscure smaller but informative context-dependent variation. We study this problem in a translation-efficiency atlas of 9,494 5′ UTRs across 78 cellular and tissue contexts. We present DisenTE, a sequence-conditioned neural model that combines separate sequence and context branches with a sparse low-rank pattern–context channel. Each module pairs a sequence-derived activation with context-specific deployment weights, forming a dictionary whose sequence and context components can be examined separately. Under five-fold within-panel entry masking, DisenTE achieves a UTR-centered residual Spear-man correlation of 0.641±0.005, compared with 0.304±0.003 for the strongest reference model. The learned dictionary retains 11 of 20 candidate modules. CTM 6 has the largest overlap with an external TOP set and a cap-proximal pyrimidine pattern; CTMs 5 and 7 also overlap the set but have purine-containing consensuses. The evidence supports CTM 6 as a TOP sequence anchor and CTMs 5 and 7 as TOP-set-associated factors. On this dataset, DisenTE improves completion over the evaluated references and provides module-level summaries of its fitted context-dependent variation.
Topic modeling still struggles to combine contextual expressiveness with lexical interpretability. Sparse models are transparent but semantically limited, whereas dense contextual models capture meaning at the cost of direct interpretability. We argue that Learned Sparse Representations (LSR) offer a promising alternat...
A. Machado, C. França, Antônio Pereira et al.· 0 citations
NDR (Neuron Representation Disentanglement) is proposed, a two-stage hybrid framework that couples training-time specialization with inference-time filtering that effectively purifies semantic representations by mitigating syntactic interference, paving the way for more robust cross-lingual generalization.
Jia-Rui Zhang, Yi-Fan Deng· International Conference on...· 0 citations
In-context learning (ICL) can be amortized into latent objects (task vectors, function vectors, context vectors) that recover few-shot behavior at zero-shot inference cost, but recent theory shows a static vector acts as a single synthetic demonstration and must fail on high-rank mappings such as word-level bijections....
Dense linear probes assess task readability from language model (LM) representations, but do not directly reveal how many features suffice for prediction. We measure readout sparsity with \(k^*\), the smallest feature count found by a fixed search protocol that recovers 95% of dense-probe AUROC above chance. We evaluat...
LOCUS, a method that selects a task-aware low-rank adaptation subspace to minimize output-token cost subject to a utility constraint, is presented, a method that selects a task-aware low-rank adaptation subspace to minimize output-token cost subject to a utility constraint.
Dongfang Zhao· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.