Diffusion language models generate code by repeatedly updating a partially masked sequence. We ask whether their internal activations encode code correctness and whether that information can improve generation. Across six diffusion models, linear probes distinguish passing from failing attempts, with the strongest read...
A. Miglani, Samrath Singh Chadha, Kevin Li et al.· 0 citations
Across six diffusion models, linear probes distinguish passing from failing attempts, with the strongest reads generally appearing beyond the early layers, and probe point estimates offer no consistent advantage in comparisons with model confidence.
A. Miglani, Samrath Singh Chadha, Kevin Li et al.· 0 citations
This work follows probe accuracy, model output, and steering response across public pretraining and post-training checkpoints and explains why decodability on errors alone cannot establish discarded output information.
Manas Venkata Sai Ravulapalli, Samrath Singh Chadha· 0 citations
This work derives why interference can lower measured capacity by reducing single-binding recall even when the load-dependent recall profile is unchanged, and characterize capacity at the model's query interface.
Manas Venkata Sai Ravulapalli, Samrath Singh Chadha· 0 citations
Where recent studies report that probe-detected errors are resistant to interventions, it is found that in-context binding is a setting in which probes are actionable.
Manas Venkata Sai Ravulapalli, Samrath Singh Chadha, Abhinav M. Hari· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.