Limitations of Automated Simulatability: LLM Simulators Can Bypass Explanations
This work qualitatively replicate and extend ConSim's ranking of explanation methods across the tested datasets, explanation families, and simulator LLMs, and identifies two limitations: when class names are meaningful, simulators can obtain high simulatability by solving the classification task directly, without relyi...