S. Im

How Much Do Activation Monitors Benefit from Original Model Computations?

One powerful way to safeguard AI systems is to monitor their internal computations--where signs of undesirable behavior might be more easily detectable than from models'inputs and outputs alone. However, recent studies begin to question the premise that activations provide a unique window into model behavior. In this p...