The Informationally Integrated Alignment Hypothesis: Integrated Information Aligns with and Predicts Final Reward in Reinforcement Learning Agents
A hallmark of life on Earth is the ability of agents to exert causal power and be drivers of subsequent events. This is key to cognition at all scales. Integrated information, measuring the degree to which an agent exerts unique predictive power on its future, is one consequence of causal power. Indeed, recent discover...