Dynamic Topology Dependency Neural Networks (TDNN)

Abstract

This paper introduces Dynamic Topology Dependency Neural Networks (TDNN), a novel neural network architecture designed to address the limitations of traditional static neural networks when processing dynamic and complex data streams. The core concept of TDNN revolves around a dynamically adaptable neural network topology governed by a reinforcement learning algorithm and a dependency graph. The network learns to optimize its internal connections and topology in real-time based on the input data's evolution. This allows TDNN to achieve more efficient and robust representations and processing capabilities compared to conventional neural networks. Specifically, the algorithm adjusts both connection weights and the topology (adding, removing, or modifying connections) guided by a dependency graph that reflects the interdependent activation states of neurons. The dependency graph evolves during training, forming a 'neural topology map' that captures the underlying structure of the input data. This dynamic adaptation enables TDNN to effectively handle non-stationary data scenarios.