Transformation Equivalence of Neural Networks

Abstract

Multilayer perceptrons (MLPs) are considered as a singular model of learning machines. Singularities cause local minima and plateaus in the learning process. I/O-equivalence, where two different MLPs are regarded as the same multivariable function, is an important concept for understanding singularities in neural networks. In this paper, I/O-equivalence is extended to T-equivalence, which is a concept where two MLPs yield the same results through a transformation of input and output. We provide constructive families and procedures for obtaining T-equivalent networks of real-, complex-, and quaternion-valued neural networks. In particular, T-equivalence of quaternion-valued neural networks is much more complicated than that of the others.