Discrete Differential Geometry as a Regularizer for Deep Learning on Manifold-Valued Data

Abstract

Deep learning models created today usually fail to handle non-Euclidean high-dimensional data due to manifold drift. Other common regularization methods like L2 weight decay and Dropout make the assumption of a flat Euclidean geometry and the lack of an inbuilt topological structure of the data manifold. This work introduces a new regularization model which is based on Discrete Differential Geometry (DDG). The application of Dirichlet and Willmore energy functional is added to the neural network loss function by embedding simplicial complexes formed with k-nearest-neighbor (k-NN) graph triangulation with the Discrete Laplace-Beltrami operator. This brings in geometrical smoothness and topological regularity in training. Empirical results using the ModelNet40 and QM9 show that it gains the 3D classification accuracy on a 1.2 percentage point on the baseline of the PointNet++, 21 percent reduction in the error rate of predicting the molecular properties, 30 reduction in the time required to converge, and a Stability Score of 0.88 using controlled Gaussian vertex perturbation compared to an 0.45 score, which is confirmed by an ablation study and a hyperparameter sensitivity analysis, the value of each energy term is useful and provides practical operating ranges.