Mathematical Foundation of Interpretable Equivariant Surrogate Models

This paper introduces a rigorous mathematical framework for neural network explainability, and more broadly for the explainabil-ity of equivariant operators called Group Equivariant Operators (GEOs), based on Group Equivariant Non-Expansive Operators (GENEOs) transformations. The central concept involves quantifying the distance between GEOs by measuring the non-commutativity of specific diagrams. Additionally, the paper proposes a definition of interpretability of GEOs according to a complexity measure that can be defined according to each user’s preferences. Moreover, we explore the formal properties of thisframework and show how it can be applied in classical machine learningscenarios, like image classification with convolutional neural networks.