Approximation Error of Sobolev Regular Functions with tanh Neural Networks: Theoretical Impact on PINNs

Benjamin Girault (1) , Rémi Emonet (2, 3, 1, 4) , Amaury Habrard (2, 3, 1) , Jordan Patracone (1) , Marc Sebban (1)

1 Apprentissage automatique avec intégration des connaissances en ingénierie de surface : théorie et algorithmes - MALICE
2 LHC - Laboratoire Hubert Curien
3 UJM - Université Jean Monnet - Saint-Étienne
4 IUF - Institut universitaire de France

Considering the key role played by derivatives in Partial Differential Equations (PDEs), using the tanh activation function in Physics-Informed Neural Networks(PINNs) yields useful smoothness properties to derive theoretical guarantees in Sobolev norm. In this paper, we conduct an extensive functional analysis, unveiling tighter approximation bounds compared to prior works, especially for higher order PDEs. These better guarantees translate into smaller PINN architectures and improved generalization error with arbitrarily small Sobolev norms of the PDE residuals.

Keywords: Approximation error; Sobolev space; Physics-informed neural network; tanh activation function

Domains: Machine Learning [cs.LG]

Benjamin Girault, Rémi Emonet, Amaury Habrard, Jordan Patracone, Marc Sebban. Approximation Error of Sobolev Regular Functions with tanh Neural Networks: Theoretical Impact on PINNs. 2024. ⟨hal-04518335⟩