ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks [Paper]
This paper provides a formal and in-depth study of the propagation failure phenomenon in PINNs, which brings the following progress for PINNs' research:
- Theoretical understanding: This paper proves that the root cause of propagation failure is the lower gradient correlation of PINN models on nearby collocation points.
- Effective backbone: The theoretical finding also inspires us to present a new PINN architecture, named ProPINN, which can effectively unite the gradients of region points for better propagation.
- Significant Performance: ProPINN can reliably resolve PINN failure modes and significantly surpass advanced Transformer-based models with 46% relative improvement.
(1) Propagation Failure is first noticed by Daw et al. (ICML 2023), which (we believe) is one of the fundamental issues of PINNs. It describes a weird situation: as shown below, the equation constraint loss (see residual loss) of PINN is sufficiently small, but the approximated solution is still far from the ground truth (see error map).
Figure 1. Illustration of Propagation Failure.
(2) FEMs vs. PINNs: With a detailed comparison between finite element methods (FEMs) and PINNs, we theoretically prove that FEMs are under active propagation, while PINNs are more prone to suffer propagation failure due to their single-point processing paradigm.
Figure 2. Comparison between FEMs and PINNs.
(3) Theoretical results: We prove that the gradient correlation of PINNs on nearby collocation points is the root cause of propagation failure. As presented in Figure 1, the gradient correlation can also be an accurate criterion to identify propagation failure.
The theoretical finding also inspires us to present a new PINN architecture, named ProPINN. ProPINN includes a multi-region mixing mechanism to augment the previous single-point processing paradigm, which can effectively unite the gradients of region points for better propagation.
Figure 3. Overall design of ProPINN.
- Prepare experiment environments: Python 3.8, PyTorch 1.13.0, CUDA 11.7
pip install -r requirements.txt- Run the scripts under the
./scriptsfolder:
bash ./scripts/convection_ProPINN.shYou can also find the pre-trained checkpoints under the ./checkpoints folder.
Table 1. Comparison between ProPINN and previous methods.
We conduct experiments on standard benchmarks and challenging fluid dynamics.
Figure 4. Showcases on standard benchmarks.
Figure 5. Showcases on fluid dynamics.
@inproceedings{wu2025propinn,
title={ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks},
author={Haixu Wu and Yuezhou Ma and Hang Zhou and Huikun Weng and Jianmin Wang and Mingsheng Long},
booktitle={arXiv preprint arXiv:2502.00803},
year={2025}
}
If you have any questions or want to use the code, please contact wuhx23@mails.tsinghua.edu.cn.
We appreciate the following GitHub repos a lot for their valuable code base or datasets: