TensorDecompositions4PINNs

TL;DR:

Functional Tensor Decompositions explores how tensor decomposition techniques can unlock new possibilities for variable separation in Physics-Informed Neural Networks (PINNs), overcoming the curse of dimensionality in solving high-dimensional PDEs

Abstract:

Physics-Informed Neural Networks (PINNs) have shown great promise in approximating partial differential equations (PDEs), although they remain constrained by the curse of dimensionality. In this paper, we propose a generalized PINN version of the classical variable separable method. To do this, we first show that, using the universal approxima- tion theorem, a multivariate function can be approximated by the outer product of neural networks, whose inputs are separated variables. We leverage tensor decomposition forms to separate the variables in a PINN setting. By employing Canonic Polyadic (CP), Tensor-Train (TT), and Tucker decomposition forms within the PINN framework, we create ro- bust architectures for learning multivariate functions from separate neu- ral networks connected by outer products. Our methodology significantly enhances the performance of PINNs, as evidenced by improved results on complex high-dimensional PDEs, including the 3D Helmholtz and 5D Poisson equations, among others. This research underscores the poten- tial of tensor decomposition-based variably separated PINNs to surpass the state-of-the-art, offering a compelling solution to the dimensionality challenge in PDE approximation.

Links:

For more details, refer to our paper:

• Published version: Springer Link
• Preprint version: arXiv

Required Packages:

• tqdm
• jax
• pina
• matplotlib

Citation:

If you find this work useful, please consider citing:

@inproceedings{vemuri2024,
author = {Vemuri, Sai Karthikeya and B\"{u}chner, Tim and Niebling, Julia and Denzler, Joachim},
title = {Functional Tensor Decompositions for Physics-Informed Neural Networks},
year = {2024},
isbn = {978-3-031-78388-3},
publisher = {Springer-Verlag},
address = {Berlin, Heidelberg},
url = {https://doi.org/10.1007/978-3-031-78389-0_3},
doi = {10.1007/978-3-031-78389-0_3},
abstract = {Physics-Informed Neural Networks (PINNs) have shown continuous and increasing promise in approximating partial differential equations (PDEs), although they remain constrained by the curse of dimensionality. In this paper, we propose a generalized PINN version of the classical variable separable method. To do this, we first show that, using the universal approximation theorem, a multivariate function can be approximated by the outer product of neural networks, whose inputs are separated variables. We leverage tensor decomposition forms to separate the variables in a PINN setting. By employing Canonic Polyadic (CP), Tensor-Train (TT), and Tucker decomposition forms within the PINN framework, we create robust architectures for learning multivariate functions from separate neural networks connected by outer products. Our methodology significantly enhances the performance of PINNs, as evidenced by improved results on complex high-dimensional PDEs, including the 3d Helmholtz and 5d Poisson equations, among others. This research underscores the potential of tensor decomposition-based variably separated PINNs to surpass the state-of-the-art, offering a compelling solution to the dimensionality challenge in PDE approximation.},
booktitle = {Pattern Recognition: 27th International Conference, ICPR 2024, Kolkata, India, December 1–5, 2024, Proceedings, Part XXV},
pages = {32–46},
numpages = {15},
keywords = {Tensor Decomposition, Physics-Informed Neural Networks},
location = {Kolkata, India}
}

@article{vemuri2026scalable,
  author   = {Sai Karthikeya Vemuri and Tim B{\"u}chner and Julia Niebling and Joachim Denzler},
  title    = {Scalable and Expressive Physics-Informed Neural Networks via Functional Tensor Decomposition},
  journal  = {Pattern Recognition Letters},
  year     = {2026},
  issn     = {0167-8655},
  doi      = {https://doi.org/10.1016/j.patrec.2026.06.027},
  url      = {https://www.sciencedirect.com/science/article/pii/S0167865526002291},
  abstract = {Physics-Informed Neural Networks (PINNs) offer a promising framework for solving partial differential equations (PDEs) by embedding physical laws into neural network training. In our prior work, we introduced Functional Tensor Decomposition PINNs (FTD-PINNs), which leverage tensor decomposition to improve the scalability and efficiency of PINNs, especially for high-dimensional PDEs. This work extends our previous study on Functional Tensor Decomposition PINNs by systematically examining how decomposition structure and backend activations affect performance, efficiency, and stability across representative partial differential equations. Using Helmholtz and Klein-Gordon systems as benchmarks, we compare three decomposition modes (CP, Tucker, Tensor-Train) combined with four activation backends (Tanh, Tanh + PE, SIREN, and WIRE). Multi-seed and compute-aware evaluations reveal consistent accuracy-efficiency trade-offs: Tensor-Train provides the most balanced decomposition, while frequency-aware backends such as WIRE and SIREN improve convergence for oscillatory regimes. Pareto analyses across rank and collocation density highlight a clear inflection point of diminishing returns and show that backend choice shifts the optimal configuration across PDE types. Together, these results extend the original FTD-PINN framework with a broader empirical foundation and provide practical guidance for selecting decomposition ranks, collocation densities, and backend activations for efficient PINN design.},
  code     = {https://github.com/cvjena/TensorDecompositions4PINNs},
  note     = {},
}