Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

Journal Article · Sat Nov 03 00:00:00 EDT 2018 · Journal of Computational Physics

DOI:https://doi.org/10.1016/j.jcp.2018.10.045· OSTI ID:1595805

^[1]; ^[2]; Karniadakis, George Em ^[3]

Brown Univ., Providence, RI (United States); University of Pennsylvania
Univ. of Pennsylvania, Philadelphia, PA (United States)
Brown Univ., Providence, RI (United States)

Hejre, we introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct types of algorithms, namely continuous time and discrete time models. The first type of models forms a new family of data-efficient spatio-temporal function approximators, while the latter type allows the use of arbitrarily accurate implicit Runge–Kutta time stepping schemes with unlimited number of stages. Moreover, the effectiveness of the proposed framework is demonstrated through a collection of classical problems in fluids, quantum mechanics, reaction–diffusion systems, and the propagation of nonlinear shallow-water waves.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of Pennsylvania, Philadelphia, PA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

Grant/Contract Number:: SC0019116

OSTI ID:: 1595805

Alternate ID(s):: OSTI ID: 1635941

Journal Information:: Journal of Computational Physics, Journal Name: Journal of Computational Physics Journal Issue: C Vol. 378; ISSN 0021-9991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

A hybrid neural network-first principles approach to process modeling Psichogios, Dimitris C.; Ungar, Lyle H. AIChE Journal, Vol. 38, Issue 10 https://doi.org/10.1002/aic.690381003	journal	October 1992
When Are Quasi-Monte Carlo Algorithms Efficient for High Dimensional Integrals? Sloan, Ian H.; Woźniakowski, Henryk Journal of Complexity, Vol. 14, Issue 1 https://doi.org/10.1006/jcom.1997.0463	journal	March 1998
Neural Network Modeling for Near Wall Turbulent Flow Milano, Michele; Koumoutsakos, Petros Journal of Computational Physics, Vol. 182, Issue 1 https://doi.org/10.1006/jcph.2002.7146	journal	October 2002
On the limited memory BFGS method for large scale optimization Liu, Dong C.; Nocedal, Jorge Mathematical Programming, Vol. 45, Issue 1-3 https://doi.org/10.1007/BF01589116	journal	August 1989
Why Does Deep and Cheap Learning Work So Well? Lin, Henry W.; Tegmark, Max; Rolnick, David Journal of Statistical Physics, Vol. 168, Issue 6 https://doi.org/10.1007/s10955-017-1836-5	journal	July 2017
Spectral and finite difference solutions of the Burgers equation Basdevant, C.; Deville, M.; Haldenwang, P. Computers & Fluids, Vol. 14, Issue 1 https://doi.org/10.1016/0045-7930(86)90036-8	journal	January 1986
Multilayer feedforward networks are universal approximators Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert Neural Networks, Vol. 2, Issue 5 https://doi.org/10.1016/0893-6080(89)90020-8	journal	January 1989
A paradigm for data-driven predictive modeling using field inversion and machine learning Parish, Eric J.; Duraisamy, Karthik Journal of Computational Physics, Vol. 305 https://doi.org/10.1016/j.jcp.2015.11.012	journal	January 2016
Inferring solutions of differential equations using noisy multi-fidelity data Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em Journal of Computational Physics, Vol. 335 https://doi.org/10.1016/j.jcp.2017.01.060	journal	April 2017
Machine learning of linear differential equations using Gaussian processes Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em Journal of Computational Physics, Vol. 348 https://doi.org/10.1016/j.jcp.2017.07.050	journal	November 2017
Sparse grids Bungartz, Hans-Joachim; Griebel, Michael Acta Numerica, Vol. 13 https://doi.org/10.1017/S0962492904000182	journal	May 2004
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance Ling, Julia; Kurzawski, Andrew; Templeton, Jeremy Journal of Fluid Mechanics, Vol. 807 https://doi.org/10.1017/jfm.2016.615	journal	October 2016
Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning Alipanahi, Babak; Delong, Andrew; Weirauch, Matthew T. Nature Biotechnology, Vol. 33, Issue 8 https://doi.org/10.1038/nbt.3300	journal	July 2015
Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty Ling, J.; Templeton, J. Physics of Fluids, Vol. 27, Issue 8 https://doi.org/10.1063/1.4927765	journal	August 2015
Discovering governing equations from data by sparse identification of nonlinear dynamical systems Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan Proceedings of the National Academy of Sciences, Vol. 113, Issue 15 https://doi.org/10.1073/pnas.1517384113	journal	March 2016
Large Sample Properties of Simulations Using Latin Hypercube Sampling Stein, Michael Technometrics, Vol. 29, Issue 2 https://doi.org/10.1080/00401706.1987.10488205	journal	May 1987
Understanding deep convolutional networks Mallat, Stéphane Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 374, Issue 2065 https://doi.org/10.1098/rsta.2015.0203	journal	April 2016
Artificial neural networks for solving ordinary and partial differential equations Lagaris, I. E.; Likas, A.; Fotiadis, D. I. IEEE Transactions on Neural Networks, Vol. 9, Issue 5 https://doi.org/10.1109/72.712178	journal	January 1998
Data-driven discovery of partial differential equations Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L. Science Advances, Vol. 3, Issue 4 https://doi.org/10.1126/sciadv.1602614	journal	April 2017
Human-level concept learning through probabilistic program induction Lake, B. M.; Salakhutdinov, R.; Tenenbaum, J. B. Science, Vol. 350, Issue 6266 https://doi.org/10.1126/science.aab3050	journal	December 2015
Bayesian Numerical Homogenization Owhadi, Houman Multiscale Modeling & Simulation, Vol. 13, Issue 3 https://doi.org/10.1137/140974596	journal	January 2015
Multifidelity Information Fusion Algorithms for High-Dimensional Systems and Massive Data sets Perdikaris, Paris; Venturi, Daniele; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 38, Issue 4 https://doi.org/10.1137/15M1055164	journal	January 2016
Wavelet Scattering Regression of Quantum Chemical Energies Hirn, Matthew; Mallat, Stéphane; Poilvert, Nicolas Multiscale Modeling & Simulation, Vol. 15, Issue 2 https://doi.org/10.1137/16M1075454	journal	January 2017
Brittleness of Bayesian inference under finite information in a continuous world Owhadi, Houman; Scovel, Clint; Sullivan, Tim Electronic Journal of Statistics, Vol. 9, Issue 1 https://doi.org/10.1214/15-EJS989	journal	January 2015

Data‐Driven Materials Science: Status, Challenges, and Perspectives Himanen, Lauri; Geurts, Amber; Foster, Adam Stuart Advanced Science, Vol. 6, Issue 21 https://doi.org/10.1002/advs.201900808	journal	September 2019
Data‐Driven Materials Science: Status, Challenges, and Perspectives Himanen, Lauri; Geurts, Amber; Foster, Adam Stuart Advanced Science, Vol. 7, Issue 2 https://doi.org/10.1002/advs.201903667	journal	November 2019
Simulation of nonlinear fractional dynamics arising in the modeling of cognitive decision making using a new fractional neural network Hadian Rasanan, Amir Hosein; Bajalan, Nastaran; Parand, Kourosh Mathematical Methods in the Applied Sciences, Vol. 43, Issue 3 https://doi.org/10.1002/mma.5981	journal	December 2019
A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence Pawar, Suraj; San, Omer; Rasheed, Adil Theoretical and Computational Fluid Dynamics, Vol. 34, Issue 4 https://doi.org/10.1007/s00162-019-00512-z	journal	January 2020
General solutions for nonlinear differential equations: a rule-based self-learning approach using deep reinforcement learning Wei, Shiyin; Jin, Xiaowei; Li, Hui Computational Mechanics, Vol. 64, Issue 5 https://doi.org/10.1007/s00466-019-01715-1	journal	May 2019
Prediction of aerodynamic flow fields using convolutional neural networks Bhatnagar, Saakaar; Afshar, Yaser; Pan, Shaowu Computational Mechanics, Vol. 64, Issue 2 https://doi.org/10.1007/s00466-019-01740-0	journal	June 2019
Neural networks catching up with finite differences in solving partial differential equations in higher dimensions Avrutskiy, Vsevolod I. Neural Computing and Applications, Vol. 32, Issue 17 https://doi.org/10.1007/s00521-020-04743-8	journal	January 2020
Multiscale Modeling Meets Machine Learning: What Can We Learn? Peng, Grace C. Y.; Alber, Mark; Buganza Tepole, Adrian Archives of Computational Methods in Engineering https://doi.org/10.1007/s11831-020-09405-5	journal	February 2020
Interactive design and variation of hull shapes: pros and cons of different CAD approaches Vernengo, Giuliano; Villa, Diego; Gaggero, Stefano International Journal on Interactive Design and Manufacturing (IJIDeM), Vol. 14, Issue 1 https://doi.org/10.1007/s12008-019-00613-3	journal	September 2019
Solving partial differential equations by a supervised learning technique, applied for the reaction–diffusion equation Zakeri, Behzad; Khashehchi, Morteza; Samsam, Sanaz SN Applied Sciences, Vol. 1, Issue 12 https://doi.org/10.1007/s42452-019-1630-x	journal	November 2019
Massive computational acceleration by using neural networks to emulate mechanism-based biological models Wang, Shangying; Fan, Kai; Luo, Nan Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-12342-y	journal	September 2019
Integrating machine learning and multiscale modeling—perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences Alber, Mark; Buganza Tepole, Adrian; Cannon, William R. npj Digital Medicine, Vol. 2, Issue 1 https://doi.org/10.1038/s41746-019-0193-y	journal	November 2019
Deep neural network learning of complex binary sorption equilibria from molecular simulation data Sun, Yangzesheng; DeJaco, Robert F.; Siepmann, J. Ilja Chemical Science, Vol. 10, Issue 16 https://doi.org/10.1039/c8sc05340e	journal	January 2019
A deep learning enabler for nonintrusive reduced order modeling of fluid flows Pawar, S.; Rahman, S. M.; Vaddireddy, H. Physics of Fluids, Vol. 31, Issue 8 https://doi.org/10.1063/1.5113494	journal	August 2019
Memory embedded non-intrusive reduced order modeling of non-ergodic flows Ahmed, Shady E.; Rahman, Sk. Mashfiqur; San, Omer Physics of Fluids, Vol. 31, Issue 12 https://doi.org/10.1063/1.5128374	journal	December 2019
Solving Fokker-Planck equation using deep learning Xu, Yong; Zhang, Hao; Li, Yongge Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 30, Issue 1 https://doi.org/10.1063/1.5132840	journal	January 2020
Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data Vaddireddy, Harsha; Rasheed, Adil; Staples, Anne E. Physics of Fluids, Vol. 32, Issue 1 https://doi.org/10.1063/1.5136351	journal	January 2020
Uniformly accurate machine learning-based hydrodynamic models for kinetic equations Han, Jiequn; Ma, Chao; Ma, Zheng Proceedings of the National Academy of Sciences, Vol. 116, Issue 44 https://doi.org/10.1073/pnas.1909854116	journal	October 2019
Neural network models for the anisotropic Reynolds stress tensor in turbulent channel flow Fang, Rui; Sondak, David; Protopapas, Pavlos Journal of Turbulence, Vol. 21, Issue 9-10 https://doi.org/10.1080/14685248.2019.1706742	journal	December 2019
Machine learning and artificial intelligence to aid climate change research and preparedness Huntingford, Chris; Jeffers, Elizabeth S.; Bonsall, Michael B. Environmental Research Letters, Vol. 14, Issue 12 https://doi.org/10.1088/1748-9326/ab4e55	journal	November 2019
Nonintrusive reduced order modeling framework for quasigeostrophic turbulence Rahman, Sk. M.; Pawar, S.; San, O. Physical Review E, Vol. 100, Issue 5 https://doi.org/10.1103/physreve.100.053306	journal	November 2019
Discovering Physical Concepts with Neural Networks Iten, Raban; Metger, Tony; Wilming, Henrik Physical Review Letters, Vol. 124, Issue 1 https://doi.org/10.1103/physrevlett.124.010508	journal	January 2020
Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations Raissi, Maziar; Yazdani, Alireza; Karniadakis, George Em Science, Vol. 367, Issue 6481 https://doi.org/10.1126/science.aaw4741	journal	January 2020
Discovering a universal variable-order fractional model for turbulent Couette flow using a physics-informed neural network Mehta, Pavan Pranjivan; Pang, Guofei; Song, Fangying Fractional Calculus and Applied Analysis, Vol. 22, Issue 6 https://doi.org/10.1515/fca-2019-0086	journal	December 2019
Modal Analysis of Fluid Flows: Applications and Outlook Taira, Kunihiko; Hemati, Maziar S.; Brunton, Steven L. AIAA Journal, Vol. 58, Issue 3 https://doi.org/10.2514/1.j058462	journal	March 2020
Modal Analysis of Fluid Flows: Applications and Outlook Taira, Kunihiko; Hemati, Maziar S.; Brunton, Steven L. arXiv https://doi.org/10.48550/arxiv.1903.05750	preprint	January 2019
Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations Han, Jiequn; Ma, Chao; Ma, Zheng arXiv https://doi.org/10.48550/arxiv.1907.03937	text	January 2019
A deep learning enabler for non-intrusive reduced order modeling of fluid flows Pawar, S.; Rahman, S. M.; Vaddireddy, H. arXiv https://doi.org/10.48550/arxiv.1907.04945	text	January 2019
Data-driven materials science: status, challenges and perspectives Himanen, Lauri; Geurts, Amber; Foster, Adam S. arXiv https://doi.org/10.48550/arxiv.1907.05644	text	January 2019
Neural Network Models for the Anisotropic Reynolds Stress Tensor in Turbulent Channel Flow Fang, Rui; Sondak, David; Protopapas, Pavlos arXiv https://doi.org/10.48550/arxiv.1909.03591	text	January 2019
A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence Pawar, Suraj; San, Omer; Rasheed, Adil arXiv https://doi.org/10.48550/arxiv.1910.07132	text	January 2019
Memory embedded non-intrusive reduced order modeling of non-ergodic flows Ahmed, Shady E.; Rahman, Sk. Mashfiqur; San, Omer arXiv https://doi.org/10.48550/arxiv.1910.07649	text	January 2019
Solving Fokker-Planck equation using deep learning Xu, Yong; Zhang, Hao; Li, Yongge arXiv https://doi.org/10.48550/arxiv.1910.10503	text	January 2019
Multiscale modeling meets machine learning: What can we learn? Peng, Grace C. Y.; Alber, Mark; Tepole, Adrian Buganza arXiv https://doi.org/10.48550/arxiv.1911.11958	preprint	January 2019

Similar Records

Learning Only on Boundaries: A Physics-Informed Neural Operator for Solving Parametric Partial Differential Equations in Complex Geometries

Journal Article · Thu Feb 15 23:00:00 EST 2024 · Neural Computation · OSTI ID:2339519

A Heuristic Fast Method to Solve the Nonlinear Schroedinger Equation in Fiber Bragg Gratings with Arbitrary Shape Input Pulse

Journal Article · Thu Aug 13 00:00:00 EDT 2009 · AIP Conference Proceedings · OSTI ID:21317003

Nonlinear analysis of a simply-supported composite beam under random excitations

Conference · Fri Dec 30 23:00:00 EST 1994 · OSTI ID:481991

Related Subjects

97 MATHEMATICS AND COMPUTING
Data-driven scientific computing
Machine learning
Nonlinear dynamics
Predictive modeling
Runge–Kutta methods

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

References (24)

Cited By (34)

Similar Records

Related Subjects

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

Citation Formats

References (24)

Cited By (34)

Similar Records

Related Subjects