Version of Record: https://www.sciencedirect.

com/science/article/pii/S0045782520302127
Manuscript_3b61981c45657ae8864749c940128a0c

Conservative Physics-Informed Neural Networks On Discrete Domains For

Ameya D. Jagtap1 , Ehsan Kharazmi1 , George Em Karniadakis1,2,∗

Abstract
We propose a conservative physics-informed neural network (cPINN) on discrete domains for nonlinear conservation
laws. Here, the term discrete domain represents the discrete sub-domains obtained after division of the computational
domains, where PINN is applied and the conservation property of cPINN is obtained by enforcing the flux conti-
nuity in the strong form along these sub-domain interfaces. In case of hyperbolic conservation laws, the convective
flux contributes at the interfaces, whereas in case of viscous conservation laws, both convective and diffusive fluxes
contribute. Apart from the flux continuity condition, an average solution (given by two different neural networks) is
also enforced at the common interface between two sub-domains. One can also employ a deep neural network in the
domain, where the solution may have complex structure, whereas a shallow neural network can be used in the sub-
domains with relatively simple and smooth solutions. Another advantage of the proposed method is the additional
freedom it gives in terms of the choice of optimization algorithm and the various training parameters like residual
points, activation function, width and depth of the network etc. Various forms of errors involved in cPINN such as
optimization, generalization and approximation errors and their sources are discussed briefly. In cPINN, locally adap-
tive activation functions are used, hence training the model faster compared to its fixed counterparts. Both, forward
and inverse problems are solved using the proposed method. Various test cases ranging from scalar nonlinear con-
servation laws like Burgers, Kortveg-de-Varies (KdV) equations to systems of conservation laws, like compressible
Euler equations are solved. The lid-driven cavity test case governed by incompressible Navier-Stokes equation is
also solved and the results are compared against a benchmark solution. The proposed method enjoys the property of
domain decomposition with separate neural networks in each sub-domain, and it efficiently lends itself to parallelized
computation, where each sub-domain can be assigned to a different computational node.
Keywords: cPINN, Domain decomposition, Machine learning, Compressible Euler equations, Incompressible
Navier-Stokes equations, Inverse problems.

1. Introduction

In the last ten years, deep learning in the form of deep neural networks has been extensively used in many diverse
disciplines ranging from the classification problem including speech recognition [7], natural language translation [32],
and computer vision [15], to complex regression problems like approximation of nonlinear/discontinuous functions.
5 However, their applications are less explored in the area of scientific computing. Recently, solving the underlying
governing partial differential equations (PDEs) of physical phenomena using deep learning has emerged as a new
field of scientific machine learning thanks to the universal approximation and high expressivity of neural networks
(NNs) as an ansatz for the solution space of governing PDEs; yet such networks in general are ignorant of the
underlying physics. The idea of constructing physics-informed learning machines, which makes use of systematically
10 structured prior information about the solution goes back to earlier study of Owhadi [21], which showed the promising

∗ Corresponding author Emails: ameyadjagtap@gmail.com, ameya jagtap@brown.edu (A.D.Jagtap), ehsan kharazmi@brown.edu

Preprint submitted to Computer Methods in Applied Mechanics and Engineering March 18, 2020

© 2020 published by Elsevier. This manuscript is made available under the Elsevier user license
https://www.elsevier.com/open-access/userlicense/1.0/
 approach of exploiting such prior information. Later, Raissi et al [26, 28] employed Gaussian process regression to
obtain representation of functionals of linear operators, infering the solution accurately and also providing uncertainty
estimates for many physical problems. They further extended this method to nonlinear problems, solution inference
and system identification [23, 24]. More recently in [25], they developed the physics-informed neural networks
15 (PINNs) that can accurately solve both forward problems of approximating the solutions of governing mathematical
model, as well as inverse problems, where the model parameters are inferred from the observed data. The PINN
algorithm infuses the governing equation into the network and thus enriches the loss function by adding a residual term
from that equation, which essentially acts as a penalizing term to constrain the space of admissible solutions. In this
setting, the problem of inferring solutions of PDEs is transformed into an optimization problem of the loss function.
20 A major advantage of this method is providing a mesh-free algorithm as the differential operators in the governing
PDEs are approximated by automatic differentiation [4]. Therefore, PINNs are fundamentally different from the
traditional numerical methods like the finite element method and finite difference method, where the governing PDEs
are eventually discretized over the computational domain. However, there are two limitations of PINNs. The first one
is the accuracy of the solution, i.e, the absolute error does not go below levels of about 10−5 due to the inaccuracy
25 involved in solving high-dimensional non-convex optimization problem that may result in local minima. The second
limitation is the large training cost associated with deep neural networks and long time-integration of the governing
PDEs.
The performance of this algorithm, however, is not yet fully investigated in the literature in solving hyperbolic
conservation laws such as the compressible Euler equations that can admit discontinuous solutions like shock and
30 contact waves even with sufficiently smooth initial conditions. Such discontinuities lead to excessive complications
in choosing the hyper-parameters of PINN and thus obtaining accurate solution close to steep gradients. In [13], the
authors showed that the proper clustering of training data points near high gradient region can improve the solution
accuracy in that region and thus prevent the propagation of error to the whole domain. This improvement suggests the
idea of decomposing the computational domain into several sub-domains and employing a separate localized powerful
35 network in the region of high gradients solution, and thus developing a set of individual local PINNs with different
characteristics complied to the known prior knowledge of solution in each sub-domain. The hp-refinement via domain
decomposition and projection onto space of high-order polynomials has been recently formulated by Kharazmi et al.
[34, 35], where the authors develop a general framework, namely hp-variational physics-informed neural networks.
Such idea is also used by Li et al. in [14] where they employed a local neural network on discrete sub-domains
40 and solved PDE using the variational principle. In [10], the authors used PINN to solve conservation laws in graph
topologies where domains are connected by interface boundary conditions, and they employed a single network to
solve the complete system of conservation laws. In this work, we present a conservative physics-informed neural
network (cPINN) on discrete sub-domains where we use a separate PINN in each sub-domains, and then stitch back
all sub-domains through the corresponding conservative quantity like flux, which makes this approach unique. Given
45 the sub-domains of a finite, manageable size, a massively parallel computations can be performed using clusters
of CPUs and GPUs. This is the best strategy to tackle the problem of curse of dimensionality, which is related
to the numerical approximation of a high-dimensional problems, both in terms of computational cost and memory
requirements. Thus, the proposed cPINN method can efficiently handle high-dimensional problems. The advantages
of the cPINN are multi-fold:

50 • Domain decomposition : PINNs do not require the discretization of PDE, yet it can benefit from decomposing
the computational domain. In this setting, a tailored network is employed to locally obtain the solution in each
sub-domain while preserving global requirements via proper interface conditions. This will address the second
limitation of PINN as it is a promising approach for parallelization of PINN codes.
• Reduction of error propagation in the domain : Single network used in PINN can pollute the solution in
55 the domain, especially during early training period due to only training data points available from initial or
boundary conditions (in the absense of any experimental data). But in cPINN, different networks in each sub-
domain provide additional information about the fluxes at the interfaces along with initial or boundary training
data points. This results in reduction of error propagation in the neighbouring sub-domains as well as faster
convergence.

60 • Flux continuity at the interfaces : The underlying conservation law of the physical phenomena controls the
2
 required continuity conditions of local solution of individual PINNs at the interface of sub-domains. Additional
term(s) are added to the loss function of cPINN to account for such conditions.
• Efficient hyper-parameter adjustment : Based on prior (and sparse) knowledge of the solution regularity in
each sub-domain, the hyper-parameter set of corresponding PINN is properly adjusted, i.e., the choice of depth,
65 width, activation function, network structure, number and distribution (clustering) of training/penalizing points
etc.
• Parallelization capacity : The partial independence of individual PINNs in decomposed domains can be fur-
ther employed to implement cPINN in a parallelized algorithm, in which each sub-domain is assigned to a
computational node that interacts with other computational nodes through interface conditions.

70 • Representation capacity : Due to deployment of individual network in each sub-domain by the proposed
cPINN method, the representation capacity of the network increases. This is important for accurately predicting
the complex solution of given PDEs.
The rest of paper is organized as follows: In section 2 we discussed about the problem setup. Section 3 gives the
methodology of the proposed method including a brief discussion on sub-domain loss function, interface conditions
75 and the optimization methods. Numerical experiments are performed in section 4, where various comparisons are
made between PINN and cPINN methods. One- and two-dimensional test cases like Burgers equation, KdV equation,
coupled two-dimensional Burgers equations, incompressible Navier-Stokes equations and compressible Euler equa-
tions are solved to verify our claim. Inverse problems are also solved using the proposed methodology. All the test
cases are chosen according to the degree of complexity, for example, compressible Euler equations admit discontinu-
80 ous solution, which are difficult to capture accurately for any numerical method. Finally, in section 5 we summarize
our findings.

2. Problem Setup

The parameterized nonlinear conservation law is given by

ut + f [u; λ] = 0 , x ∈ Ω ⊂ Rd , t > 0, (1)

with initial u(x, 0) = u0 and appropriate boundary conditions. u(t, x) denotes the solution of conservation laws, f [; λ]
is a nonlinear flux parametrized by λ. As an example, the one-dimensional viscous Burgers equation flux is given as
85 f = u2 /2 − λu x , where the parameter λ plays the role of viscosity coefficient.
In this paper, we shall solve both forward problems where solutions of partial differential equations are inferred
given fixed model parameters λ as well as inverse problems, where the unknown parameter λ is learned from the
observed data. The given mathematical model is converted into a surrogate model, more specifically, a given problem
of solving PDE is converted into a minimization problem where global minima of loss function correspond to the
90 solution of the PDE. The loss function can be defined using training data points like initial and boundary conditions
and the residual of the given PDE. For equation (1), the residual F is given by F B ut + f [u; λ].

3. Methodology

Let N L : RDi → RDo be a feed-forward neural network of L layers and Nk neurons in kth layer (N0 = Di , and
NL = Do ). The weight matrix and bias vector in the kth layer (1 ≤ k ≤ L) are denoted by Wk ∈ RNk ×Nk−1 and
bk ∈ RNk respectively. The input vector is denoted by z ∈ RDi and the output vector at kth layer is denoted by
N k (z) and N 0 (z) = z. We denote the activation function by Φ which is applied layer-wise along with the scalable
parameters nak , where n is the scaling factor. Layer-wise introduction of the additional parameters ak changes the
slope of activation function in each hidden-layer thereby increasing the training speed. Moreover, these activation

3
 4.0 Sub-domain 1 4.0 Sub-domain 1
Sub-domain 2 Sub-domain 2
3.5 Sub-domain 3 3.5 Sub-domain 3
Sub-domain 4 Sub-domain 4
Sub-domain 5 Sub-domain 5
3.0 Sub-domain 6 3.0 Sub-domain 6
Sub-domain 7 Sub-domain 7
2.5 Sub-domain 8 2.5 Sub-domain 8
Sub-domain 9 Sub-domain 9
2.0 Sub-domain 10 Sub-domain 10
Sub-domain 11 2.0 Sub-domain 11
Sub-domain 12 Sub-domain 12
1.5 Exact 1.5 Exact

1.0 1.0
0.5 0.5
0.0 0.0
0 5000 10000 15000 20000 25000 30000 35000 40000 0 5000 10000 15000 20000 25000 30000 35000 40000
# iterations # iterations

Figure 27: Variation of ν in all 12 sub-domains versus number of iterations. Left figure is for clean data and right figure is for the data with 1%
noise.

The number of hidden layers in 12 sub-domains, starting from 1 to 12 are [6, 3, 3, 6, 3, 3, 6, 3, 3, 6, 3, 3] with 20
neurons in each layer, and the corresponding initial values of ν in each sub-domain are [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12]/3. Figure 27 shows the variation of ν in all 12 sub-domains versus number of iterations. Left figure is for clean
335 data and right figure is for the data with 1% noise. The corresponding relative L2 error in the whole domain for both
cases is given in the table 12. The network correctly identifies the viscosity ν value in both the cases. Also, it can be
observed that the presence of noise in the training data reduces the convergence of ν towards its exact value. Clean
data takes around 9000 iterations whereas the data with 1% noise takes 13500 iterations for convergence.

5. Summary

340 To summarize, a conservative physics-informed neural network (cPINN) is proposed on discrete domains for
nonlinear conservation laws. This approach is an extension of the PINN, but has the flexibility of domain decom-
position into several non-overlapping sub-domains. The conserved quantities like fluxes, are preserved by enforcing
their continuity at the common interfaces of neighboring sub-domains. Various forms of errors involved in cPINN
such as optimization, generalization and approximation errors and their sources are discussed. We examined our pro-
345 posed method for solving both forward and inverse problems of high-dimensional nonlinear high order differential
equations, including one- and two-dimensional Burgers equation, the KdV equations, the coupled Burgers equation,
incompressible Navier- Stokes equations, and compressible Euler equations. In these cases, we showed that in addi-
tion to conservation of fluxes at interfaces, enforcing the average solution drastically increases the convergence rate;
this additional constraint is important as it reduces the training cost while maintaining the accuracy of approximation.
350 The factor in our proposed method is the flexibility of domain decomposition. In the case of the two-dimensional
Burgers equation, we showed that by having a rough a priori knowledge of the location of shock, one can properly
decompose the domain (even in an unstructured manner) to more accurately capture the steep descents in solution.
Moreover, having separate networks in each sub-domain offers the flexibility of adopting a properly chosen network
in the corresponding sub-domain, e.g. a deep network in a sub-domain with complex solution. Each sub-domain
355 is an individual PINN that can have its own set of parameters, including width and depth of the network, activation
function, optimization method, other training parameters, and penalizing points. This will further open up the possi-
bility of formulating a meta-learning algorithm, which optimizes the overall structure and parameters of networks in
cPINNs. Another major advantage of cPINNs in solving complex problems is its parallelization capacity, which can
efficiently decrease the training cost. The implementation of parallel computation, however, was not carried in this
360 paper, and we intend to systematically parallelize our method in future work.

28