PHYSICS-INFORMED NEURAL NETWORKS AND FUNCTIONAL INTERPOLATION

FOR INITIAL VALUE PROBLEMS WITH APPLICATIONS TO

INTEGRO-DIFFERENTIAL AND STIFF DIFFERENTIAL EQUATIONS

by

Mario De Florio

Copyright © Mario De Florio 2022

A Dissertation Submitted to the Faculty of the

DEPARTMENT OF SYSTEMS AND INDUSTRIAL ENGINEERING

In Partial Fulfillment of the Requirements

For the Degree of

DOCTOR OF PHILOSOPHY

In the Graduate College

THE UNIVERSITY OF ARIZONA

2022
2
Contents

List of Figures 5

Abstract 6

1 Introduction 7
1.1 Scope of Research and Contributions . . . . . . . . . . . . . . . . . . . . . . 8

2 Literature review 10
2.1 Physics-Informed Machine Learning . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Data-Driven Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Personal Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3 PINNs and X-TFC 19

3.1 Physics-Informed Neural Networks . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Extreme Theory of Functional Connections . . . . . . . . . . . . . . . . . . . 21

4 X-TFC for Integro-Differential Equations 22

4.1 TFC for Integro-Differential Equations (De Florio et al., 2021) . . . . . . . . 22
4.2 Transport Theory Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2.1 Radiative Transfer (De Florio et al., 2021) . . . . . . . . . . . . . . . 24
4.2.2 Rarefied-Gas Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.2.1 Thermal creep flow (De Florio et al., 2021) . . . . . . . . . . 25
4.2.2.2 Poiseuille flow (De Florio et al., 2022) . . . . . . . . . . . . 27
4.2.2.3 Couette flow (De Florio et al., under review) . . . . . . . . . 28

5 X-TFC for Stiff Differential Equations 31

5.1 Stiff Chemical Kinetics (De Florio et al., 2022) . . . . . . . . . . . . . . . . . 31

6 Conclusions and Future Works 33

Bibliography 44

A De Florio et al. (2021) TFC for Integro-Differential Equations 45

3
B De Florio et al. (2021) Radiative Transfer 63

C De Florio et al. (2021) Thermal Creep Flow 80

D De Florio et al. (2022) Poiseuille Flow 95

E De Florio et al. (under review) Couette Flow 114

F De Florio et al. (2022) Stiff Chemical Kinetics 125

4
List of Figures

2.1 Schema of the three possible scenarios based on the availability of observed
data and knowledge of physics. . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 3D temperature (left) and pressure (right) fields derived and reconstructed
based on the 2D images from the six cameras. [53] . . . . . . . . . . . . . . . 15
2.3 Schema of the NNs loss functions for the different approaches. . . . . . . . . 16

3.1 Schema of the regular Physics-Informed Neural Networks framework for solv-
ing a Partial Differential Equation subject to initial and boundary conditions. 20
3.2 Schema of the Extreme Theory of Functional Connections framework for solv-
ing a Partial Differential Equation subject to initial and boundary conditions. 21

5
Abstract

This dissertation aims to show the development and adaptation of the Extreme Theory of
Functional Connections (X-TFC) framework, a Physics-Informed Neural Network (PINN)
combined with Theory of Functional Connections (TFC), to solve Initial Value Problems
(IVPs) modeled by differential equations (DEs). In particular, this research focuses on two
branches of IVPs: Linear Integro-Differential Equations and Stiff Systems of Non-Linear
Differential Equations. The first types of IVPs are DEs where an integral of the unknown
function is present, i.e., Integro-Differential Equations. To compute the integral of the
unknown function two approaches are used. The first one is the analytical integration of the
unknown function’s approximation, which is represented by the TFC’s constrained expression
(CE). The second one approximates the definite integral of the unknown function via Gauss–
Legendre quadrature. This approach is used to solve problems arising from the Boltzmann
Partial Integro-Differential Equation for Transport, such as Radiative Transfer and Rarefied-
Gas Dynamics problems. The second types of IVPs are large-scale Stiff Systems of non-
linear ODEs, mathematical models governing a wide range of real-world problems such as
chemistry, biology, epidemiology, engineering, neuroscience, financial systems, and so on. For
this reason, recently, there has been a resurgence in the interest in developing new numerical
methods capable of solving large time-scale stiff problems. Regular PINNs frameworks are
proven to be not accurate (or even not capable) to solve problems when the dynamics are
particularly complex (e.g., stiff). This is due to unbalanced back-propagated gradients during
the model training. This dissertation shows how X-TFC with a domain decomposition
technique overcomes the difficulties of regular PINNs in solving systems of stiff IVPs, and it
outperforms the state-of-the-art numerical methods in terms of accuracy and computational
times.

6
Chapter 1

Introduction

Differential Equations (DEs) represent one of the most effective tools for building mathe-
matical and physical models. They are used in the most varied fields of applied sciences,
such as physics, chemistry, biology, and epidemiology, to name a few. The study of DEs
began with the introduction of the infinitesimal calculus by Newton and Leibniz in the
17th century and subsequently deepened by Bernoulli, d’Alembert, Euler, and Lagrange [1].
Among the more classical methods for solving DEs, we can refer to the Euler, Backward
Euler, Crank-Nicolson, and Runge-Kutta methods, still widely used today. Over the years,
many different algorithms have been developed, and many more are emerging. Recently,
Neural Networks (NNs) have become widespread across many different scientific fields, and
their applications for the solutions of DEs are no less. A new class of methods has been
developed in the machine learning community, and it is represented by the Physics-Informed
Neural Networks (PINNs) [2]. The methods underlying this category can be generalized
as algorithms that include the physics of the problem into a data-driven representation of
functional relationships underlying collections of input-output pairs. The DEs are added in
the training of the NN as a penalty for the loss function. The flaw of the PINNs framework
is that the solutions on the constraints are not analytically satisfied but numerically approx-
imated. This means there are competing objectives during the PINN training: learning the
DE solution on the domain points and the constraints. This leads to unbalanced gradients
when the NN is trained via gradient-based techniques, and this affects the ability of PINNs
to approximate the DE solution accurately [3]. The risk is getting stuck in limit cycles or
diverging if multiple competing objectives are present. To overcome this issue, the Theory
of Functional Connections (TFC) [4] can be adopted. The idea of TFC is to approximate
the DE solutions via the so-called Constrained Expressions (CEs), which are made up of
the sum of two terms: a free-function and a functional. Regardless of the choice of the free
function, the constraints will always be analytically satisfied since they are embedded in the
functional (chosen accordingly). The free-chosen function, thus, can be a single-layer NN
trained via the Extreme Learning Machine (ELM) [5]. The Extreme Theory of Functional
Connections (X-TFC) [6] method was born from this synergy between PINN and TFC and
would enhance one’s ability to solve challenging differential equations governing real-world
physics phenomena, which is the main driving motivation behind this dissertation.

7
1.1 Scope of Research and Contributions
The scope of this dissertation is the development of the X-TFC framework to solve two major
classes of initial value problems (IVPs) modeled by DEs, i.e., integro-differential equations
and stiff systems of non-linear differential equations, which are among the most challenging
to be solved with traditional numerical methods. The result of this research is the produc-
tion of six articles published (and under review) in scientific journals, in which I am the lead
author, presenting the methodology and real-world applications of X-TFC for the problems
under consideration. My contributions to these manuscripts are the conceptualization, cod-
ing, validation, and drafting, with the invaluable help of the co-authors. The main novelty
of my research journey is merging an interpolation technique (TFC) with an ELM training
of the NN, which opens a new research pathway in applied mathematics, to push the family
of PINNs methods to obtain ever more accurate solutions of DEs with extremely low com-
putational times. It has been proved that such interpolation technique is able to decrease
the generalization error by analytically satisfying the problems’ constraints, and an ELM
algorithm allows us to train a single-layer neural network in a fast and concise way, making
PINNs methods attractive for real-time and online applications.

The first paper presented in Appendix A and published on Mathematical and Computational
Applications - MDPI [7] is the result of a collaboration between my research team – led by
Roberto Furfaro – and Daniele Mortari, who conceived the original idea of the TFC. This
work aims to extend the TFC concept of CE for solving linear ordinary DEs (ODEs) sub-
ject to integral constraints and linear ordinary integro-differential equations. Although this
extension of TFC proved to work well for the considered problems by overcoming the com-
parative numerical methods, its implementation for more arduous integral kernels would be
too laborious or even impossible. To overcome this hurdle, a Gauss-Legendre quadrature ap-
proach can be used to approximate the integral of the unknown function and subsequently
solve it via X-TFC. Two classes of Transport Theory problems were chosen to test this
methodology due to my background in nuclear engineering and my advisor Dr. Furfaro’s
doctoral research on Radiative Transport. This choice led to the production of an article [8]
on the solution of the Chandrasekhar’s basic problem in radiative transfer in Appendix B,
and a trilogy of articles on classical problems in rarefied-gas dynamics in Appendix C, D,
and E: Thermal creep flow [9], Poiseuille flow [10], and Couette flow [11]. The X-TFC results
compared with the most accurate benchmarks in the literature showed the accuracy of our
methodology in solving physical phenomena modeled by the Boltzmann Integro-Differential
equations for transport of radiation and particles. These works were made possible thanks
to the exchange of ideas with my advisor Roberto Furfaro, the implementation of the codes
in MATLAB with my colleague Enrico Schiassi, and the valuable advice of two recognized
experts in the field of transport theory, Barry D. Ganapol and Liliane B. Barichello.

The motivation for producing the sixth paper, presented in Appendix F and published on
Chaos: An Interdisciplinary Journal of Nonlinear Science - AIP Publishing LLC [12], lies in

8
the lack of a PINN framework capable of solving problems subject to great stiffness, and in
the difficulty of traditional numerical methods in solving the most complex stiff problems.
The main idea of this work is to employ a domain decomposition technique in the X-TFC
framework to control and minimize the carryover of the generalization error along with the
time domain. This new methodology has been tested to solve stiff systems of non-linear dif-
ferential equations governing chemical kinetics reactions. The choice of this class of problems
lies in the wide range of real applications, such as climate modeling and energy conversions.
X-TFC proved to be efficient, accurate, and robust in solving all the faced problems, learning
the solutions for huge time domains, where all other competitive methods fail.

This dissertation aims to tell the story of my research path, starting with a brief history of
the development of humankind’s scientific knowledge, up to a literature review of Physics-
Informed Machine Learning state-of-the-art approaches and Data-Driven Machine Learning
techniques for the discovery of unknown laws of physics, in Chapter 2. Then, Chapter 3
introduces the regular PINN framework, its drawbacks, and how the X-TFC method aims
to avoid or mitigate regular PINN’s limitations. Chapters 4 and 5 represent the body of my
research, where all the journal articles are described, with introductions on the IVPs modeled
by integro-differential and stiff differential equations, respectively. Finally, conclusions and
future ideas for continuing my research in the direction I started are given in Chapter 6.

9
Chapter 2

Literature review

The history of the discovery of physics has very ancient roots, dating back to the dawn of hu-
man civilization. In the prehistoric era, knowledge of nature and its secrets was handed down
mainly through oral tradition and was presented closely related to religion. This knowledge
was essentially practical, therefore based on technological and mathematical skills, which
will lead to the development of advanced civilizations such as the Indian, Mesopotamian and
Egyptian ones, where mathematics would have originated, according to Herodotus [13]. The
birth of writing allowed the conservation of knowledge and its transmission with greater accu-
racy. Many ancient civilizations thus collected systematic astronomical information in detail
through observation of the sky and advanced calculation techniques [14]. The time could be
measured using a stick planted in the ground, which allowed to observe the variations of its
shadow produced by the different paths of the Sun during days and years. Through simple
triangulations, it was possible to observe the variations in the height of stars or groups of
stars and the irregular motion of the planets. Such simple but effective tools led to creating
the first catalogs of stars and the first tables of planetary motions to make astronomical
predictions in the third century BC. In those years in ancient Greece, the mathematician,
geographer, and astronomer Eratosthenes formulated the method for calculating the size of
the Earth [15].

Over time, the thirst for knowledge of the physics of nature was not quenched. Indeed, there
was a widespread need to develop more sophisticated and technological tools for observing
the nature of things among scientists. There was, therefore, a growing need to acquire more
and higher quality data. At the beginning of the 17th century, the telescope was born,
subsequently perfected by Galileo Galilei [16]. Thanks to this technology, Galileo improved
the quality of data acquisition and discovered new celestial bodies and features of the so-
lar system, previously unknown. He initiated a new way of describing and explaining the
mechanisms that exist at the basis of certain events. Thus was born the “scientific method",
which has as its first step the implementation of numerous experiments to obtain many ob-
served data [17]. Riding the wave of new technological inventions, Kepler formulated his laws
relating to the motion of the planets starting from the data collected by Tycho Brahe and
relating to their apparent positions, in particular of Mars [18]. Kepler accurately determined

10
the orbit with which the Earth rotated around the Sun, which was almost circular with the
Sun placed in an eccentric position. He also found that the Earth moved faster when closer
to the Sun. From observations of Mars, he showed that the orbit had an elliptical shape.
The scientist, therefore, developed a data-driven model for planetary motion, formulating his
three laws – the famous “Kepler’s laws of planetary motion” [19] – empirically, that is, based
on experimental observation. However, Kepler did not explore the fundamental dynamics
involved in planetary motion. For this, we will have to wait for the derivation of Newton’s
law of motion, in which a dynamic relationship between momentum and energy describing
the underlying processes responsible for these elliptic orbits is presented [20, 21]. Newton
produced a universal model that can predict the behavior of a system in a no-data-available
scenario, published in the collection “Philosophiæ Naturalis Principia Mathematica", 1687
[22]. A decade earlier, in “De Methodis Serierum et Fluxionum" [23] he published a pow-
erful tool to study many problems in the natural sciences and technology, still extensively
employed in mechanics, astronomy, physics, and in many problems of chemistry and biology,
now called Differential Equation (DE). DEs still belong today among the main Physics-based
models that govern a physical phenomenon through mathematical language, and they can
be used to accurately predict the state of a system by imposing determined constraints and
boundary domains [24].

The study of DEs has been widely extended in the following centuries, and classical methods
for their solutions have been developed and extensively applied. Among these methods,
we can refer to the Euler, Backward Euler, Crank-Nicolson, and Runge-Kutta methods
[25, 26, 27]. In the years, many different algorithms have been developed, and many more are
emerging [28]. To date, humankind has reached the highest level of knowledge of physics and
development of technology ever seen so far 1 . However, there are still situations in which the
physics is unknown and it is difficult to retrieve it from data, or the phenomenon is difficult
to observe, leading to scarce low-quality data availability. In both cases, Machine Learning
(ML) algorithms can come to the aid of scientists to quickly and efficiently solve physics
problems still unsolved and shrouded in mystery. The need for ML algorithms began in the
last few decades when there was significant progress in understanding multi-scale physics
for different science applications. ML approaches have been proven to help discover the
physical laws and mathematical expressions underlying experimental data [29, 30]; however,
purely data-driven models that fit observed data can bring inconsistency in the physics of the
phenomena. To overcome this problem, ML algorithms can be informed by the knowledge of
the physics as “informative priors”. Depending on the mathematical and physical knowledge
and on the data available, three possible categories of the problem under consideration can
be defined, as represented in Figure 2.1.
1
NB, this does not at all mean that it (humankind) is able to use it properly and responsibly.

11
Figure 2.1: Schema of the three possible scenarios based on the availability of observed data and
knowledge of physics.

The first category represents the case when no data are observed while the physics (differ-
ential equations) governing the problem is well known, therefore, the only known variables
are the boundary and initial conditions. This represents a forward problem that requires a
numerical method (physics-driven approach) to find the solution of DE. The second category
is the most common case in engineering applications, and it occurs when only a few data are
observed and only part of the physics governing the phenomena is known. For example, we
know the differential equations governing a chemical reaction but not the exact values of the
reaction rate constants. In this case, physics-informed machine learning methods integrate
available data and the partially known mathematical physics model to learn both the DE’s
solution and the parameters (data-physics-driven approach). The third category regards the
case where lots of observed data are available, but the physics model that can describe the
phenomena is completely unknown (data-driven approach), and the whole physics, or some
missing parts, needs to be learned [31, 32].

12
2.1 Physics-Informed Machine Learning
Creating a Physics-Informed Machine Learning (PIML) algorithm means introducing ap-
propriate biases helping the learning process in identifying physically consistent solutions.
Predictive models can not be built without assumptions; thus, creating a generalized ML
model without appropriate biases is impossible [33]. To improve and accelerate the physics-
informed learning process by introducing physics in the ML models, three pathways can
be followed, individually or effectively combined, to yield a class of hybrid approaches. An
observational bias is conceptually the simplest bias that can be introduced in ML, and it
is done through the embedding of the observed data that reflect the physics that should
generate them. As long as the available data cover the domain of the training points, ML
algorithms have been proved to be accurate in the interpolation process between the scat-
tered data [34, 35], learning functions, operators, and vector fields describing the structure of
the data. An inductive bias consists of embedding some predictions in the form of physical
laws into the architecture of the ML model to satisfy their mathematical constraints. The
most common ML architectures that fall into this category are the Convolutional Neural
Networks (NNs) [36] for computer vision and Graph NNs [37] for processing data best rep-
resented by graph data structures. These frameworks proved to work efficiently when the
physics of the problem is simple and well defined. However, their extension to more complex
and less understood physical problems can bring some difficulties in the learning task, and
their embedding into the NN architecture can be tough to code. When the goal is solving
DEs, the algorithm architecture can be modified to exactly solve the constraints. X-TFC
[6] is one of the methods [38, 39, 40, 41] lying in this category. The third school of thought
is introducing learning biases into the NNs without designing a specific architecture that
enforces the physics law. The idea is to penalize the loss functions by a set of physical con-
straints (e.g., the conservation mass) while fitting the observed data. Deep Galerkin method
[42] and Physics-Informed NNs (PINNs) [2] are good examples in this respect.

PIML models have been already extended and applied to different physical and mathemati-
cal fields, such as fractional PDEs [43], stochastic PDEs [44], and conservation laws [45], and
for multi-scale problems a domain decomposition is performed. In this way, a different NN
represents each subdomain and is assigned to a different GPU [46, 47]. As explained before,
PIML can combine data with the knowledge of physics, even when both are poor. For exam-
ple, in forward and inverse problems when the constraints are not defined, and parameters
of the PDEs are unknown (where many numerical methods fail), works like Ref. [48] show
the capabilities of PIML frameworks in finding robust solutions even with low resolution and
substantial noise in the observation data. When tackling problems in a small data regime,
PIML finds its strong advantage in the generalization of the solutions by embedding all that
is known about physics.
For some real-world problems involving the evolution forecast of multi-physics systems, a
quantification of uncertainties is required. In particular, three different sources of uncertain-
ties can be defined: the physics, the data, and the learning models.

13
 I. When the source of uncertainty is the physics, we are dealing with stochastic DEs
because of the randomness of the parameters. To quantify this uncertainty, Ref. [49]
proposes a methodology for learning deep NNs surrogate models based on a parameter-
ization of the Deep NN structure, and it is demonstrated with a problem on uncertainty
propagation in a stochastic elliptic partial differential equation with uncertain diffusion
coefficient.

II. The uncertainty can arise from the noise and the gaps in the observed data. To
compensate these deficiencies during data acquisition, a valid method that can be the
Bayesian approach. In this regard, a PIML framework for encoding physical laws
described by PDEs into Gaussian process priors for nonparametric Bayesian regression
was developed [50]. This algorithm aims to infer solutions to time-dependent and non-
linear PDEs and it effectively quantifies and propagates uncertainty due to noisy initial
or boundary data. Another framework named Bayesian PINNs (B-PINNs) [51] was
proposed to quantify uncertainty and to improve the accuracy of PINNs. This method
uses a Bayesian NN, subject to the PDE constraint as a prior, and Hamiltonian Monte
Carlo (or the variational inference) as an estimator for the posterior.

III. Finally, uncertainties coming from the learning models, such as training, generaliza-
tion, and approximation errors, are generally difficult to quantify. The first attempt
to quantify the parametric uncertainty in the PINNs was made by by employing the
arbitrary polynomial chaos (aPC) expansion for the representation of the stochastic
solution [52]. To solve forward and stochastic inverse problems, the authors use ob-
served data to build a set of arbitrary polynomial basis and learn the modal functions
of the aPC expansion through deep NNs encoding the underlying stochastic DE.

An interesting application of physics-informed learning for ill-posed inverse hidden fluid

mechanics problems applied to the heat flow over an espresso cup [53]. This problem is
considered ill-posed since no constraints are provided, and inverse, since information about
the physics is sought. As previously mentioned, physics-informed NN can be trained by using
both some observed data and some knowledge of the physics underlying the phenomena. For
this problem, the observed data is given by the temperature (or density) field acquired using
six cameras and processed with LaVision’s Tomographic BOS software, as shown in Figure
2.2.

14
Figure 2.2: 3D temperature (left) and pressure (right) fields derived and reconstructed based on
the 2D images from the six cameras. [53]

On the other side, the information of the physics is given into the PINN by the Boussinesq
approximation of the incompressible Navier–Stokes equations and the corresponding heat
transfer equation. Thus, data and equations are part of the loss that needs to be minimized
in the training. The three space coordinates and the time are used as training inputs of the
NN, giving the velocity and pressure fields as outputs. The results of this approach shows
the high accuracy and flexibility in dealing with observed data of various fluid mechanics
problems [53]. Thus, PINNs can become a promising direction in experimental fluid mechan-
ics. Although the PINNs term appeared for the first time in the paper from Raissi et al. [2]
in 2019, the first attempt made to use NN to tackle DEs dates back to 1992 by Psichogios
and Ungar [54], and subsequently by Lagaris et al. [38]. The schema in Figure 2.3 shows
the loss functions of the NNs for the physics-driven approach, data-driven approach, and
data-physics-driven approach (for both forward and inverse problems).

15
 Figure 2.3: Schema of the NNs loss functions for the different approaches.

2.2 Data-Driven Machine Learning

The Scientific Method represents a Human Learning algorithm in which a scientist observes,
questions, hypothesizes, predicts, tests, and eventually iterates this process. Machine Learn-
ing is a branch of Artificial Intelligence and Computer Science that focuses on using algo-
rithms powered by observed data that mimic human behaviors in the learning phase and
improve with experience. The analogy of these two learning algorithms lies in their cognitive
systems to build knowledge by interacting with the surrounding environment and learning
from it. In the history of science, it may have taken decades for scientists to formulate new
physical laws through countless observations, experiments, and failures; or even centuries for
subsequent groups of scientists to improve and redefine past studies that led to a final formu-
lation of a generalized law. The human mind has limited computational power and memory,
but it has the shrewdness and the intelligence to create machines that can analyze data and
work out hypotheses for it. Today, ML algorithms are the protagonists of many technologi-
cal advancements. They have been used to automatize the analysis of observed data, which
saved time and effort for scientists in various fields of science, such as the identification of
exoplanets [55], exploring galaxies [56], or to stay grounded, to help in creating the new
industry of self-driven vehicles [57]. Supervised ML is widely used today in medicine as a
classification algorithm to help medical doctors in disease diagnosis, for example, by predict-
ing whether a tumor is benign or malignant [58]. In short, it is clear that ML has immensely
helped the human being in his path toward a further technological and social evolution.
Therefore, a question arises spontaneously in this circumstance. Can ML help scientists bet-
ter understand the physics around us? Can ML help humanity to discover new physical laws
of phenomena that we can observe but fail to conceptualize through mathematical language?

16
This is a recent, pervasive question in the scientific community yet not resolved, but sev-
eral possible avenues have been opened. One of the first attempts to extrapolate governing
equations from observed data is presented in the well-known work by Brunton et al. [31], in
which the authors propose a new school of thought for dynamical system discovery problem
from the perspective of sparse regression [59] and compressed sensing [60]. In particular,
they take advantage of the fact that most physical systems are described by only a few
relevant terms governing the dynamics, making the governing equations sparse in a high-
dimensional non-linear function space. This method named SINDy – Sparse Identification
of Nonlinear Dynamics – depends on the choice of the candidate non-linear functions library
and the availability and quality of the data. Thus, it is not a generalized method and works
better if guided by available knowledge of the phenomena under study. For example, given
the trend of the observed data, one can approximately understand if it is a trigonometric
or polynomial trend and build the library accordingly. SINDy has shown its capability in
identifying non-linear dynamical systems from data without previous assumptions of the
forms of the differential equations governing the phenomena. A particular example that
proves its power is the discovery of the PDE for vortex shedding behind an obstacle. The
greatness of this result lies in the fact that this PDE formulation required about 30 years
and a team of fluid-dynamics experts. Of course, SINDy is not the only attempt to retrieve
governing equations from data. Another method has been proposed by Raissi et al. [61],
which aims to learn general parametric linear equations from data containing noise by using
multi-output Gaussian process regression [62]. In its form, the method is designed to be
generalized for linear equations, but it cannot deal with non-linear ones. However, this task
may be addressed by transforming the non-linear equation in a system of linear equations
and then finding some specific non-linear operators.

Another new and interesting work published by Purdue University researchers proposes a
new ML method to identify physical laws from data [63]. The basics of this method lie on
the concept of parsimony, for which ML neural networks are often too large and complex to
efficiently learn a simple explanation of an easily observed phenomenon. Based on this, the
authors aim to enforce parsimony on the NN through stochastic optimization to find a better
balance between simplicity and accuracy, thus, effectively retrieving interesting physics from
data. In particular, these so-called Parsimonious Neural Networks have been trained with
observed data to discover the Lindemann melting law and Newton’s second law of motion.
These physics laws, which required decades for Newton and Lindemann to be found, took
only a few days for Purdue’s supercomputers to produce models that demonstrated them.
All the approaches mentioned above using Neural Networks (excluding SINDy) require prior
knowledge of the DE’s form to discover physics laws, or they will fail even for the most
straightforward set of DEs. Although several techniques have been found to be accurate and
reliable for identifying already known physical models, the challenge of attempting to predict
and model unforeseeable phenomena remains. Questioning where artificial intelligence can
be applied to reveal the most unsolved physical doubts can be a good first step that a

17
scientist must face. Will ML help us to explain the flyby anomaly? Can ML help us to
generalize the unexplained empirical Koide formula? Will ML one day be able to find a
singular, all-encompassing, coherent theoretical framework of physics that fully explains and
links together all physical aspects of the universe? [64]

2.3 Personal Perspectives

Observed data and knowledge of physics are two central pillars of ML algorithms for mod-
eling and forecasting the dynamics of multi-physics and multi-scale systems. After a brief
historical overview of the development and contributions of these two factors in the science
evolution, three main categories of study problems have been identified. The first category
represents the case in which, for a given problem, the knowledge of physics is complete but
there is a total lack of data. For this case, the physics model is well known by a set of
governing DEs, and the data can be obtained by solving the latter with an explicit or im-
plicit numerical method (where a closed solution is not available). In the second category,
where we want to trace information from physics with the help of few experimental data.
This is the case where most of the engineering problems belong, and it is the typical case of
DE parameters discovery, where the inference of parameters governing a dynamical system
is sought. Physics-Informed Machine Learning algorithms aim to solve these problems by
leveraging the empirical data with the available knowledge of physics to penalize or regu-
larize the Neural Network in use. The third category represents the scenario in which there
is no physics consciousness to explain a particular behavior of a system or an observable
phenomenon whose acquired data are challenging to interpret.

The scientific community is pooling efforts to efficiently create and refine algorithms to
address all the aforementioned categories’ possible problems. To date, there is no general
and perfect algorithm, and perhaps never will be. However, there are several algorithms with
particular strengths developed to overcome specific obstacles in physics and mathematics.
The schools of thought carried out by different research groups are many and varied and
I believe that combinations of these in a targeted and rational sense is the way forward
for the trend toward perfecting machine learning in helping scientists in their modeling
and forecasting purposes. Particularly fascinating is the position of the third category in
the timeline of scientific evolution. It represents the birth of critical scientific thinking,
characterized by the total absence of knowledge of physics and mathematics and by the
observation with the naked eye (or through rudimentary tools) of everyday phenomena of
the surrounding world. Paradoxically, it also represents the contemporary epoch of scientific
evolution, characterized by a vast knowledge of physics and the technological ability to
acquire high-resolution data in large quantities. Thanks to the immutable nature of human
curiosity in both eras, the frontiers of science are infinite, and we push ourselves to research
and investigate what is unknown to us and the physics known so far. The difference is that
today we can do it with a telescope and a Python code, instead of with a gnomon and an
hourglass.

18
Chapter 3

PINNs and X-TFC

As mentioned above, the first category of Figure 2.1 represents the case of differential equa-
tions (DEs) subject to particular constraints. DEs represent one of the most effective tools
for describing physical phenomena through mathematical models, and they link one or sev-
eral unknown functions with their derivatives. Usually, the unknown function is a physical
quantity and its derivative is its rate of change. For example, let’s consider the following
general differential equation
d
y(t) = f (t, y(t)) subject to y(t0 ) = y0
dt
This is the case of initial value problem (IVP) where we study the evolution of the unknown
quantity starting from a specific initial condition. Not for all DEs there is an available ana-
lytical solution, thus an efficient and reliable numerical method is required.

In the last century, many numerical methods have been developed, adapted, and improved
to solve the most challenging DEs modeling real-world problems. Among these, we can men-
tion the most classic Euler’s method [65], Runge-Kutta methods [66], and Milne’s Predictor-
Corrector method [67]. For high-dimension Partial DEs (PDEs), the well known finite dif-
ference method [68], finite element method [69], and finite volume method [70] have proven
to be particularly effective, even for complex geometries. However, like all things, these
methods have some limitations. A feature in common is the need to adopt a domain dis-
cretization or a mesh domain decomposition. This implies the evaluation of the solution
only on the discretization points, or mesh-cells (or training points, according to the machine
learning terminology). To evaluate the solution on other points of the domain (test points)
one can resort to an interpolation or to a re-computation of the problem. An interpolation is
a numerical estimation of the solution in a point between two training points, in which the
solutions are already numerically approximated. Thus, this technique will inevitably lead to
a loss in accuracy. The re-computation of the problem by adding the new test points among
the set of training points will lead to an increase of computational efforts, that in some cases
can not be affordable. Another drawback is that once the numerical solution is computed,

19
we may need to find its derivative or integral, where another numerical approximation is
required. This, again, would bring a further loss of accuracy. Last but not least, all these
methods are designed to solve forward problems only. To tackle inverse problems, they need
to be modified or coupled with other methods, and the code implementation can be intense,
which is not a trivial task. Now the question naturally arises: how can we avoid, or at least
mitigate, these limitations? An answer can be: Neural Networks (NNs).

3.1 Physics-Informed Neural Networks

Figure 3.1 shows the PINN algorithm for solving a PDE subject to initial and boundary
constraint. The PDE’s independent variables (in this example x and t) are the inputs of
the deep NN, which approximate the solution (u) of the PDE. The loss function includes
the residual of the PDE and the initial and boundary constraints, and the goal of the NN is
to find the optimal set of NN parameters θ (which are the input weights, bias, and output
weights) that minimizes the loss function, by using gradient-based methods [71]. The main
flaw of this framework lies in the use of a deep NN for the approximation of the solution, due
to the complexity of such a network, which leads to a high computational cost, can cause
divergence, and treats Plinear problems as non-linear problem, because of the expression of
the activation function Li=1 βi σ(wiT x+bi ) in the ith neuron. Another negative aspect of this
framework is the inclusion in the loss function of both the PDE residual and the constraints,
since they represent an unsupervised learning and supervised learning, respectively. Thus,
we are facing the presence of competing objectives from which gradient pathologies can arise,
causing the training failure in learning the PDE constraints [3].

Figure 3.1: Schema of the regular Physics-Informed Neural Networks framework for solving a
Partial Differential Equation subject to initial and boundary conditions.

20
3.2 Extreme Theory of Functional Connections
The Extreme Theory of Functional Connections (X-TFC) is a method I have co-developed
with my research team [6], to speed up and optimize the training process of the NN in
learning the solution of DEs. The novelty lies on the introduction of two main features into
the regular PINN framework. To approximate the solution of the DE, we make use of the
constrained expression (CE) from the Theory of Functional Connections [4], which is made
by the sum of two terms. The first term is a a free-chosen function, while the second term
is a functional in which the DE constraints are embedded. No matter what free-chosen has
been selected by the user, the CE will always analitically satisfies the initial and boundary
conditions. This, will avoid the presence of competing objectives in the loss function, as
mentioned in the previous section. The second novel feature is the choice of the free-chosen
function, which is a single layer forward NN, trained via Extreme Learning Machine (ELM)
algorithm [5]. According to ELM, the NN input weights and bias are randomly selected
before the training process, thus they become NN hyperparameters. This extremely simplify
the structure of the network speeding up the training process, since the only NN unknown
parameters are now the β output weights, which can be found by minimizing the loss function
with least-squares methods.

Figure 3.2: Schema of the Extreme Theory of Functional Connections framework for solving a
Partial Differential Equation subject to initial and boundary conditions.

21
Chapter 4

X-TFC for Integro-Differential Equations

Integro-Differential Equations (IDEs) are differential equations in which the unknown func-
tion appears both in an integral and derivative form and often appear in many scientific
disciplines. Historically, the first studies on Integral Equations can be traced back to Abel’s
problem [72], Lotka’s population analysis [73], Fredholm Theory [74], and Volterra Integral
Equation [75]. These studies have been of fundamental importance for the development of
mathematical models for problems in mechanics, biology, and economics. Often, for real-
world problems, an analytical solution of IDEs is not available, and a numerical method is
required, such as Laplace transform method [76], Wavelet–Galerkin method [77], and Euler-
Chebyshev methods [78].
This chapter shows how X-TFC can be adapted and employed to solve IDEs. In particular,
the adaptation of the TFC methodology to solve IDEs and DEs subject to integral con-
straints is presented in Section 4.1, and the application of X-TFC to solve some Transport
Theory problems is shown in Section 4.2.

4.1 TFC for Integro-Differential Equations (De Florio et

al., 2021)
This section aims to introduce the paper [7] in Appendix A:
De Florio et al. “Theory of functional connections applied to linear odes subject
to integral constraints and linear ordinary integro-differential equations.” Math-
ematical and Computational Applications 26.3 (2021): 65.
This methodology paper shows how TFC can perform functional interpolation to solve
(i) linear ordinary differential equations subject to integral constraints, and

(ii) linear integro-differential equations.

For the first kind of problem (i), we can find three different sets of constraints: definite
integral constraints, integral and linear constraints, and mixed constraints. Both cases show

22
how to accommodate the integral constraints into the TFC’s CE. In particular, after building
the CE accordingly to TFC framework [4], we need to retrieve the coefficients ηi (where
i = 1, ..., N , and N is the number of constraints) by imposing the constraints in the CE,
thus by analytically integrating the latter to embed the integral constraint. Similarly, we
show how to solve the second kind of problem (iii), where the unknown function in the
differential equation appears under the integral sign. Since we aim to test both TFC and
X-TFC in solving some toy problems, to evaluate the integral of the CE for the two different
methodologies, we need to compute the integration of both types of CE’s free-chosen function,
such as Chebyshev Polynomials and NN’s activation function. This integration can be more
or less arduous to implement based on the kernel of the integral (i.e., the function that
multiplies the unknown function under the integral). For each problem solved, an analytical
solution was available to test each method’s accuracy, which has been compared with a state-
of-the-art method based on the differential transformation [79]. For the more straightforward
problems of type (i) (results of Ref. [79] not available), both TFC and X-TFC computed the
solution with error machine accuracy (absolute error with respect to the analytical solution of
the order of 10−15/−16 ) with a computational time of the order of 10−4 seconds. For the more
challenging integro-differential equations (ii), TFC and X-TFC outperformed the results in
Ref. [79] in terms of accuracy and computational efficiency. As stated in the Discussion
section of the paper, the significant difference in accuracy among problems solved is due to
the difference in challenging the kernel functions.
My contributions in this work are the conceptualization of the idea of extending the TFC’s
constrained expression for integro-differential equations, creating the MATLAB functions
that analytically integrate the activation functions of the neural network, coding the TFC
and X-TFC algorithms to solve test problems published in the literature, comparing our
results with other available numerical methods results, and writing the journal paper.

4.2 Transport Theory Problems

As explained in Section 4.1, the complexity of the integral kernel function can determine
the degree of the accuracy of the solution of the integro-differential equation and can even
result in a too arduous implementation of the analytical integration of the kernel times CE’s
free-chosen function (i.e., Chebyshev polynomials or NN’s activation function). In more
challenging real-world physical phenomena, such as physics problems in Transport Theory,
the complexity of the integral kernel would make the integration of the CE prohibitive. For
example, in the Radiative Transfer problem discussed in Section 4.2.1, the integral kernel
represents the phase function expressed in terms of Legendre polynomials, while for the
Rarefied-Gas Dynamics problems in Section 4.2.2, the integral kernel is a weight function
defined as an exponential function. To tackle these problems, the integrals can be approxi-
mated with Gauss-Legendre quadrature
Z 1 N
X
f (x)dx ≈ wi f (xi )
−1 i=1

23
as shown in the following sections.

4.2.1 Radiative Transfer (De Florio et al., 2021)

This section aims to introduce the paper [8] in Appendix B:
De Florio et al. “Solutions of Chandrasekhar’s basic problem in radiative transfer
via theory of functional connections.” Journal of Quantitative Spectroscopy and
Radiative Transfer 259 (2021): 107384.
This paper aims to show the capability of TFC in solving the Chandrasekhar’s basic problem
in radiative transfer arising from the Boltzmann Integro-Differential Equation. This equation
describes the propagation behavior of electromagnetic radiation (considered as particles go-
ing through several physics processes, such as absorption, scattering, and emission) through
a medium. In particular, the Chandrasekhar’s problem models the behavior of scattered
photons through a finite slab illuminated by incident light on the surface. The histori-
cal method for solving such an equation for photons transport was the Discrete Ordinate
Method (DOM), proposed by Wick [80], extended by Chandrasekhar [81], and upgraded and
improved over the years [82, 83, 84].

In this paper, after an overview of the state-of-the-art DOM methods and an introduction
to the TFC framework to solve general ordinary differential equations and a simple case
of radiative transfer problem in the two-stream approximation, we presented the physics
problem formulation of the radiative transfer equation, of fundamental importance to apply
TFC and compare our results with published benchmarks. In its original form, the Chan-
drasekhar’s problem is a boundary value problem since we have two conditions on the two
surfaces of the slab. The unknown function is the intensity of the photons as a function
of the slab’s optical depth, scattering angle, and azimuthal angle, and it is defined as the
sum of scattered term and unscattered term. The scattered term (or diffuse term) can be
expressed as a Fourier expansion with L components, whose mth component satisfies another
partial integro-differential equation (see Eq. (36) in Appendix B), which can be split into two
integro-differential equations with one constraint per each. To notice that we transformed a
boundary value problem into two initial value problems. Then, by approximating the inte-
grals with a Gauss-Legendre quadrature, we transform two partial differential equations into
a system of 2 × N ordinary differential equations, where N is the number of Gauss-Legendre
discretization points.

After reformulating the problem, the TFC formulation can begin. To notice that the inde-
pendent variable of our 2 × N differential equations is the slab optical depth τ that ranges
from 0 to a certain τ0 ., and that regular TFC plans to use orthogonal polynomials (e.g.,
Chebyshev) in the range [−1, 1] as free-chosen function, thus a linear mapping is required.
Once built the CEs accordingly, we replace them in our 2 × N system of differential equa-
tions, where all the terms are known but the polynomials coefficient ξ. Thus, we rearrange
the terms in the 2 × N system to get a linear system in the form Aξ = B, which can be

24
solved for ξ via a least-squares method. By doing this, we are able to compute the value of
the CEs, and then the solution of our problem. This needs to be done for all the L Fourier
components that then will be used to compute the diffuse term (see Eq. (60) in Appendix
B). Our solution is obtained for specific scattering angles, which are the Gauss-Legendre
Quadrature nodes. However, in many real-world applications, we must compute the solu-
tions in arbitrary scattering angles. For our case, we need to compute the solutions in the
same scattering angles of the literature published benchmarks to compare our results. Thus,
we used a post-processing (performed via TFC, see section 5 of appendix B).

To validate our results, we solved the radiative transfer equation with two different phase
functions, such as Mie Scattering (9 Fourier components) and Haze L problem (83 Fourier
components), for both one and two-angle cases. The algorithm described above has been
implemented in both Matlab and Python platforms, and the simulation ran with an Intel
Core i7 - 9700 CPU PC with 64 GB of RAM. All results in the paper demonstrate the
precision and the efficiency of TFC by achieving all the digits (8) of accuracy of the most
accurate benchmarks in literature.
My contributions in this work are the formulation of the problem to reach a convenient
form of the integro-differential equation to be solved by X-TFC, the X-TFC formulation to
transform the constrained problem into an unconstrained linear system to be solved via least-
square, coding the X-TFC algorithm to solve two challenging problems in the literature and
comparing our results with published and recognized benchmarks, and writing the journal
paper.

4.2.2 Rarefied-Gas Dynamics

The second chapter of my research on Transport Theory problems is represented by the
class of Rarefied-Gas Dynamics in the Bhatnagar–Gross–Krook (BGK) approximation. As
for the radiative transfer, the rarefied-gas flows studied in this dissertation arise from the
Boltzmann Integro-Differential Equation. The rarefied-gas dynamics describes a gas flow
where the molecules’ mean free path λ is greater than the size d of the chamber under test.
Usually, a gas can be considered rarefied if the its Knudsen number Kn = λd > 1. In the
next Sections 4.2.2.1, 4.2.2.2, and 4.2.2.3, three classical flows of a gas in rarefied state are
presented: Thermal creep flow, Poiseuille flow, and Couette flow, respectively.

4.2.2.1 Thermal creep flow (De Florio et al., 2021)

This section aims to introduce the paper [9] in Appendix C:

De Florio et al. “Physics-informed neural networks for rarefied-gas dynamics:

Thermal creep flow in the Bhatnagar–Gross–Krook approximation.” Physics of
Fluids 33.4 (2021): 047110.

In this first paper of the trilogy on Rarefied-Gas Dynamics, a thermal creep flow in a micro-
channel is studied. In the thermal creep flow, a rarefied gas is confined between two parallel

25
walls, and its flow is driven by a temperature gradient in a direction parallel to the walls
of the channel. It usually occurs between long parallel walls very close to each other (in
the order of µm or nm), flowing from the colder to the warmer molecules until an equi-
librium temperature is reached. In particular, the classical BGK model assumes that the
molecules’ collision frequency is constant and non-dependent on the velocity. The thermal
creep phenomenon was initially studied by Knudsen [85], Maxwell [86], and Reynolds [87],
and subsequently mathematically modeled via Boltzmann Integro-Differential Equation by
Cercignani [88]. Among the classical numerical methods for its solution we can mention
the DOM methods [80, 82, 83, 84], and more recently the Direct Simulation Monte Carlo
(DSMC) for both monoatomic and polyatomic gases [89].

This paper provided an overview of the classical methods for solving the thermal creep flow
and an introduction to the PINNs and X-TFC frameworks to tackle DEs. To formulate the
physics problem, we started from the definition of the full non-linear Boltzmann equation
for the gas atom space and velocity distribution. By applying the BGK approximation, we
assume a constant collision frequency equal to the molecular mean free path, independent of
the velocity. A Maxwell–Boltzmann distribution is assumed to define a diffusely and specu-
larly reflecting wall, which we can use to leverage the computational cost of the simulation
by computing our solution on half domain only. Another assumption we can make for the
thermal creep flow is that the fluid has zero velocity relative to the wall (no-slip condition).
We finally get a partial integro-differential equation (see Eq. (37) in Appendix C) depending
on the half-channel width τ ∈ [0, a] and the fluid velocity u ∈ [−∞, ∞]. Here, we need
to do a linear mapping of both the independent variable. Since we need to approximate
the integral in du with a Gauss-Legendre quadrature, we use the logarithmic transformation
u = − log(|µ|), with µ ∈ [−1, 1]. Also, τ will need to be mapped according to the domain of
the CE’s free-chosen function. Since the constraints of our problem are defined for positive
and negative values of µ (see Eq. (48) in Appendix C), it is convenient the split the equation
into two partial differential equations with one constraint per each. After approximating the
integral with Gauss-Legendre quadrature, we obtain a system of 2 × N ordinary differential
equations. To solve the thermal creep flow problem in this form, we can build the CEs
accordingly, expanding the free-chosen function with a single-layer NN trained via the ELM
algorithm. By replacing the CEs into our 2 × N system and reorganizing the terms, we can
obtain the linear system Aβ = B, solvable with a least-squares method. Once the NN’s
output weights β are computed, we can build the solution of the unknown function, which
represents the velocity momentum of the flow. We can compute other quantities of interest
from this physical quantity to compare our results with available benchmarks.

In the results section of the paper, we show how to compute the macroscopic velocity profile
and the mass flow rate for the thermal creep flow as a function of the half-width of the
channel a and the accommodation coefficient α, which describe the interaction between gas
and wall. Our results are then reported for a wide range of Knudsen numbers, achieving up
to 8 digits of accuracy for all the benchmark cases published in the literature. The problem

26
has been coded in the MATLAB R2019a platform, and all our simulations ran with an Intel
Core i7–9700 CPU PC with 64 GB of RAM.
My contributions in this work are the conceiving of the idea of solving the thermal creep
flow to test the ability of X-TFC to solve rarefied-gas dynamics problems, the formulation of
the problem to reach a convenient form of the integro-differential equation to be solved by
X-TFC, the X-TFC formulation to transform the constrained problem into an unconstrained
linear system to be solved via least-square, coding the X-TFC algorithm to solve the thermal
creep flow problem and obtain numerical solutions of some physical quantities of interest in
the literature, comparing our results with published and recognized benchmarks, and writing
the journal paper.

4.2.2.2 Poiseuille flow (De Florio et al., 2022)

This section aims to introduce the paper [10] in Appendix D:
De Florio et al. “Physics-Informed Neural Networks for rarefied-gas dynamics:
Poiseuille flow in the BGK approximation.” Zeitschrift für angewandte Mathe-
matik und Physik 73.3 (2022): 1-18.
In this second paper of the trilogy on Rarefied-Gas Dynamics, a Poiseuille flow in a micro-
channel is studied. In the Poiseuille flow, a rarefied gas is confined between two parallel walls,
and its flow is induced by a pressure gradient along the walls of the channel. The Boltzmann
integro-differential equation has been the most used mathematical model to study this clas-
sical rarefied-gas dynamics flow [88], and subsequently, several approximations have been
implemented (i.e., BGK [90], and Shakhov [91]). One of the first numerical methods used to
solve the Poiseuille flow problem was the DOM method of Chandrasekhar [81], which subse-
quently has been improved to solve many other problems in transport theory. Among these
upgrades we can mention the Sn , Cn , PN , and FN methods [83, 82, 92, 93], with the most
recent adding and doubling method proposed by Ganapol [94] that achieved the benchmark
with the highest accuracy to date.

In this paper, we aim to adapt the X-TFC method to solve the classical and challenging
Poiseuille flow of a gas in a rarefied state with the BGK approximation. After introducing
the PINNs methodology and why it can bring advances in the field of numerical simulation
for rarefied-gas dynamics, the X-TFC approach for solving general linear ODEs is presented.
Moreover, a particular focus is given to the choice of CE’s free function, explaining the dif-
ference between Chebyshev polynomials and single-layer NN using an activation function. It
is shown why an expansion of Chebyshev polynomials can be considered a NN by selecting
the NN input weights equal to 1 and using the Chebyshev polynomials instead of the acti-
vation functions in each NN neuron, resulting in the so-called “Chebyshev Neural Network”
(ChNN) [95, 96]. Following the formulation of the problem presented by Siewert et al. [97],
our problem reduces to a partial integro-differential equation depending on the space and
velocity of the fluid and subject to certain boundary conditions (see Eqs. (4.6) and (4.7) in
Appendix D). To transform this boundary value problem into two initial value problems, it

27
is convenient to split the PDE into two PDEs, with one condition per each. The velocity
independent variable u is subject to a logarithmic transformation to map its range to [−1, 1]
in order to approximate the integrals on the velocity field with Gauss-Legendre quadrature,
and the new velocity variable µ is discretized according to the Gauss-Legendre quadrature
scheme. Finally, we obtain a set of 2 × N ordinary differential equations to be solved via
X-TFC. Depending on the domain of the activation function chosen as CE free function,
the space independent variable needs to be mapped accordingly. Subsequently, the 2 × N
CEs are built and substituted into our problem, which now becomes unconstrained. The
NN output weights β (which are the only unknown parameters of the linear system Aβ = B
are computed via a least-squares method. When using a ChNN as a CE’s free function,
we found it more efficient to find the least-squares solution of the vector β by computing
β = (AT A)−1 (AT B), while when using an activation function σ it is convenient using the
MATLAB function lsqminnorm for the minimum norm least-squares solution to a linear
equation. With the retrieved β outputs, we can compute the velocity momentum through
the CEs, and then compute the macroscopic velocity and the mass flow rate.

In the paper, a plot of the macroscopic velocity as a function of the channel width for a
fixed accommodation coefficient shows the qualitative results of the computation, providing
the characteristic arched profile of the Poiseuille flow. The numerical results provided in
the tables show the accuracy of X-TFC, achieving up to 8 digits of accuracy of the state-of-
the-art benchmarks published by Barichello et al. [98, 99] and Ganapol [94]. The problem
has been coded in the MATLAB R2019a platform, and all our simulations ran with an Intel
Core i7–9700 CPU PC with 64 GB of RAM.
My contributions in this work are the formulation of the Poiseuille problem to reach a
convenient form of the integro-differential equation to be solved by X-TFC, the X-TFC
formulation to transform the constrained problem into an unconstrained linear system to
be solved via least-square, coding the X-TFC algorithm to solve the Poiseuille problem and
obtain numerical solutions of some physical quantities of interest in the literature, comparing
our results with published and recognized benchmarks, and writing the journal paper.

4.2.2.3 Couette flow (De Florio et al., under review)

This section aims to introduce the paper [11] in Appendix E:
De Florio et al. “Numerical analysis of the plane Couette flow of a rarefied gas
with Physics-Informed Neural Networks and Functional Interpolation.” Physics
of Fluids (under review).
In this third paper of the trilogy on Rarefied-Gas Dynamics, a Couette flow in a micro-
channel is studied. In a steady Couette flow, a rarefied gas is confined between two parallel
plates in relative motion with opposite velocity directions. The flow of the gas is induced
by shear stress imposed by the sliding of the walls. The Couette flow is widely used in real-
world applications, such as lubrication theory with application in bearings [100], polymer
solutions [101], and food processing industry [102, 103]. The pioneering work on the method

28
of elementary solutions in the gas kinetics theory by Cercignani provided all the essential
equations for describing the Couette flow [104]. Over the years, many different techniques
have attempted to produce solutions to this problem, for example, the relevant elementary
solution Fredholm Integral equations by Williams [105], and Loyalka [106], and subsequently
making use of the BGK approximation, solving the Couette problem for a wide range of
Knudsen numbers [107, 108].

This paper aims to solve the Couette flow problem in the BGK approximation with the
X-TFC method, to achieve the accuracy of the most challenging benchmarks in the litera-
ture. In the introduction, a general overview of the birth of the formulation of the problem
and the different numerical techniques used in the last century to solve it are given, with a
discussion on the benefits of using a PINN-based method for solving problems arising from
the Boltzmann integro-differential equation. We start from the formulation of the problem
provided by Williams [109] in the form of a partial integro-differential equation (see Eq. (14)
in Appendix E) as a function of space and fluid velocity subject to two boundary conditions,
one on the wall and one on the center of the channel, by taking advantage of the geometrical
rotated symmetry of the flow. As for the Thermal creep and Poiseuille problems, to apply
X-TFC, there is the need to approximate the integrals with Gauss-Legendre quadrature,
mapping the independent variables, and creating a system of 2 × N of ordinary differential
equations with one constraint per each. By building the CEs and replacing them in our
system, we compute the NN output weights β via a linear least-squares method. Solving the
Couette flow problem means computing the velocity momentum of the flow, with which we
can retrieve three quantities of interest to compare our results with published benchmarks:
macroscopic velocity, flow mass rate, and pressure tensor, which is caused by the motion of
the plates on the gas.

To show the potential of the X-TFC framework in solving the Couette flow, we firstly present
the 3D plot of the velocity momentum as a function of channel width and flow velocity. One
can see the discontinuity for velocity equal to zero, which could not have been possible
to compute without splitting the original boundary value problem into two initial value
problems. Also, two plots show the macroscopic velocity profiles as a function of channel
width for fixed accommodation coefficient and as a function of accommodation coefficient
for a fixed value of channel width. The accuracy of X-TFC is proved by the macroscopic
velocity and flow mass rate numerical results matching eight digits of accuracy, but its real
strength is shown in the results for the pressure tensor. This physical quantity has been
computed for an extended interval of inverse Knudsen numbers, ranging from 10−7 up to
102 , and achieving all the digits of accuracy (8) according to the state-of-the-art benchmark
produced by Barichello et al. [99]. The problem has been coded in the MATLAB R2019a
platform, and all our simulations ran with an Intel Core i7–9700 CPU PC with 64 GB of
RAM.
My contributions in this work are the conceiving of the idea of solving the Couette flow to
test the ability of X-TFC to solve rarefied-gas dynamics problems, the formulation of the

29
problem to reach a convenient form of the integro-differential equation to be solved by X-
TFC, the X-TFC formulation to transform the constrained problem into an unconstrained
linear system to be solved via least-square, coding the X-TFC algorithm to solve the Couette
flow problem and obtain numerical solutions of some physical quantities of interest in the
literature, comparing our results with published and recognized benchmarks, and writing
the journal paper.

30
Chapter 5

X-TFC for Stiff Differential Equations

5.1 Stiff Chemical Kinetics (De Florio et al., 2022)

This section aims to introduce the paper [12] in Appendix F:

De Florio et al. “Physics-informed neural networks and functional interpolation

for stiff chemical kinetics.” Chaos: An Interdisciplinary Journal of Nonlinear
Science 32.6 (2022): 063107.

This methodology paper shows how the domain decomposition technique can be employed in
the X-TFC framework to solve stiff systems on non-linear differential equations with charac-
teristic timescales spanning a wide range of magnitude. The motivation that prompted the
choice of this research is the failure of the regular PINN methods in solving stiff problems. Re-
cently, there have been a few attempts to use NNs to solve these kinds of problems, but with
barely acceptable results, both in terms of accuracy and computational time [110, 111, 112].

In this paper, after an introduction to the definition of “stiffness” and an overview of tra-
ditional and NNs numerical methods developed over the years, a general introduction to
the classic X-TFC framework to solve initial value problems is given. For the first time,
the derivation of an estimate of the generalization error of X-TFC in solving a non-linear
IVP is proved, showing that it is bounded in terms of training error and quadrature er-
ror. Subsequently, the domain decomposition technique merged with the X-TFC framework
is presented, providing instructions on building the CE for each sub-domain. To test the
method, we solved stiff systems on non-linear differential equations for chemical kinetics.
The choice of this typology of problems comes from the great challenge in solving these
highly non-linear dynamic systems and for their multiple applications in the real world, such
as energy conversions, including combustion engines [113], and battery and fuel cells [114].
Also, understanding the chemical reactions is essential for climate modeling [115] and kinet-
ics of methane combustion [116]. The problems we solved are among the most challenging
stiff chemical kinetics systems in the literature, widely used to test new numerical methods.
The first problem is given by the ROBER problem [117], which is modeled by a system of 3

31
ODEs, with reaction rate constants values varying in a range of 9 orders and a time interval
of 105 seconds. The second problem is the chemical Akzo Nobel problem [118], and it de-
scribes a chemical process in which the two species MBT and CHA are initially mixed while
oxygen is continuously added. It consists of a set of 6 non-linear differential equations in a
time domain of 180 seconds. The third problem we considered is the Belousov–Zhabotinsky
reaction mechanism [119], which is represented by five homogeneous chemical reactions mod-
eled by a system of 7 differential equations. This problem is usually reported in the literature
for a time interval of 120 seconds, which we extended up to 5000 seconds to show the real
strength and robustness of X-TFC for stiff problems. Finally, the last problem is a realistic
air pollution model [120, 121], also known as the POLLU problem, developed at The Dutch
National Institute of Public Health and Environmental Protection and modeled by a system
of 20 non-linear ODEs [122], in a time domain of 60 seconds.

X-TFC with domain decomposition outperformed the state-of-the-art methods and regular
PINNs frameworks for all the problems reported in both terms of accuracy and computa-
tional cost, finding challenging solutions where the other methods failed. All the codes have
been implemented in MATLAB, and the simulation ran with an Intel Core i7 - 9700 CPU
PC with 64 GB of RAM.
My contributions in this work are the conceptualization of the time-domain decomposition
technique to lighten the computational efforts of the X-TFC algorithm in solving stiff prob-
lems, and finding the most challenging stiff problems in the literature that deserve further
investigation. Then, for each problem, I coded the X-TFC algorithm for different time
domains obtaining quantitative and qualitative results, which have been compared against
results published in the literature, and against two MATLAB functions (ode15s and ode15i)
which I have implemented and run for the same problems. Finally, I took care of the drafting
of the journal manuscript.

32
Chapter 6

Conclusions and Future Works

In this dissertation, a new Physics-Informed Neural Networks framework named Extreme

Theory of Functional Connections is developed to learn the solution of initial value prob-
lems, focusing on integro-differential equations and stiff systems of non-linear differential
equations.

Two different possible approaches are defined to solve integro-differential equations, depend-
ing on the complexity of the integral kernel at hand. The Theory of Functional Connections
has been adapted to build the integrated constrained expression by analytically integrating
the free chosen function (i.e., orthogonal polynomials or NNs activation function). In re-
alistic problems whose dynamic behavior is modeled by integro-differential equations, the
integral kernel can be non-trivial, for example, an exponential function or an expansion of
Legendre polynomials. Such integral kernel can be found in the challenging Transport The-
ory problems, which describe particles and radiation transport processes in a specific system.
We proposed the X-TFC methodology with a Gauss-Legendre quadrature for the integral
approximation to solve the Boltzmann integro-differential equation for real-world problems
in radiative transfer (transport of photons in a medium) such as the Mie scattering problem
and the Haze-L problem, and for rarefied-gas dynamics, solving the classical Thermal Creep
Flow, Poisueille flow, and Couette flow. The accuracy, efficiency, and robustness of the X-
TFC method have been proved by comparing our results with the most accurate benchmarks
in the literature to date.

The second part of the dissertation focuses on the development of a new framework to solve
the arduous stiff systems of non-linear differential equations, which lies on domain decom-
position technique merged with X-TFC methodology, to capture the steep changes of highly
non-linear dynamical behaviors in relatively wide time domains. To test and validate this
new approach, X-TFC has been employed to solve real-world stiff problems in chemical ki-
netics. The choice of this class of problems comes from their numerical challenge and their
wide applications in real problems, such as energy conversions, batteries, fuel cells, and cli-
mate modeling. X-TFC with domain decomposition showed its accuracy and efficiency by
outperforming the traditional numerical methods designed to solve stiff differential equations

33
and the regular PINNs frameworks, learning the solutions of non-linear dynamical systems
for extraordinary wide time integration ranges where other methods fail.

The research presented in this dissertation is just the beginning of a path that leads to im-
proving and optimizing the X-TFC methodology to solve physical problems in both physics
and data-driven fashion, and its adaptation to solve more challenging and complex prob-
lems. An important research direction that needs to be pursued is to extend the derivation
of bounds on the generalization error of X-TFC to boundary value problems and PDEs, to
provide a rigorous justification for its successful use. This will help to highlight the im-
provements brought by X-TFC compared with the regular PINNs in terms of training error
and error due to a quadrature rule. The forecast would be to derive smaller upper bounds
on the generalization error thanks to the constrained expressions, which analytically satisfy
the problem constraints. Future directions will focus on using the domain decomposition
technique to solve stiff boundary value problems and developing a data-physics-driven ap-
proach for inferring parameters and hidden-physics of stiff systems of non-linear differential
equations. This will help scientists in many fields of science, such as chemistry, biology, and
medicine, to retrieve the dynamics of physical quantities that it is impossible to observe and
acquire data in laboratories.

34
Bibliography

[1] C. J. Edwards, The historical development of the calculus. Springer Science & Business
Media, 2012.
[2] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A
deep learning framework for solving forward and inverse problems involving nonlinear
partial differential equations,” Journal of Computational physics, vol. 378, pp. 686–707,
2019.
[3] S. Wang, Y. Teng, and P. Perdikaris, “Understanding and mitigating gradient flow
pathologies in physics-informed neural networks,” SIAM Journal on Scientific Com-
puting, vol. 43, no. 5, pp. A3055–A3081, 2021.
[4] D. Mortari, “The theory of connections: Connecting points,” Mathematics, vol. 5, no. 4,
p. 57, 2017.
[5] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: a new learning
scheme of feedforward neural networks,” in 2004 IEEE international joint conference
on neural networks (IEEE Cat. No. 04CH37541), vol. 2, pp. 985–990, Ieee, 2004.
[6] E. Schiassi, R. Furfaro, C. Leake, M. De Florio, H. Johnston, and D. Mortari, “Extreme
theory of functional connections: A fast physics-informed neural network method for
solving ordinary and partial differential equations,” Neurocomputing, vol. 457, pp. 334–
356, 2021.
[7] M. De Florio, E. Schiassi, A. D’Ambrosio, D. Mortari, and R. Furfaro, “Theory of func-
tional connections applied to linear odes subject to integral constraints and linear or-
dinary integro-differential equations,” Mathematical and Computational Applications,
vol. 26, no. 3, p. 65, 2021.
[8] M. De Florio, E. Schiassi, R. Furfaro, B. D. Ganapol, and D. Mostacci, “Solutions
of chandrasekhar’s basic problem in radiative transfer via theory of functional con-
nections,” Journal of Quantitative Spectroscopy and Radiative Transfer, vol. 259,
p. 107384, 2021.
[9] M. De Florio, E. Schiassi, B. D. Ganapol, and R. Furfaro, “Physics-informed neural
networks for rarefied-gas dynamics: Thermal creep flow in the bhatnagar–gross–krook
approximation,” Physics of Fluids, vol. 33, no. 4, p. 047110, 2021.

35
[10] M. De Florio, E. Schiassi, B. D. Ganapol, and R. Furfaro, “Physics-informed neural net-
works for rarefied-gas dynamics: Poiseuille flow in the bgk approximation,” Zeitschrift
für angewandte Mathematik und Physik, vol. 73, no. 3, pp. 1–18, 2022.

[11] M. De Florio, E. Schiassi, L. B. Barichello, and R. Furfaro, “Numerical analysis of

the plane couette flow of a rarefied gas with physics-informed neural networks and
functional interpolation,” Physics of Fluids, under-review.

[12] M. De Florio, E. Schiassi, and R. Furfaro, “Physics-informed neural networks and func-
tional interpolation for stiff chemical kinetics,” Chaos: An Interdisciplinary Journal of
Nonlinear Science, vol. 32, no. 6, p. 063107, 2022.

[13] D. M. Burton, “The history of mathematics: An introduction,” Group, vol. 3, no. 3,

p. 35, 1985.

[14] O. Neugebauer, The exact sciences in antiquity, vol. 9. Courier Corporation, 1969.

[15] J. Dutka, “Eratosthenes’ measurement of the earth reconsidered,” Archive for History
of Exact Sciences, pp. 55–66, 1993.

[16] H. C. King, The history of the telescope. Courier Corporation, 2003.

[17] M. A. Finocchiaro, Galileo and the art of reasoning: Rhetorical foundation of logic and
scientific method, vol. 61. Springer Science & Business Media, 2012.

[18] D. Brewster, The martyrs of science, or, The lives of Galileo, Tycho Brahe, and Kepler.
Chatto and Windus, 1874.

[19] J. L. Russell, “Kepler’s laws of planetary motion: 1609–1666,” The British journal for
the history of science, vol. 2, no. 1, pp. 1–24, 1964.

[20] B. S. Baigrie, “Kepler’s laws of planetary motion, before and after newton’s principia:
An essay on the transformation of scientific problems,” Studies in History and Philos-
ophy of Science Part A, vol. 18, no. 2, pp. 177–208, 1987.

[21] J. T. Katsikadelis, “Derivation of newton’s law of motion from kepler’s laws of planetary
motion,” Archive of Applied Mechanics, vol. 88, no. 1, pp. 27–38, 2018.

[22] I. Newton, Philosophiae naturalis principia mathematica, vol. 1. G. Brookman, 1833.

[23] I. Newton, “De methodis serierum et fluxionum,” Mathematical Papers, vol. 3, pp. 32–
353, 1967.

[24] M. Braun and M. Golubitsky, Differential equations and their applications, vol. 1.
Springer, 1983.

[25] L. Euler, Institutionum calculi integralis, vol. 1. impensis Academiae imperialis scien-
tiarum, 1824.

36
[26] J. Crank and P. Nicolson, “A practical method for numerical evaluation of solutions of
partial differential equations of the heat-conduction type,” in Mathematical Proceedings
of the Cambridge Philosophical Society, vol. 43, pp. 50–67, Cambridge University Press,
1947.

[27] J. C. Butcher and G. Wanner, “Runge-kutta methods: some historical notes,” Applied
Numerical Mathematics, vol. 22, no. 1-3, pp. 113–151, 1996.

[28] J. C. Butcher, Numerical methods for ordinary differential equations. John Wiley &
Sons, 2016.

[29] R. Iten, T. Metger, H. Wilming, L. Del Rio, and R. Renner, “Discovering physical
concepts with neural networks,” Physical review letters, vol. 124, no. 1, p. 010508,
2020.

[30] J. P. Crutchfield and B. McNamara, “Equations of motion from a data series,” Complex
systems, vol. 1, pp. 417–452, 1987.

[31] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from

data by sparse identification of nonlinear dynamical systems,” Proceedings of the na-
tional academy of sciences, vol. 113, no. 15, pp. 3932–3937, 2016.

[32] M. Schmidt and H. Lipson, “Distilling free-form natural laws from experimental data,”
science, vol. 324, no. 5923, pp. 81–85, 2009.

[33] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang,

“Physics-informed machine learning,” Nature Reviews Physics, vol. 3, no. 6, pp. 422–
440, 2021.

[34] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and

A. Anandkumar, “Fourier neural operator for parametric partial differential equations,”
arXiv preprint arXiv:2010.08895, 2020.

[35] A. Kashefi, D. Rempe, and L. J. Guibas, “A point-cloud deep learning framework for
prediction of fluid flow fields on irregular geometries,” Physics of Fluids, vol. 33, no. 2,
p. 027104, 2021.

[36] Y. LeCun, Y. Bengio, et al., “Convolutional networks for images, speech, and time
series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995,
1995.

[37] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, “Geomet-

ric deep learning: going beyond euclidean data,” IEEE Signal Processing Magazine,
vol. 34, no. 4, pp. 18–42, 2017.

37
[38] I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solving ordi-
nary and partial differential equations,” IEEE transactions on neural networks, vol. 9,
no. 5, pp. 987–1000, 1998.

[39] H. Sheng and C. Yang, “Pfnn: A penalty-free neural network method for solving a
class of second-order boundary-value problems on complex geometries,” Journal of
Computational Physics, vol. 428, p. 110085, 2021.

[40] K. S. McFall and J. R. Mahan, “Artificial neural network method for solution of bound-
ary value problems with exact satisfaction of arbitrary boundary conditions,” IEEE
Transactions on Neural Networks, vol. 20, no. 8, pp. 1221–1233, 2009.

[41] R. S. Beidokhti and A. Malek, “Solving initial-boundary value problems for systems
of partial differential equations using neural networks and optimization techniques,”
Journal of the Franklin Institute, vol. 346, no. 9, pp. 898–913, 2009.

[42] J. Sirignano and K. Spiliopoulos, “Dgm: A deep learning algorithm for solving partial
differential equations,” Journal of computational physics, vol. 375, pp. 1339–1364, 2018.

[43] G. Pang, L. Lu, and G. E. Karniadakis, “fpinns: Fractional physics-informed neural

networks,” SIAM Journal on Scientific Computing, vol. 41, no. 4, pp. A2603–A2626,
2019.

[44] L. Yang, D. Zhang, and G. E. Karniadakis, “Physics-informed generative adversarial

networks for stochastic differential equations,” SIAM Journal on Scientific Computing,
vol. 42, no. 1, pp. A292–A317, 2020.

[45] A. D. Jagtap, E. Kharazmi, and G. E. Karniadakis, “Conservative physics-informed

neural networks on discrete domains for conservation laws: Applications to forward and
inverse problems,” Computer Methods in Applied Mechanics and Engineering, vol. 365,
p. 113028, 2020.

[46] E. Kharazmi, Z. Zhang, and G. E. Karniadakis, “hp-vpinns: Variational physics-

informed neural networks with domain decomposition,” Computer Methods in Applied
Mechanics and Engineering, vol. 374, p. 113547, 2021.

[47] A. D. Jagtap and G. E. Karniadakis, “Extended physics-informed neural networks

(xpinns): A generalized space-time domain decomposition based deep learning frame-
work for nonlinear partial differential equations,” Communications in Computational
Physics, vol. 28, no. 5, pp. 2002–2041, 2020.

[48] M. Raissi, A. Yazdani, and G. E. Karniadakis, “Hidden fluid mechanics: Learning veloc-
ity and pressure fields from flow visualizations,” Science, vol. 367, no. 6481, pp. 1026–
1030, 2020.

38
[49] R. K. Tripathy and I. Bilionis, “Deep uq: Learning deep neural network surrogate mod-
els for high dimensional uncertainty quantification,” Journal of computational physics,
vol. 375, pp. 565–588, 2018.
[50] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Numerical gaussian processes for time-
dependent and nonlinear partial differential equations,” SIAM Journal on Scientific
Computing, vol. 40, no. 1, pp. A172–A198, 2018.
[51] L. Yang, X. Meng, and G. E. Karniadakis, “B-pinns: Bayesian physics-informed neural
networks for forward and inverse pde problems with noisy data,” Journal of Compu-
tational Physics, vol. 425, p. 109913, 2021.
[52] D. Zhang, L. Lu, L. Guo, and G. E. Karniadakis, “Quantifying total uncertainty in
physics-informed neural networks for solving forward and inverse stochastic problems,”
Journal of Computational Physics, vol. 397, p. 108850, 2019.
[53] S. Cai, Z. Wang, F. Fuest, Y. J. Jeon, C. Gray, and G. E. Karniadakis, “Flow over an
espresso cup: inferring 3-d velocity and pressure fields from tomographic background
oriented schlieren via physics-informed neural networks,” Journal of Fluid Mechanics,
vol. 915, 2021.
[54] D. C. Psichogios and L. H. Ungar, “A hybrid neural network-first principles approach
to process modeling,” AIChE Journal, vol. 38, no. 10, pp. 1499–1511, 1992.
[55] N. Schanche, A. C. Cameron, G. Hébrard, L. Nielsen, A. Triaud, J. Almenara, K. Al-
subai, D. Anderson, D. Armstrong, S. Barros, et al., “Machine-learning approaches
to exoplanet transit detection and candidate validation in wide-field ground-based
surveys,” Monthly Notices of the Royal Astronomical Society, vol. 483, no. 4, pp. 5534–
5547, 2019.
[56] N. M. Ball, R. J. Brunner, A. D. Myers, and D. Tcheng, “Robust machine learning
applied to astronomical data sets. i. star-galaxy classification of the sloan digital sky
survey dr3 using decision trees,” The Astrophysical Journal, vol. 650, no. 1, p. 497,
2006.
[57] J. Stilgoe, “Machine learning, social learning and the governance of self-driving cars,”
Social studies of science, vol. 48, no. 1, pp. 25–56, 2018.
[58] A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun,
“Dermatologist-level classification of skin cancer with deep neural networks,” nature,
vol. 542, no. 7639, pp. 115–118, 2017.
[59] R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal
Statistical Society: Series B (Methodological), vol. 58, no. 1, pp. 267–288, 1996.
[60] D. L. Donoho, “Compressed sensing,” IEEE Transactions on information theory,
vol. 52, no. 4, pp. 1289–1306, 2006.

39
[61] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Machine learning of linear differen-
tial equations using gaussian processes,” Journal of Computational Physics, vol. 348,
pp. 683–693, 2017.

[62] P. Boyle and M. Frean, “Dependent gaussian processes,” Advances in neural informa-
tion processing systems, vol. 17, 2004.

[63] S. Desai and A. Strachan, “Parsimonious neural networks learn interpretable physical
laws,” Scientific reports, vol. 11, no. 1, pp. 1–9, 2021.

[64] S. Weinberg, Dreams of a final theory: The scientist’s search for the ultimate laws of
nature. Vintage, 2011.

[65] S. Fadugba, B. Ogunrinde, and T. Okunlola, “Euler’s method for solving initial value
problems in ordinary differential equations.,” Euler’s Method for Solving Initial Value
Problems in Ordinary Differential Equations., vol. 13, no. 2, pp. 1–7, 2012.

[66] J. C. Butcher, “A history of runge-kutta methods,” Applied numerical mathematics,

vol. 20, no. 3, pp. 247–260, 1996.

[67] W. B. Gragg and H. J. Stetter, “Generalized multistep predictor-corrector methods,”

Journal of the ACM (JACM), vol. 11, no. 2, pp. 188–209, 1964.

[68] S. Godunov and I. Bohachevsky, “Finite difference method for numerical computation
of discontinuous solutions of the equations of fluid dynamics,” Matematičeskij sbornik,
vol. 47, no. 3, pp. 271–306, 1959.

[69] J. N. Reddy, Introduction to the finite element method. McGraw-Hill Education, 2019.

[70] R. Eymard, T. Gallouët, and R. Herbin, “Finite volume methods,” Handbook of nu-
merical analysis, vol. 7, pp. 713–1018, 2000.

[71] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison,

L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” OpenReview, 2017.

[72] R. Gorenflo and S. Vessella, Abel integral equations, vol. 1461. Springer, 1991.

[73] A. J. Lotka, “On an integral equation in population analysis,” The Annals of Mathe-
matical Statistics, vol. 10, no. 2, pp. 144–161, 1939.

[74] I. Fredholm, “Sur une classe d’équations fonctionnelles,” Acta mathematica, vol. 27,
pp. 365–390, 1903.

[75] Z. B. Tsalyuk, “Volterra integral equations,” Itogi Nauki i Tekhniki. Seriya" Matem-
aticheskii Analiz", vol. 15, pp. 131–198, 1977.

[76] R. Bellman and R. S. Roth, The laplace transform, vol. 3. World Scientific, 1984.

40
[77] A. Avudainayagam and C. Vani, “Wavelet–galerkin method for integro–differential
equations,” Applied Numerical Mathematics, vol. 32, no. 3, pp. 247–254, 2000.

[78] P. Van der Houwen and B. Sommeijer, “Euler-chebyshev methods for integro-
differential equations,” Applied Numerical Mathematics, vol. 24, no. 2-3, pp. 203–218,
1997.

[79] P. Darania and A. Ebadian, “A method for the numerical solution of the integro-
differential equations,” Applied Mathematics and Computation, vol. 188, no. 1, pp. 657–
668, 2007.

[80] G. C. Wick, “über ebene diffusionsprobleme,” Zeitschrift für Physik, vol. 121, no. 11,
pp. 702–718, 1943.

[81] S. Chandrasekhar, Radiative transfer. Courier Corporation, 2013.

[82] P. Benoist and A. Kavenoky, “A new method of approximation of the boltzmann equa-
tion,” Nuclear Science and Engineering, vol. 32, no. 2, pp. 225–232, 1968.

[83] M. Vilhena, C. Segatto, and L. Barichello, “A particular solution for the sn radia-
tive transfer problems,” Journal of Quantitative Spectroscopy and Radiative Transfer,
vol. 53, no. 4, pp. 467–469, 1995.

[84] B. D. Ganapol, “The response matrix discrete ordinates solution to the 1d radia-
tive transfer equation,” Journal of Quantitative Spectroscopy and Radiative Transfer,
vol. 154, pp. 72–90, 2015.

[85] M. Knudsen, “Thermischer molekulardruck der gase in röhren,” Annalen der Physik,
vol. 338, no. 16, pp. 1435–1448, 1910.

[86] J. C. Maxwell, “Iii. on stresses in rarefied gases arising from inequalities of tempera-
ture,” Proceedings of the Royal Society of London, vol. 27, no. 185-189, pp. 304–308,
1878.

[87] O. Reynolds, “Xviii. on certain dimensional properties of matter in the gaseous state.-
part i. experimental researches on thermal transpiration of gases through porous plates
and on the laws of transpiration and impulsion, including an experimental proof that
gas is not a continuous plenum.-part ii. on an extension of the dynamical theory of
gas, which includes the stresses, tangential and normal, caused by a varying condition
of gas, and affords an explanation of the phenomena of transpiration and impulsion,”
Philosophical Transactions of the Royal Society of London, no. 170, pp. 727–845, 1879.

[88] C. Cercignani, Theory and application of the Boltzmann equation. Elsevier Science &
Technology, 1975.

[89] G. A. Bird, “Molecular gas dynamics and the direct simulation of gas flows,” Molecular
gas dynamics and the direct simulation of gas flows, 1994.

41
 [90] P. L. Bhatnagar, E. P. Gross, and M. Krook, “A model for collision processes in gases.
i. small amplitude processes in charged and neutral one-component systems,” Physical
review, vol. 94, no. 3, p. 511, 1954.

[91] E. Shakhov, “Generalization of the krook kinetic relaxation equation,” Fluid dynamics,
vol. 3, no. 5, pp. 95–96, 1968.

[92] C. Siewert and J. Thomas Jr, “A particular solution for the pn method in radiative
transfer,” Journal of Quantitative Spectroscopy and Radiative Transfer, vol. 43, no. 6,
pp. 433–436, 1990.

[93] C. Siewert and P. Benoist, “The fn method in neutron-transport theory. part i: Theory
and applications,” Nuclear Science and Engineering, vol. 69, no. 2, pp. 156–160, 1979.

[94] B. D. Ganapol, “Poiseuille channel flow by adding and doubling,” in AIP Conference
Proceedings, vol. 1786, p. 070009, AIP Publishing LLC, 2016.

[95] A. Namatame, “Connectionist learning with chebychev networks and analyses of its
internal representation,” in Applications of Learning and Planning Methods, pp. 35–
48, World Scientific, 1991.

[96] S. Mall and S. Chakraverty, “Single layer chebyshev neural network model for solving
elliptic partial differential equations,” Neural Processing Letters, vol. 45, no. 3, pp. 825–
840, 2017.

[97] C. Siewert, R. Garcia, and P. Grandjean, “A concise and accurate solution for poiseuille
flow in a plane channel,” Journal of Mathematical Physics, vol. 21, no. 12, pp. 2760–
2763, 1980.

[98] L. B. Barichello and C. Siewert, “A discrete-ordinates solution for poiseuille flow in

a plane channel,” Zeitschrift für angewandte Mathematik und Physik ZAMP, vol. 50,
no. 6, pp. 972–981, 1999.

[99] L. Barichello, M. Camargo, P. Rodrigues, and C. Siewert, “Unified solutions to classical

flow problems based on the bgk model,” Zeitschrift für angewandte Mathematik und
Physik ZAMP, vol. 52, no. 3, pp. 517–534, 2001.

[100] E. Barić and H. Steiner, “Extended lubrication theory for generalized couette flow
through converging gaps,” International Journal of Heat and Mass Transfer, vol. 99,
pp. 149–158, 2016.

[101] A. Groisman and V. Steinberg, “Mechanism of elastic instability in couette flow of

polymer solutions: experiment,” Physics of Fluids, vol. 10, no. 10, pp. 2451–2463,
1998.

42
[102] G. A. Krintiras, J. G. Diaz, A. J. Van Der Goot, A. I. Stankiewicz, and G. D. Stefanidis,
“On the use of the couette cell technology for large scale production of textured soy-
based meat replacers,” Journal of Food Engineering, vol. 169, pp. 205–213, 2016.

[103] H. Masuda, Enhancement of Heat Transfer Using Taylor Vortices in Thermal Process-
ing for Food Process Intensification. IntechOpen, 2021.

[104] C. Cercignani et al., Mathematical methods in kinetic theory, vol. 1. Springer, 1969.

[105] M. Williams, “Rarefied gas flow between parallel plates,” in Mathematical Proceedings
of the Cambridge Philosophical Society, vol. 66, pp. 189–196, Cambridge University
Press, 1969.

[106] S. Loyalka, N. Petrellis, and T. Storvick, “Some exact numerical results for the bgk
model: Couette, poiseuille and thermal creep flow between parallel plates,” Zeitschrift
für angewandte Mathematik und Physik ZAMP, vol. 30, no. 3, pp. 514–521, 1979.

[107] E. Gross, E. Jackson, and S. Ziering, “Boundary value problems in kinetic theory of
gases,” Annals of Physics, vol. 1, no. 2, pp. 141–167, 1957.

[108] D. R. Willis, “Comparison of kinetic theory analyses of linearized couette flow,” The
Physics of Fluids, vol. 5, no. 2, pp. 127–135, 1962.

[109] M. Williams, “A review of the rarefied gas dynamics theory associated with some
classical problems in flow and heat transfer,” Zeitschrift für angewandte Mathematik
und Physik ZAMP, vol. 52, no. 3, pp. 500–516, 2001.

[110] W. Ji, W. Qiu, Z. Shi, S. Pan, and S. Deng, “Stiff-pinn: Physics-informed neural
network for stiff chemical kinetics,” The Journal of Physical Chemistry A, vol. 125,
no. 36, pp. 8098–8106, 2021.

[111] S. Kim, W. Ji, S. Deng, Y. Ma, and C. Rackauckas, “Stiff neural ordinary differential
equations,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 31, no. 9,
p. 093122, 2021.

[112] T. S. Brown, H. Antil, R. Löhner, F. Togashi, and D. Verma, “Novel dnns for stiff odes
with applications to chemically reacting flows,” in International Conference on High
Performance Computing, pp. 23–39, Springer, 2021.

[113] F. Battin-Leclerc, E. Blurock, R. Bounaceur, R. Fournet, P.-A. Glaude, O. Herbi-

net, B. Sirjean, and V. Warth, “Towards cleaner combustion engines through ground-
breaking detailed chemical kinetic models,” Chemical Society Reviews, vol. 40, no. 9,
pp. 4762–4782, 2011.

[114] M. Winter and R. J. Brodd, “What are batteries, fuel cells, and supercapacitors?,”
Chemical reviews, vol. 104, no. 10, pp. 4245–4270, 2004.

43
[115] M. Alvanos and T. Christoudias, “Accelerating atmospheric chemical kinetics for cli-
mate simulations,” IEEE Transactions on Parallel and Distributed Systems, vol. 30,
no. 11, pp. 2396–2407, 2019.

[116] M. Frenklach, H. Wang, and M. J. Rabinowitz, “Optimization and analysis of large

chemical kinetic mechanisms using the solution mapping method—combustion of
methane,” Progress in Energy and Combustion Science, vol. 18, no. 1, pp. 47–73, 1992.

[117] H. Robertson, “The solution of a set of reaction rate equations,” Numerical analysis:
an introduction, vol. 178182, 1966.

[118] K. Eriksson, C. Johnson, and A. Logg, “Explicit time-stepping for stiff odes,” SIAM
Journal on Scientific Computing, vol. 25, no. 4, pp. 1142–1157, 2004.

[119] A. M. Zhabotinsky, “Periodical oxidation of malonic acid in solution (a study of the

belousov reaction kinetics),” Biofizika, vol. 9, pp. 306–311, 1964.

[120] A. C. Hindmarsh, “Lsode and lsodi, two new initial value ordinary differential equation
solvers,” ACM Signum Newsletter, vol. 15, no. 4, pp. 10–11, 1980.

[121] F. De Leeuw, “Numerical solution of ordinary differential equations arising from chem-
ical kinetics,” RIVM Rapport 228603005, 1988.

[122] J. G. Verwer, “Gauss–seidel iteration for stiff odes from chemical kinetics,” SIAM
Journal on Scientific Computing, vol. 15, no. 5, pp. 1243–1250, 1994.

44
Appendix A

De Florio et al. (2021) TFC for

Integro-Differential Equations

Reproduced from De Florio, M., Schiassi, E., D’Ambrosio, A., Mortari, D., & Furfaro, R.
(2021). Theory of functional connections applied to linear odes subject to integral con-
straints and linear ordinary integro-differential equations. Mathematical and Computational
Applications, 26(3), 65. https://doi.org/10.3390/mca26030065 [7], with the permission of
MDPI.

45
 Mathematical
and Computational
Applications

Article
Theory of Functional Connections Applied to Linear ODEs
Subject to Integral Constraints and Linear Ordinary
Integro-Differential Equations
Mario De Florio 1 , Enrico Schiassi 1 , Andrea D’Ambrosio 2 , Daniele Mortari 3, * and Roberto Furfaro 1

1 Department of Systems and Industrial Engineering, The University of Arizona, 1127 E. James E. Rogers Way,
Tucson, AZ 85721, USA; mariodf@email.arizona.edu (M.D.F.); eschiassi@email.arizona.edu (E.S.);
robertof@email.arizona.edu (R.F.)
2 School of Aerospace Engineering, Università degli Studi di Roma “La Sapienza”, Via Salaria 851,
00138 Rome, Italy; andrea.dambrosio@uniroma1.it
3 Department of Aerospace Engineering, College of Engineering, Texas A&M University, 401 Joe Routt Blvd.,
College Station, TX 77843, USA
* Correspondence: mortari@tamu.edu

Abstract: This study shows how the Theory of Functional Connections (TFC) allows us to obtain fast
and highly accurate solutions to linear ODEs involving integrals. Integrals can be constraints and/or
terms of the differential equations (e.g., ordinary integro-differential equations). This study first
summarizes TFC, a mathematical procedure to obtain constrained expressions. These are functionals





 representing all functions satisfying a set of linear constraints. These functionals contain a free function,
g( x ), representing the unknown function to optimize. Two numerical approaches are shown to
Citation: De Florio, M.; Schiassi, E.;
D’Ambrosio, A.; Mortari, D.; Furfaro,
numerically estimate g( x ). The first models g( x ) as a linear combination of a set of basis functions,
R. Theory of Functional Connections such as Chebyshev or Legendre orthogonal polynomials, while the second models g( x ) as a neural
Applied to Linear ODEs Subject to network. Meaningful problems are provided. In all numerical problems, the proposed method
Integral Constraints and Linear produces very fast and accurate solutions.
Ordinary Integro-Differential
Equations. Math. Comput. Appl. 2021, Keywords: Theory of Functional Connections; Ordinary Differential Equations; integro-differential
26, 65. https://doi.org/10.3390/ equations; Extreme Learning Machine; numerical methods
mca26030065

Academic Editor: Miguel Ángel

Moreles 1. Introduction
This paper shows how to solve linear Ordinary Differential Equations (ODEs) and
Received: 29 August 2021
Accepted: 10 September 2021
linear Integro-Differential Equations (IDEs) using a new mathematical framework to per-
Published: 12 September 2021
form functional interpolation, called Theory of Functional Connections (TFC). TFC derives
functionals, called constrained expressions, containing a free function and representing all
Publisher’s Note: MDPI stays neutral
possible functions satisfying a set of linear constraints [1–4]. The most important feature of
with regard to jurisdictional claims in
the constrained expressions is: they always satisfy all the constraints no matter what the free
published maps and institutional affil- function is.
iations. Although it was recently developed (2017), TFC has already found several applications,
especially in solving differential equations [5–8]. The free function can be expressed by
any set of linearly independent functions, such as an expansion of orthogonal polynomials
(e.g., Chebyshev, Legendre, etc.) or Neural Networks (NN), such as shallow NN with
Copyright: © 2021 by the authors.
random features, or Deep NNs (DNNs). When the free function is expanded by a set of
Licensee MDPI, Basel, Switzerland.
orthogonal polynomials the method is here identified as “classic TFC”. When shallow NNs
This article is an open access article
with random features are used, the method has been identified as Extreme-TFC (X-TFC)
distributed under the terms and [9], and when DNNs are used the method is called Deep-TFC [10].
conditions of the Creative Commons The best expansion of the free function depends on the differential equation considered.
Attribution (CC BY) license (https:// Usually, for linear/nonlinear ODEs and simple bivariate PDEs, orthogonal polynomials
creativecommons.org/licenses/by/ represent the best choice in terms of accuracy. Indeed, orthogonal polynomials are generally
4.0/). the best mathematical tools for approximating and for convergence properties [11,12].
46

Math. Comput. Appl. 2021, 26, 65. https://doi.org/10.3390/mca26030065 https://www.mdpi.com/journal/mca

Math. Comput. Appl. 2021, 26, 65 2 of 17

Nevertheless, for multivariate and more complex PDEs, orthogonal polynomials suffer the
“curse of dimensionality”. For these problems, NNs represent a better choice as they better
tolerate the curse of dimensionality.
In all previous applications, the free function has been expressed as a linear combina-
tion of known basis functions and unknown constant coefficients. These coefficients are
then estimated by least-squares for linear Differential Equations (DEs) [5] and by nonlinear
least-squares for nonlinear DEs [6]. The problems considered in this work will be solved
with classic TFC using Chebyshev orthogonal polynomials and X-TFC.
Note that X-TFC and Deep-TFC are also identified as Physics-Informed Neural Net-
works (PINN) frameworks [9]. PINNs are recently developed machine-learning methods
that employ NNs for data-physics-driven regression, data-physics-driven solution of DEs,
and data-physics-driven discovery of parameters governing DEs [13]. Since X-TFC and
Deep-TFC use NNs as free function, they are considered part of the PINNs family. Thanks
to the constrained expression, developed by TFC, both X-TFC and Deep-TFC are more
robust and accurate than the standard PINN frameworks as introduced by Raissi et al. [13].
TFC has been developed for univariate and multivariate scenarios [2,7,8] to solve a
variety of mathematical problems: a homotopy continuation algorithm for dynamics and
control problems [14], domain mapping [15], data-driven parameters discovery applied
to epidemiological compartmental models [16], transport theory problems such as radia-
tive transfer [17] and rarefied-gas dynamics [18], nonlinear programming under equality
constraints [19], Timoshenko–Ehrenfest beam [20], boundary-value problems in hybrid sys-
tems [21], eighth-order boundary value problems [22], and in Support Vector Machine [23].
TFC has been widely used for solving optimal control problems for space application,
solved via indirect methods [24]: orbit transfer and propagation [25–28], energy-optimal in
relative motion [29], energy-optimal and fuel-efficient landing on small and large planetary
bodies [30,31], the minimum time-energy optimal intercept problem [32].
The aim of this paper is to show how TFC can accommodate integral constraints in
linear differential equations and solve linear integro-differential equations, is organized as
follows. In Section 2, a summary of the TFC framework is provided, with the explanation
of how to derive constrained expressions. In Section 3, the application of TFC to solve
ODEs with integral constraints is shown, and some case problems are reported as examples.
In Section 4, the TFC framework is used to solve linear integro-differential equations.
In all test problems considered in this article, the results have been obtained using
MATLAB R2020a software on an Intel Core i7-9700 CPU PC with 64 GB of RAM. The results’
accuracy is provided in terms of the absolute error. That is,

err = |yTFC − ytrue |,

where yTFC is the TFC approximated solution, and ytrue is the true analytical solution.
Since the classic TFC uses Chebyshev or Legendre orthogonal polynomials as a basis
set, the final Appendix A provides a definition, orthogonality, derivatives, and integral
expressions and properties for both Chebyshev and Legendre orthogonal polynomials.

2. Theory of Functional Connections Summary

A mathematical generalization of interpolation, called Theory of Functional Connec-
tions, has recently been developed and successfully applied to solve, by least-squares,
initial, boundary, and multi-value problems of linear [5] and nonlinear [6] ODEs and
PDEs [8,10]. The theory has been developed for univariate [1] and multivariate rectangular
domains [2,8].
The generalization of interpolation consists of a mathematical procedure to obtain
analytical expressions describing all possible functions subject to n linear constraints. These
expressions are functionals that are called constrained expressions. Two formally equivalent
approaches [1,8] can be used to derive them. These are,

47
Math. Comput. Appl. 2021, 26, 65 3 of 17

n
y( x, g( x )) = g( x ) + ∑ ηk (x, g(x)) sk (x), (1)
k =1
n
y( x, g( x )) = g( x ) + ∑ φk (x, s(x)) ρk (x, g(x)), (2)
k =1

where n is the number of the linear constraints, g( x ) is a function that can be freely chosen,
ηk ( x, g( x )) are functional coefficients, s( x ) = {s1 ( x ), · · · , sn ( x )} is a set of n user-defined
support functions that must be linearly independent (necessary conditions), φk are switching
functions, and ρk ( x, g( x )) are projection functionals.
By imposing the n constraints to Equation (1), the values of the n functional coef-
ficients, ηk ( x, g( x )), are obtained for some set of n user-assigned functions, sk ( x ). Once
the n functional coefficients, ηk ( x, g( x )), are computed, by substituting them back into
Equation (1) we obtain the constrained expression. To give an example, using s1 ( x ) = x
and s2 ( x ) = x2 , the constrained expression,

x (2x2 − x ) x ( x − 2x1 )
f ( x, g( x )) = g( x ) + ( f − gx ( x )) + ( f x2 − gx ( x2 )),
2( x2 − x1 ) | x1 {z 1 } 2( x2 − x1 ) | {z }
| {z } ρ1 ( x,g( x )) | {z } ρ2 ( x,g( x ))
φ1 ( x ) φ2 ( x )

where the switching functions, φk ( x ), and projection functionals, ρk ( x, g( x )), are identified,
always satisfies the two derivative constraints, f x ( x1 ) = f x1 and f x ( x2 ) = f x2 , no matter
what g( x ) is. In this article, the use of both formulations, provided by Equations (1) and (2),
will be shown.
The projection functional, ρk ( x, g( x )), projects the free function, g( x ), on the k-th
constraint. Here is an example of projection functionals associated to a set of constraints,
 
 f (3) = 9
  ρ1 = 9 − g (3)

3 f (0) − f (2) = 0 → ρ2 = −3g(0) + g(2) .

¨ 

f (1) = −1 ρ3 = −1 − g̈(1)

As for the switching functions, φk ( x ), they are expressed as a linear combination of a

set of support functions, s( x ). These functions satisfy φk ( x ) = 1 when the k-th constraint
is verified and φk ( x ) = 0 when any other constraint is verified. For instance, given the
support functions,

s1 ( x ) = x, s2 ( x ) = e x , and s3 ( x ) = sin x,

3
the switching functions are computed as φj ( x ) = ∑ α ji si (x), where the α ji coefficients are
i =1
computed by inverting the support matrix. For this example,
    
α11 α12 α13 s1 (3) 3s1 (0) − s1 (2) s̈1 (1) 1 0 0
α21 α22 α23 s2 (3) 3s2 (0) − s2 (2) s̈2 (1) = 0 1 0.
α31 α32 α33 s3 (3) 3s3 (0) − s3 (2) s̈3 (1) 0 0 1

Reference [4] presents, in detail, how to derive the TFC constrained expressions using
the formalism defined by Equation (2).
Constrained expressions have been used to solve differential equations. This is done
by expressing the solution using the TFC constrained expressions from Equation (1) or,
equivalently, from Equation (2). These functionals allow us to reduce the whole functions
space to only the functions subspace satisfying the constraints. This is particularly impor-
tant when solving differential equations. In fact, when substituting these functionals into
the DE, a new differential equation is obtained in terms of g( x ). This new DE is subject to no
constraints because the constrained expression fully satisfies the constraints. The unknown
48
Math. Comput. Appl. 2021, 26, 65 4 of 17

free function, g( x ), is then expressed as a linear combination of basis functions. In the

classic TFC, g( x ) is a linear combination of orthogonal polynomials. That is,
m
g( x, ξ ) = ∑ ξ j h j (x) = ξ T
h ( x ),
j =0

where m is the number of the basis functions h j ( x ) (orthogonal polynomials). In X-TFC,

g( x, ξ ) is a shallow NN trained with Extreme Learning Machine (ELM) [33],
 
L
 σ1 ( x ) 
 
..
g( x, ξ ) = ∑ ξ j σj wj x + bj = ξ T σ (x) = ξ T
 . 
j =1  
σL ( x ),

where L is the number of hidden neurons, w j ∈ R is the input weights vector connecting
the j-th hidden neuron and the input nodes, ξ j ∈ R with j = 1, · · · , L is the j-th output
weight connecting the j-th hidden neuron and the output node, and b j is the bias of the
j-th hidden neuron, σj (·) are activation functions, and σ = {σ1 , · · · , σL }T . According to
the ELM algorithm [33], biases and input weights are randomly selected and not tuned
during the training, thus they are known hyper-parameters. The activation functions, σj (·),
are also known as they are user selected. Thus, the only unknown NN hyper-parameters
to compute are the output weights ξ = {ξ 1 , · · · , ξ L }T . Therefore, for both classic TFC
and X-TFC, the unknown vector, ξ, which actually constitutes the only unknown of our
problem, is then estimated numerically, as for instance by least-squares for linear [5] and
nonlinear [6] differential equation s.
In general, the independent variable of the DE (for instance, time) is defined in the
t ∈ [t0 , t f ] range, while the selected basis functions may be defined as a different range,
x ∈ [ x0 , x f ] (for instance, Chebyshev and Legendre orthogonal polynomials are defined in
the x ∈ [−1, +1]). Thus, a change of variable is needed. The most simple mapping between
these two variables is linear,
x f − x0 t f − t0
x = x0 + ( t − t0 ) ←→ t = t0 + ( x − x0 ). (3)
t f − t0 x f − x0

x f − x0
By setting the range ratio, c = , the derivatives in terms of the new variable are
t f − t0

dk f dk f
k
= ck , (4)
dt dx k
and the derivative constraints can be written as:

dk f (k) dk f (k)
= f ti = c k = c k f xi . (5)
dtk dx k
ti xi

The change of variable in integrals takes advantage from the fact that the mean value
is independent from the independent variable. Therefore,
Z t Z x Z t Z x
1 f 1 f f f
f (t) dt = f ( x ) dx → c f (t) dt = f ( x ) dx.
t f − t0 t0 x f − x0 x0 t0 x0

When expressing the free function as g( x, ξ ) = ξ T h( x ), then, the derivatives of g( x, ξ )

can be written as:

dk g dk h dk g dk h
k
= ξT and = ξT .
dx dx k dx k dx k
xi xi

49
Math. Comput. Appl. 2021, 26, 65 5 of 17

This procedure can be applied to linear differential equations with non-constant

coefficients. The final expression obtained is linear in terms of the unknown vector, ξ,
and can be written as:
aT ( x ) ξ = b ( x ). (6)
To solve the problem numerically, this new equation is discretized for a set of N distinct
values of x. A different discretization scheme can be used. Usually, the discretization
points are either randomly selected or linearly uniformly spaced. When using Chebyshev
polynomials the best discretization is to use the zeros of the Chebyshev polynomials, also
called Chebyshev points or nodes or, more formally, Chebyshev–Gauss points [12]. They
are defined by the cosine distribution,
 
(2k − 1)π
xk = cos for k = 1, · · · , N.
2N

By specifying Equation (6) for these xk values a system of N linear equations is

obtained in m (or L) unknowns that is then solved for ξ by least-squares. Several least-
squares methods can be used to solve (6). The optimal least-squares method to use for each
problem depends on the problem itself and on what TFC technique is used to solve the
problem (e.g., in this paper, classic TFC or X-TFC). For instance, for the classic TFC and
linear DEs, our analysis identifies the QR decomposition on a scaled coefficient matrix as
the best approach minimizing the condition number of the matrix to invert.

3. TFC for ODEs with Integral Constraints

In this section, we show how TFC is applied to solve linear ODEs with integral
constraints. After showing how to derive the constrained expression when dealing with
a general integral constraint, we will solve a couple of linear ODEs subjects to boundary
conditions and integral constraints.
For the first two examples in this section, we will derive the constrained expression
by following the formulation of Equation (1). For subsequent examples and problems,
however, the second formulation, Equation (2), will be adopted.

3.1. Definite Integral Constraint

Let us consider the integral constraint,
Z b
f ( x ) dx = I. (7)
a

The constrained expression has the form,

f ( x, g( x )) = g( x ) + η1 s1 ( x ), (8)

where g( x ) is a free function and s1 ( x ) is a user-defined support function, and η1 is a

coefficient that is derived by imposing the constraint. By integrating Equation (8) the
following equation:
Z b Z b
I= g( x ) dx + η1 s1 ( x ) dx
a a
is obtained, which can be rearranged to obtain the expression for η1 ,
Z b
I− g( x ) dx
a
η1 = Z b .
s1 ( x ) dx
a

50
Math. Comput. Appl. 2021, 26, 65 6 of 17

Substituting this value into Equation (8), we obtain a constrained expression for f ( x ),
that is, an expression that always satisfies the integral constraint for any expression of g( x ),
 Z b 
s1 ( x )
f ( x, g( x )) = g( x ) + Z b
I− g( x ) dx . (9)
a
s1 ( x ) dx
a
Z b
This function becomes undefined when s1 ( x ) dx = 0, and thus s1 ( x ) must be
a
selected to avoid this condition. By simply selecting s1 ( x ) = 1 Equation (9) becomes,
 Z b 
1
f ( x, g( x )) = g( x ) + I− g( x ) dx .
b−a a

3.2. Integral and Linear Constraints

In this second example, let us consider the more complex case where, in addition to the
integral constraint given in Equation (7), we also consider the additional linear constraints,

α f ( x0 ) + β f ( x f ) = 1,

where α, β, x0 , and x f are all assigned. A sketch of this example is shown in Figure 1.

Figure 1. Integral and linear constraints example.

The constrained expression has the form,

f ( x, g( x )) = g( x ) + η1 s1 ( x ) + η2 s2 ( x ). (10)

Then, by applying the constraints the system of linear equations,

   
 1 − α g0 − β g f  α s1 ( x0 ) + β s1 ( x f ) α s2 ( x0 ) + β s2 ( x f )  
Z b
=  Z b Z b  η1 (11)
I − g( x ) dx  s1 ( x ) dx s2 ( x ) dx η2
a a a

is obtained. This system tells us that s1 ( x ) and s2 ( x ) functions can be any functions with
the exception of those making the matrix singular. The matrix singularity occurs when
 Z b  Z b
α s1 ( x0 ) + β s1 ( x f ) s2 ( x ) dx = α s2 ( x0 ) + β s2 ( x f ) s1 ( x ) dx.
a a

For instance, by selecting s1 ( x ) = 1 and s2 ( x ) = x, the previous condition becomes,

(α + β)(b + a) = 2(α x0 + β x f )

which imply that, in order to avoid singularity, the following relationship must be satisfied,
 
 b + a − 2αx0
α, β 6= 0.
b + a − 2βx f

51
Math. Comput. Appl. 2021, 26, 65 7 of 17

When this condition is satisfied, the matrix of Equation (11) can be inverted and the
coefficients, η1 and η, can be computed. Then, the constrained expression for this problem
is obtained by substituting the expressions found for η1 and η2 in Equation (10).

Problem #1
Now, let us consider the following DE to be solved on the t ∈ [0, π ] range, with a
constraint in the form of an integral over the integration range,

 f (0) = 1

f¨ + f = 0 subject to: Z π ,

 f (t) dt = π
0
whose analytical solution is
π
f = sin t + cos t.
2
In this problem, we build the constrained expression according to the formulation of
Equation (2), where the projection functionals ρk ( x, g( x )) are given by

 ρ 1 = 1 − g0
Z π
.
 ρ2 = π − g(t) dt
0

Given the support functions,

s1 (t) = 1, and s2 (t) = t,

2
the switching functions are computed as φj (t) = ∑ α ji si (t), where the α ji coefficients are
i =1
computed just by inverting a matrix. For this problem,
 Z π 
  s (0) s1 (τ ) dτ  
α11 α12  1 Z0 π  1 0
  = .
α21 α22 s (0) s2 (τ ) dτ 0 1
2
0

Then, the constrained expression can be built as:

 Z +π

f (t, g(t)) = g(t) + φ1 (t)(1 − g0 ) + φ2 (t) π − g(τ ) dτ ,
0

where the switching functions are

π − 2t 2t
φ1 (t) = and φ2 (t) = .
π π2
Now, expressing the free function according to g(t, ξ ) = ξ T h( x (t)) the terms of the
DE become:
 Z T
1 1
f ( x, ξ ) = h( x ) − φ1 ( x )h0 − φ2 ( x ) h( x ) dx ξ + φ1 ( x ) + πφ2 ( x )
c −1
f¨( x, ξ ) = c2 ḧ( x )T ξ,

where the boundary conditions have been mapped to x ∈ [ x0 , x f ]. For classic TFC x ∈
[−1, +1], and for X-TFC x ∈ [0, 1] according to Equations (3)–(5). The mapping coefficient
x f − x0
is c = .
t f − t0

52
Math. Comput. Appl. 2021, 26, 65 8 of 17

With the function now expressed in terms of ξ the differential equation can now be
transformed into a function of ξ,
 Z 1 T
1
c2 ḧ( x ) + h( x ) − φ1 ( x ) h0 − φ2 ( x ) h( x ) dx ξ = −φ1 ( x ) − πφ2 ( x ).
c −1

This equation can then be specified for a discrete values of x. This discretization yields
to an over-determined linear system that can be solved by least-squares.

Aξ=b → ξ = (AT A)−1 AT b. (12)

In this example, using n =
100 and m = 20, the classic TFC was executed with a
computational −4 s, the average absolute error on the discretization points
16

time is of O 10

is of O 10 − , the variance of the absolute error on the discretization points is of O 10−16 .
For X-TFC, we set the following hyper-parameters n = 100, L = 100, Gaussian activation
function, input weight

and bias were sampled from U [−10, +10] . The computational

time is of O 10−4 s, the average absolute error on the discretization points
is of O 10−16 ,
the variance of the absolute error on the discretization points is of O 10−16 . Table 1 reports
the absolute error with respect to the analytical solutions obtained with classic TFC and
X-TFC on 11 points uniformly distributed in the [0, 1] range.

Table 1. Absolute errors on uniform test points for problem #1.

t/π TFC X-TFC

0.0 0.0 0.0
0.1 0.0 2.22 × 10−16
0.2 0.0 2.22 × 10−16
0.3 0.0 0.0
0.4 0.0 0.0
0.5 0.0 0.0
0.6 4.44 × 10−16 4.44 × 10−16
0.7 1.11 × 10−16 2.22 × 10−16
0.8 6.66 × 10−16 0.0
0.9 1.66 × 10−16 2.77 × 10−16
1.0 2.22 × 10−16 6.66 × 10−16

3.3. Mixed Constraints

Consider a function subject to a value-level, relative, and integral constraint,
Z t
1 f
f ( t1 ) = f 1 , f ( t0 ) = f ( t f ), and f (τ ) dτ = I.
t f − t0 t0

By using Equation (2), the constrained expression has the form,

f (t, g(t)) = g(t) + φ1 (t) ρ1 (t, g(t)) + φ2 (t) ρ2 (t, g(t)) + φ3 (t) ρ3 (t, g(t))

where the projection functionals, ρk (t, g(t)), are given by



 ρ1 = f 1 − g ( t1 )


ρ2 = g ( t f ) − f ( t0 ) .
Z t


 ρ3 = I − 1 f
 g(τ ) dτ
t f − t 0 t0

53
Math. Comput. Appl. 2021, 26, 65 9 of 17

Given the support functions,

s1 (t) = 1, s2 (t) = t, and s3 ( t ) = t2

3
the α ji coefficients of the switching functions, φj (t) = ∑ α ji si (t), are computed by inverting
i =1
the support matrix. Specifically,
 1
R +π 
  s (t ) s1 ( t0 ) − s1 ( t f ) s1 (τ ) dτ  
α11 α12 α13  1 1 t f − t0
R−+ππ  1 0 0
1
 s2 ( t1 )
α23  s2 ( t0 ) − s2 ( t f ) −π s2 ( τ ) dτ  = 0
α21 α22   1 0.
t f − t0
α31 α32 α33 s (t ) 1
R +π 0 0 1
3 1 s3 ( t0 ) − s3 ( t f ) t f − t0 −π s3 ( τ ) dτ

Once the φj (t) are known, the constrained expression can be obtained as:
 
f (t, g(t)) = g(t) + φ1 (t) ( f 1 − g(t1 )) + φ2 (t) f (t f ) − g(t0 ) +
Z t
!
1 f
+ φ3 (t) I − g(τ ) dτ .
t f − t 0 t0

Problem #2
As an ultimate “stress-test” of the TFC method, a mixed-constraint case is considered
where point, relative, and integral constraints are used in the solution of a differential
equation, on the range t ∈ [−π, π ]


 y ( t0 ) = 0




...
y + sin t ÿ + (1 − t)ẏ + ty = f (t) subject to: y(t f ) = y(t0 )


Z +π


 y(τ ) dτ = −2π
−π

where the forcing term is

f (t) = (t − 1) sin2 t + (2 + 2t − t2 − 2 cos t) sin t + t(t − 1) cos t.

This problem admits the analytical solution,

y = (1 − t) sin t.

Following the procedure explained, the constrained expression for this problem is,

 Z +π

y(t, g(t)) = g(t) + φ1 (t)(− g0 ) + φ2 (t) (− g f ) + φ3 (t) −2π − g(τ ) dτ ,
−π

3
where the α ji coefficients of the switching functions φj (t) = ∑ α ji si (t) are computed by
i =1
solving the following system:
 Z +π 
  s1 ( t0 ) s1 ( t f ) s1 (τ ) dτ   
α11 α12 α13  Z−+ππ  1 0 0
α21  
α22 α23 s2 (t0 ) s2 ( t f ) s2 (τ ) dτ  = 0 1 0.
 Z−+ππ 
α31 α32 α33   0 0 1
s3 ( t0 ) s3 ( t f ) s3 (τ ) dτ
−π
54
Math. Comput. Appl. 2021, 26, 65 10 of 17

Now, expressing the free function according to g(t) = ξ T h( x (t)) and rearranging
terms leads us to the final constrained expression,

 Z x T
1 f
y( x, ξ ) = h( x ) − φ1 ( x ) h0 − φ2 ( x ) h f − φ3 ( x ) h( x ) dx ξ − 2π φ3 ( x ),
c x0

and by substituting this constrained expression into the differential equation (by evaluating
its derivatives accordingly), we obtain a linear system that can be solved by least-squares
using Equation (12).
Even in this case, the differential equation and boundary conditions must be mapped
onto x ∈ [ x0 , x f ]. For classic TFC x ∈ [−1, +1], and for X-TFC x ∈ [0, 1] according to
x f − x0
Equation (3), where the mapping coefficient is c = .
t f − t0


For classic TFC we set n = 35 and m = 30. The computational
time is of O 10−4 s,
the average absolute error on the discretization points is
of O 10−15 , the variance of the
absolute error on the discretization points is of O 10−30 . For X-TFC we set the following
hyper-parameters n = 40, L = 40, sinusoidal activation function, input weight
and bias
were sampled from U [−20, +20] . The computational
time is of O 10−4 s, the average
absolute error on the discretization points
is of O 10−15 , the variance of the absolute error
on the discretization points is of O 10−30 .

3.4. Discussions
The results from these two problems emphasize the utility and convenience of using
TFC. Once the constrained expression is defined, the method to solving the differential
equation does not change regardless of the use of different constraints. Moreover, this
makes the solution accuracy only dependent on the problem complexity. Finally, once the
solution is computed (on the discretization points), we have an analytical representation of
it. That is, no further manipulations are needed (e.g., interpolation) if we want to evaluate
the solutions in points that are different from the discretization points. Indeed, as shown in
the results of both problems we do not lose any accuracy when evaluating the solution on
test points, as it would happen with some other state-of-the-art methods such as the Finite
Element Method (FEM) [34].

4. TFC for Linear Ordinary Integro-Differential Equation

In this section we show how TFC is applied to solve linear ordinary Integro-Differential
Equations (IDEs) using all constrained expressions derived from the formalism of Equation (1).
As first test problem, the linear Fredholm integro-differential equations is considered.
Comparisons of the numerical results obtained with TFC, X-TFC, and the method published
in Ref. [35] are presented.

4.1. Problem #1
Consider the following integro-differential equation:
Z 1
1
ẏ( x ) = 1 − x+x τ y(τ ) dτ
3 0

with one constraint y(0) = 0, for x ∈ [0, 1]. The analytical solution is y( x ) = x. The con-
strained expression for this problem is simply

y( x, ξ ) = (h( x ) − h0 )T ξ,

where h0 is the vector of the basis function computed at x = 0.

For classic TFC n = 100 discrete
points and m = 29 basis functions were used.
The computational time is of O 10−4 s, the average absolute error on the discretization
55
Math. Comput. Appl. 2021, 26, 65 11 of 17

points is of
O 10−5 , and the variance of the absolute error on the discretization points is
of O 10 − 9 . For X-TFC the hyper-parameters were set to n = 20, L = 20, the activation
function was sinusoidal, and the input weight and
bias were sampled from U [−1, +1].
The computational time obtained
was of O 10−4 s, the average absolute error on the
discretization points −4 , the variance of the absolute error on the discretization
− 8

of O 10
points of O 10 . The absolute errors obtained with classic TFC and X-TFC on 10 test
points are reported in Figure 2 and compared with those reported by Ref. [35].

Figure 2. Absolute error on test points for problem #1.

4.2. Problem #2
The second test problem is on the following integro-differential equation:
Z 1
ẏ( x ) = x e x + e x − x + x y(τ ) dτ,
0

with one constraint y(0) = 0, for x ∈ [0, 1]. The analytical solution is y( x ) = xe x . Also for
this this problem the constrained expression is

y( x, ξ ) = (h( x ) − h0 )T ξ.

For classic TFC n = 25 discretization points
and m = 14 basis functions were adopted.

The computational time obtained −4 s, the average absolute error on the dis-
− 16

is of O 10
cretization points is of

O 10 , and the variance of the absolute error on the discretization
points is of O 10−32 . For X-TFC the hyper-parameters were set to n = 50, L = 50, the acti-
vation function was the hyperbolic tangent, and the input
weight and bias were sampled
from U [−1, +1]. The computational time is of O 10 −4 s, the average absolute error on


−14 , and the variance of the absolute error on the
the discretization points is of O 10

discretization points is of O 10−27 . The absolute errors on the test points obtained with
classic TFC and X-TFC, are reported in Figure 3 and compared with those reported by
Ref. [35].

56
Math. Comput. Appl. 2021, 26, 65 12 of 17

Figure 3. Absolute error on test points for problem #2.

4.3. Problem #3
The last test problem is a second-order integro-differential equation,
Z 1
ÿ( x ) = e x − x + x τ y(τ ) dτ,
0

subject to the initial value problem, y(0) = 1 and ẏ(0) = 1, where x ∈ [0, 1]. The analytical
solution is y( x ) = e x . The constrained expression for this problem is:

T
y( x, ξ ) = h( x ) − h0 − x ḣ0 ξ + x + 1.

To solve this problem, the classic TFC required n = 9000 points and m = 1000 basis
functions. The computational time was
of O(1) second, the average absolute error on
the discretization points was of O 10
−5 , and the variance of the absolute error on the
discretization points was of O 10−10 . For X-TFC the setting was n = 90 and L = 94
hyper-parameters, hyperbolic sine activation function, and input
weight and bias were
sampled from U [−1, +1]. The computational
time was of O 10−4 s, the average absolute
error on the discretization points of −4 , and the variance of the absolute error on
− 7

O 10
the discretization points of O 10 . The absolute errors on the test points obtained with
classic TFC and X-TFC, are reported in Figure 4 and compared with those reported by
Ref. [35].

57
Math. Comput. Appl. 2021, 26, 65 13 of 17

Figure 4. Absolute error on test points for problem #3.

4.4. Discussions
The discussions made for the linear ODE subjects to integral constraints are also valid
for linear ordinary integro-differential equations. Nevertheless, the attentive reader will
notice that there is a significant difference in terms of accuracy among problems 1 and 3
compared to problem 2. This is due to the fact that the kernels in the integral terms of
the equations are different than one for problems 1 and 3 (e.g., for the both problems the
kernel is t). This can have a negative impact on the solution accuracy as the function to be
integrated can become very complex, and this can lead to numerical issues. For instance,
when classic TFC is used in problems 1 and 3, where the kernel is simply t, the coefficients
in front of the integrated polynomials become enormous as the number of basis functions
(e.g., the degree of the Chebyshev polynomials) reaches above ten, causing numerical
issues that impact negatively on the accuracy. Moreover, in many real world applications,
such as in radiative transfer equation [17,36], the kernel can be very complex, making the
analytical evaluation of the integral prohibitive.
The are several ways of mitigating these issues that are worth exploring. For instance,
the integral could be evaluated with a Gauss-quadrature scheme or with a Monte Carlo
method. The authors of this manuscript are investigating the possibility of evaluating the
integral numerically using NNs. However, the investigation of different ways to overcome
these limitations is not the focus of this work, and it will be explored by the authors in
future papers.

5. Conclusions
This study presents an extension of the least-squares solution of linear differential
equations studied in an earlier publication [5]. This paper aims to explore solutions using
different boundary constraints including multi-valued, relative, and integral constraints for
linear ordinary differential equations and for linear ordinary integro-differential equations.
In all problems, a constrained expression is used to embed the constraints. This allows the
integration range to be independent from the constraints location. In these expressions,
the free function g( x ) was expressed as a linear combination of known basis functions and
58
Math. Comput. Appl. 2021, 26, 65 14 of 17

unknown constant coefficients ξ, either using Chebyshev polynomials, or shallow NNs

for the X-TFC framework. Expressing g( x ) as a linear combination of basis functions and
constant coefficients, the coefficient vector ξ remains linear and is solved by using a least-
squares method. While this paper only solves linear differential equations as a test case,
the proposed methodology can easily be extended to nonlinear differential equations by
replacing the least-squares method with a nonlinear least-squares technique as introduced
in prior research [6].
For all cases explored, the classic TFC and X-TFC methods consistently produce very
fast and accurate solutions. Additionally, for all problems, the constraints are analytically
satisfied since they are integrated into the constrained expression. In general, this means that
the TFC methods decouple the constraints from the solution of the differential equation.
Due to this, the solution range of the differential equation and, where the constraints are
specified, is independent. This fact makes the TFC method a unified framework to solve
differential equations with no more distinctions between the initial and boundary values
problems, as well as any other constraints distribution problem.

Appendix A. Chebyshev and Legendre Orthogonal Polynomials

This appendix provides compact summaries of Chebyshev, Tk ( x ), and Legendre,
Lk ( x ), orthogonal polynomials, which are defined in the xß[−1, +1] range. These are:

Appendix A.1. Definition

Starting with T0 = L0 = 1 and T1 = L1 = x these orthogonal polynomials can be
conveniently defined recursively,

2k + 1 k
Tk+1 = 2 x Tk − Tk−1 and L k +1 = x Lk − L
k+1 k + 1 k −1
and, specifically (∀k),

Tk (−1) = Lk (−1) = (−1)k and Tk (+1) = Lk (+1) = 1.

Appendix A.2. Orthogonality

The orthogonality is provided by the following integrals,

Z +1
dx  0 if i 6= j
Ti ( x ) Tj ( x ) √ = π if i = j = 0
−1 1−x 2 
π/2 if i = j 6= 0
Z +1
2
Li ( x ) L j ( x ) dx = δ .
−1 2j + 2 ij

Appendix A.3. Derivatives

All derivatives of Chebyshev orthogonal polynomials can also be computed recur-
sively. Starting from,

dT0 dT1 dd T0 dd T1
= 0, = 1, and d
= =0 (∀ d > 1),
dz dz dz dzd
the subsequent derivatives can be computed by:
!
dd Tk+1 dd−1 Tk dd Tk dd Tk−1
d
= 2 d d 1
+ z d
− (k ≥ 1, ∀ d ≥ 1).
dz dz − dz dzd

59
Math. Comput. Appl. 2021, 26, 65 15 of 17

The derivatives of Legendre orthogonal polynomials can also be computed recursively.

Starting from,

dL0 dL1 dd L 0 dd L 1
= 0, = 1, and = =0 (∀ d > 1),
dz dz dzd dzd
the subsequent derivatives can be computed by,
!
dd L k +1 2k + 1 dd −1 L k dd L k k dd L k −1
d
= d d 1
+ z d
− (k ≥ 1, ∀ d ≥ 1).
dz k+1 dz − dz k + 1 dzd

Appendix A.4. Integral

• Chebyshev indefinite.
Z x Z x
1 2

T0 (z) dz = x + 1, T1 (z) dz =
x −1 ,
−1 −1 2
Z x
k Tk+1 x Tk
then → Tk (z) dz = 2 −
−1 k − 1 k −1

• Chebyshev full range.


Z +1  0
 for k odd
Tk ( x ) dx = (−1)k +1
−1 
 for k even
1 − k2

• Chebyshev internal range (−1 ≤ a < b ≤ +1)

Z b  
k Tk+1 (b) − Tk+1 ( a) b Tk (b) − a Tk ( a)
Tk ( x ) dx = −
a k2 − 1 k−1

• Legendre indefinite.
Z x Z x
1 2

L0 (z) dz = x + 1, L1 (z) dz = x −1 ,
−1 −1 2
Z x
L k +1 ( x ) − L k −1 ( x )
then → Lk (z) dz =
−1 2k + 1

• Legendre full range.

Z +1 Z +1
L0 ( x ) dx = 2 and Lk ( x ) dx = 0, ∀ k 6= 0.
−1 −1

• Legendre internal range (−1 ≤ a < b ≤ +1)

Z b
L k +1 ( b ) − L k +1 ( a ) + L k −1 ( a ) − L k −1 ( b )
Lk ( x ) dx = .
a 2k + 1

Author Contributions: Conceptualization, D.M.; methodology, D.M., M.D.F., E.S. and A.D.; software,
M.D.F., E.S., A.D. and D.M.; validation, M.D.F., E.S. and D.M.; formal analysis, D.M., M.D.F. and
E.S.; writing—original draft preparation, D.M., M.D.F., E.S. and A.D.; writing—review and editing,
D.M., E.S. and M.D.F.; supervision, R.F. and D.M. All authors have read and agreed to the published
version of the manuscript.
Funding: This research received no external funding.

60
Math. Comput. Appl. 2021, 26, 65 16 of 17

Acknowledgments: The authors acknowledge Hunter Johnston for the initial investigation on
integral constraints using the Theory of Functional Connections.
Conflicts of Interest: The author declares no conflict of interest.

References
1. Mortari, D. The theory of connections: Connecting points. Mathematics 2017, 5, 57.
2. Mortari, D.; Leake, C. The multivariate theory of connections. Mathematics 2019, 7, 296.
3. Johnston, H.R. The theory of functional connections: A journey from theory to application. Ph.D. Thesis, Texas A&M University,
College Station, TX, USA, 2021.
4. Leake, C.D. The Multivariate Theory of Functional Connections: An n-dimensional Constraint Embedding Technique Applied to
Partial Differential Equations. Ph.D. Thesis, Texas A&M University, College Station, TX, USA, 2021.
5. Mortari, D. Least-squares solution of linear differential equations. Mathematics 2017, 5, 48.
6. Mortari, D.; Johnston, H.; Smith, L. High accuracy least-squares solutions of nonlinear differential equations. J. Comput. Appl.
Math. 2019, 352, 293–307.
7. Mortari, D.; Furfaro, R. Univariate theory of functional connections applied to component constraints. Math. Comput. Appl. 2021,
26, 9.
8. Leake, C.; Johnston, H.; Mortari, D. The multivariate theory of functional connections: Theory, proofs, and application in partial
differential equations. Mathematics 2020, 8, 1303.
9. Schiassi, E.; Furfaro, R.; Leake, C.; De Florio, M.; Johnston, H.; Mortari, D. Extreme theory of functional connections: A
fast physics-informed neural network method for solving ordinary and partial differential equations. Neurocomputing 2021,
457, 334–356.
10. Leake, C.; Mortari, D. Deep theory of functional connections: A new method for estimating the solutions of partial differential
equations. Mach. Learn. Knowl. Extr. 2020, 2, 37–55.
11. Gil, A.; Segura, J.; Temme, N.M. Numerical Methods for Special Functions; SIAM: Philadelphia, PA, USA, 2007.
12. Lanczos, C. Applied Analysis; Courier Corporation: Chelmsford, MA, USA, 1988.
13. Raissi, M.; Perdikaris, P.; Karniadakis, G.E. Physics-informed neural networks: A deep learning framework for solving forward
and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 2019, 378, 686–707.
14. Wang, Y.; Topputo, F. A TFC-based homotopy continuation algorithm with application to dynamics and control problems. J.
Comput. Appl. Math. 2021, 401, 113777.
15. Mortari, D.; Arnas, D. Bijective mapping analysis to extend the theory of functional connections to non-rectangular 2-dimensional
domains. Mathematics 2020, 8, 1593.
16. Schiassi, E.; De Florio, M.; D’Ambrosio, A.; Mortari, D.; Furfaro, R. Physics-informed neural networks and functional interpolation
for data-driven parameters discovery of epidemiological compartmental models. Mathematics 2021, 9, 2069.
17. De Florio, M.; Schiassi, E.; Furfaro, R.; Ganapol, B.D.; Mostacci, D. Solutions of Chandrasekhar’s basic problem in radiative
transfer via theory of functional connections. J. Quant. Spectrosc. Radiat. Transf. 2021, 259, 107384.
18. De Florio, M.; Schiassi, E.; Ganapol, B.D.; Furfaro, R. Physics-informed neural networks for rarefied-gas dynamics: Thermal creep
flow in the Bhatnagar–Gross–Krook approximation. Phys. Fluids 2021, 33, 047110.
19. Mai, T.; Mortari, D. Theory of functional connections applied to nonlinear programming under equality constraints. arXiv 2019,
arXiv:1910.04917.
20. Yassopoulos, C.; Leake, C.; Reddy, J.; Mortari, D. Analysis of Timoshenko–Ehrenfest beam problems using the theory of functional
connections. Eng. Anal. Bound. Elem. 2021, 132, 271–280.
21. Johnston, H.; Mortari, D. Least-squares solutions of boundary-value problems in hybrid systems. J. Comput. Appl. Math. 2021,
393, 113524.
22. Johnston, H.; Leake, C.; Mortari, D. Least-squares solutions of eighth-order boundary value problems using the theory of
functional connections. Mathematics 2020, 8, 397.
23. Leake, C.; Johnston, H.; Smith, L.; Mortari, D. Analytically embedding differential equation constraints into least squares support
vector machines using the theory of functional connections. Mach. Learn. Knowl. Extr. 2019, 1, 1058–1083.
24. Rao, A.V. A survey of numerical methods for optimal control. Adv. Astronaut. Sci. 2009, 135, 497–528.
25. De Almeida Junior, A.K.; Johnston, H.; Leake, C.; Mortari, D. Fast 2-impulse non-Keplerian orbit transfer using the theory of
functional connections. Eur. Phys. J. Plus 2021, 136, 1–21.
26. Schiassi, E.; D’Ambrosio, A.; Johnston, H.; De Florio, M.; Drozd, K.; Furfaro, R.; Curti, F.; Mortari, D. Physics-informed extreme
theory of functional connections applied to optimal orbit transfer. In Proceedings of the AAS/AIAA Astrodynamics Specialist
Conference, Lake Tahoe, CA, USA, 9–13 August 2020; pp. 9–13.
27. Johnston, H.; Mortari, D. Orbit propagation via the theory of functional connections. In Proceedings of the 2019 AAS/AIAA
Astrodynamics Specialist Conference, Portland, ME, USA, 11–15 August 2019.
28. De Almeida Junior, A.; Johnston, H.; Leake, C.; Mortari, D. Evaluation of transfer costs in the earth-moon system using the theory
of functional connections. In Proceedings of the AAS/AIAA Astrodynamics Specialist Conference, Lake Tahoe, CA, USA, 9–13
August 2020.

61
Math. Comput. Appl. 2021, 26, 65 17 of 17

29. Drozd, K.; Furfaro, R.; Schiassi, E.; Johnston, H.; Mortari, D. Energy-optimal trajectory problems in relative motion solved via
Theory of Functional Connections. Acta Astronaut. 2021, 182, 361–382.
30. Schiassi, E.; D’Ambrosio, A.; Johnston, H.; Furfaro, R.; Curti, F.; Mortari, D. Complete energy optimal landing on small and large
planetary bodies via theory of functional connections. In Proceedings of the AAS/AIAA Astrodynamics Specialist Conference,
Lake Tahoe, CA, USA, 9–13 August 2020; pp. 20–557.
31. Johnston, H.; Schiassi, E.; Furfaro, R.; Mortari, D. Fuel-efficient powered descent guidance on large planetary bodies via theory of
functional connections. J. Astronaut. Sci. 2020, 67, 1521–1552.
32. D’Ambrosio, A.; Schiassi, E.; Curti, F.; Furfaro, R. Pontryagin neural networks with functional interpolation for optimal intercept
problems. Mathematics 2021, 9, 996.
33. Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing 2006, 70, 489–501.
34. Lagaris, I.E.; Likas, A.; Fotiadis, D.I. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans.
Neural Networks 1998, 9, 987–1000.
35. Darania, P.; Ebadian, A. A method for the numerical solution of the integro-differential equations. Appl. Math. Comput. 2007,
188, 657–668.
36. Mishra, S.; Molinaro, R. Physics informed neural networks for simulating radiative transfer. J. Quant. Spectrosc. Radiat. Transf.
2021, 270, 107705.

62
Appendix B

De Florio et al. (2021) Radiative

Transfer

Reproduced from De Florio, M., Schiassi, E., Ganapol, B. D., & Furfaro, R. (2021). So-
lutions of Chandrasekhar’s basic problem in radiative transfer via theory of functional
connections. Journal of Quantitative Spectroscopy and Radiative Transfer, 259, 107384.
https://doi.org/10.1016/j.jqsrt.2020.107384 [8], with the permission of Elsevier.

63
 Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Contents lists available at ScienceDirect

Journal of Quantitative Spectroscopy & Radiative Transfer

journal homepage: www.elsevier.com/locate/jqsrt

Solutions of Chandrasekhar’s basic problem in radiative transfer via

theory of functional connections
Mario De Florio a, Enrico Schiassi a, Roberto Furfaro a,b,∗, Barry D. Ganapol b,
Domiziano Mostacci c
a
Department of Systems & Industrial Engineering, The University of Arizona, Tucson, AZ, USA
b
Department of Aerospace & Mechanical Engineering, The University of Arizona, Tucson, AZ, USA
c
Laboratorio di Ingegneria Nucleare di Montecuccolino, Alma Mater Studiorum - Università di Bologna, Bologna, Italy

a r t i c l e i n f o a b s t r a c t

Article history: We present a novel approach to solving Chandrasekhar’s problem in radiative transfer using the recently
Received 12 July 2020 developed Theory of Functional Connections. The method is designed to elegantly and accurately solve
Revised 8 October 2020
the Linear Boundary Value Problem from the angular discretization of the integrodifferential Boltzmann
Accepted 10 October 2020
equation for Radiative Transfer. The proposed algorithm falls under the category of numerical methods
Available online 14 October 2020
for the solution of radiative transfer equations. This new method’s accuracy is tested via comparison with
Keywords: the published benchmarks for Mie and Haze L scattering laws.
Transport theory
© 2020 Published by Elsevier Ltd.
Radiative transfer
Theory of functional connections
Mie scattering
Haze L problem
Least-squares

1. Introduction continuous angular variable. This DOM-based approach was named

the Sn (S stands for segment, since it uses line segments to ap-
Radiative transfer theory, i.e. the study of how electromagnetic proximate the μ-dependence of the angular flux) method and it
radiation propagates through a medium, is described by the Radia- has been widely employed for many radiative transfer problems
tive Transfer Equation (RTE). The latter mathematically describes [4–6]. Further methods were subsequently explored. For example,
how particles moving through a medium are affected by a variety in the CN method, developed by Benoist and Kavenoky [7], the
of physical processes, including absorption, scattering and emis- angular flux is approximated by an expansion of a complete or-
sion. In this work, we consider what is known as Chandrasekhar’s thogonal basis, such as a series of Legendre polynomials. The CN
basic problem in radiative transfer [1], i.e. the problem of comput- method in turn was further modified by Siewert and Benoist, who
ing the intensity of the photons in a finite slab illuminated by a developed the FN method [8] in 1979. FN was employed to solve
beam of light incident on one surface. the half-space albedo problem and the half-space constant-source
Historically, several methods have been proposed in the liter- problem. Initially used to solve problems in neutron transport the-
ature to solve the resulting RTE or the integro-differential Boltz- ory, it was subsequently applied in several radiative transfer ap-
mann equation for photons transport. The Discrete Ordinates plications [9–11]. In 1980, Benassi et al. [12] proposed a new PN
Method (DOM) has been an effective, purely numerical method method, in which exact particular solutions and the spherical har-
of solution available to the transport analysts for nearly 70 years. monics method [13] were combined to compute the partial heat
First proposed by Wick [2] in 1943 and later extensively developed fluxes for a general class of radiative heat-transfer problems. Siew-
by Chandrasekhar [1], DOM has undergone numerous changes and ert and Thomas [14] later in 1989, extended the PN approxima-
improvements over the years, with the goal of obtaining more ac- tion to the equation of transfer for the case of a general inho-
curate solutions. One of such changes was given by Carlson [3], mogeneous source term. Finally, at the end of the last millennium
who reformulated the DOM by introducing a discretization of the (1999), the first version of the Analytical Discrete Ordinates (ADO)
method was presented by Barichello and Siewert [15]. Conceived as
a modern version of the DOM, the ADO method generated one of
∗
Corresponding author at: Department of Systems & Industrial Engineering, The the most accurate numerical results for radiative transfer problems
University of Arizona, 1127 E James E Rogers Way, Tucson, AZ 85721, USA.
[16]. Recently, ADO has been extended to solve radiative transfer
E-mail address: robertof@email.arizona.edu (R. Furfaro).

https://doi.org/10.1016/j.jqsrt.2020.107384
0022-4073/© 2020 Published by Elsevier Ltd.

64
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

problems in vegetation [17]. However, research in generating accu- methodological application of the RTE to the simple Two-Stream
rate solutions for the basic Chandrasekhar problem and the general Approximation. In Section 3, the basic Chandrasekhar RTE is stated
RTE continued. In the last decade, Ganapol refined the Sn method and formulated. In Sections 4 and 5, the new TFC-based numerical
in an elegant way, giving life to the Converged Sn method, or CSn algorithm is detailed. Finally, numerical results are presented and
[18]. It is basically an extrapolation of the conventional, fully dis- discussed in Section 6.
cretized, Sn solution to near-zero spatial and angular discretiza-
tion. The distinctiveness of the CSn method lies in its simplicity, 2. TFC approach to solving ODE boundary value problems
which makes it accessible to those who are not proficient in solv-
ing transport equations, while achieving highly accurate solutions. 2.1. Method
A multi-problem strategy based on SN acceleration is presented by
Picca et al. [19]. For the convenience of the reader, how the TFC method works
All the above mentioned work has shown that in the last 70 to tackle generic linear and non-linear ODEs is summarized in
years, researchers have developed several methods in order to Fig. 1, and following, we give a simple example of the application
achieve accurate and efficient numerical solutions for many prob- of TFC to solving the class of BVPs. The detailed derivation can be
lems in the radiative transfer field. found in Mortari [21], Mortari et al. [22]. Consider solving a first-
In this paper, we propose and develop a novel approach that order boundary value problem such that:
will open new paths to solve RTEs. The proposed approach applies
F (τ , y, y˙ ) = 0 subject to: y(τ0 ) = y0 . (3)
the Theory of Functional Connections (TFC), recently designed and
developed by Mortari [20], to solve Chandrasekhar’s Basic Problem The CE is built from Eq. (1) by setting p1 (τ ) = 1(see ref.[21,22] for
in Radiative Transfer. TFC was born as a new framework for func- the details of this decision):
tional interpolation [20]. It has been successfully applied to solving
y ( τ ) = g( τ ) + η (4)
both linear and non-linear differential equations [21,22] to machine
error accuracy in milliseconds. Moreover the authors have success- By applying the constraint, we get:
fully applied TFC to space guidance problems like Energy and Fuel y ( τ0 ) = y 0 = g 0 + η ; ⇒ η1 = y 0 − g 0
Optimal Landing on large and small planetary bodies [23–25].
The method transforms ODEs with boundary and/or initial which is plugged into Eq. (4) to obtain at the final CE:
conditions into unconstrained optimization problems, which are y ( τ ) = g( τ ) + ( y 0 − g 0 ) (5)
solved via least-square (or iterative least-square for the non-linear
case), by embedding the conditions into a constrained expression which represents all possible functions satisfying the boundary
containing the ICs/BCs and a freely chosen function. Such an ex- value constraint. The derivative then follows:
pression can be expressed in the following general form [20]: y˙ (τ ) = g˙ (τ ). (6)

n
y(t ) = g(t ) + ηk pk(t ) = g(t ) + η p(t )
T
(1) By substituting Eqs. (5) and (6) into the Eq. (3), the original dif-
k=1
ferential equation is transformed into a new differential equation
F˜ ,which now is boundary free, and it is only a function of the in-
Eq. (1) is called constrained expression (CE). The pk (t) are n as-
dependent variable τ and the free-function g(τ ), that is:
signed linearly independent functions and the g(t) is a free func-
tion, which must be linearly independent from the pk (t) functions. F˜ (τ , g, g˙ ) = 0.
The values of the ηk parameters are calculated by imposing the set
This differential equation is now unconstrained and will always sat-
of n linear constraints in y(t). By doing so, the constraints for the
isfy the boundary-value due to the way Eq. (4) is built. In order
differential equations are satisfied for any chosen g(t). So far, it has
to solve this problem numerically, we define the function g(τ ) as
been demonstrated that TFC method in solving ODEs has the best
an expansion of some known basis with unknown constant coeffi-
performance when g(t) is selected as an expansion of basis func-
cients such that:
tion such as the Chebyshev orthogonal polynomials [21]. That is:
g( τ ) = h ( τ ) T ξ (7)
mop −1

g(t ) = hi (x )ξi = h (t )ξ
T
(2) where ξ is a mop × 1 vector of unknown constant coefficients
i=0 where mop is the number of basis functions used. In general, the
basis functions are defined on an inconsistent domain x ∈ [x0 , xf ]
where the polynomial constant coefficients ξ are the unknown to
(Chebyshev polynomials are defined on x ∈ [−1, +1],Fourier series
be computed, hi (x) is the ith orthogonal polynomial, and mop is the
is defined on x ∈ [−π , +π ],etc.). Thus, these functions must be lin-
number of basis considered. By substituting the CE in the ODE, for
early mapped to the independent variable τ . This can be done us-
linear problems as the one tackled in this paper, we will obtain a
ing the following linear transformation:
linear system of the type Aξ = bto be computed via LS to find the
solution y(t). x f − x0 τ f − τ0
x = x0 + ( τ − τ0 ) ↔ τ = τ0 + ( x − x0 ).
Here, we propose to extend the TFC framework to solve RTEs. t f − t0 x f − x0
Starting from Chandrasekhar’s Basic RTE and applying an angular
where τ ∈ [τ 0 , τ f ].
discretization as in Siewert [16], the basic integrodifferential Boltz-
Defining:
mann equation is transformed into a system of ODEs. The latter is
cast as a Linear Boundary Value Problem (BVP) that is solved via x f − x0
c= (8)
the TFC method. Constrained expressions are generated and the re- τ f − τ0
sulting free-function is expanded in the basis of Chebyshev orthog- by the derivative chain rule, the derivative of Eq. (7) is,
onal polynomials. Such expansion ultimately results in a linear sys-
d g( τ ) dh ( x )
tem that is solved via a Least-Square algorithm. Most importantly, = c ξT ,
the proposed method is shown to be elegant, concise, versatile and dτ dx
yields a straightforward numerical implementation. which defines all mappings of the free-function. Finally, the do-
The paper is organized as follows. In Section 2 a summary main τ must be discretized by M points. In this paper, we con-
of the TFC methodology is reported together with an example of sider g(τ ) as an expansion of Chebyshev polynomials. By defining

2
65
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Fig. 1. Schematic of the general TFC framework to solve linear and non-linear ODEs with one constraint in one point.

the free function in this way and then discretizing the domain of where h are the Chebyshev polynomials, and h0 and hf are the
the differential equations, F˜ becomes: Chebyshev polynomials computed at τ = 0and τ = τ0 ,respectively.
The derivatives of the CEs are:
F˜ (ξ ) = 0,
dy1
= c h  ξ1
T
which is now a function (linear or nonlinear) of only the unknown (15)
dτ
coefficients ξ . Eq. () is now an unconstrained optimization problem
that can be solved via different optimization schemes. In this pa-
dy2
= c h  ξ2
T
per, as we deal with a linear problem, we use a linear LS to solve (16)
for the unknown coefficients ξ . dτ
where h are the derivatives of h with respect to the variable x,
2.2. Example: radiative transfer problem with two-stream and c is the mapping coefficient from τ to x (and vice-versa) as
approximation defined in Eq. (8).
By plugging the CEs and their derivatives into the system of
This example shows how to apply the TFC method to solve a ODEs, we get the following:
ω T 
simple system of ODEs. The system is the following: 
dy ω ω τ
μ c h ξ1 + (h − h0 )T ξ1 = (h − h0 )T ξ1 + h − h f ξ2
μ 1 + y1 (τ ) = [y1 (τ ) + y2 (τ )] + S0 e− μ0 (9) 2
dτ 2 2 ω − μτ0
+ S0 e (17)
2
dy2 ω ω −τ
−μ + y2 (τ ) = [y2 (τ ) + y1 (τ )] + S0 e μ0 , (10)
dτ
ω  
2 2  T T
subject to, −μ c h ξ2 + h − h f ξ2 = h − hf ξ2 + (h − h0 )T ξ1
2
y1 ( 0 ) = 0 (11) ω − μτ
+ S0 e 0 , (18)
2
The losses are:
y 2 ( τ0 ) = 0 (12)
ω  T 
that represents an isotropic scattering radiative transfer equation L1 = μ c h ξ1 + (h − h0 )T ξ1 − (h − h0 )T ξ1 + h − h f ξ2
2
with two-stream approximation [26], arising from Chandrasekhar ω − μτ
[1], explained in details in the next section. The solutions of this − S0 e 0 (19)
2
system of ODEs represent the trend of scattered photons (forward
and backward) through a mono-dimensional slab of thickness τ 0 .
 T ω  T 
For this example,the following parameters are used: L2 = −μ c h ξ2 + h − h f ξ2 − h − hf ξ2 + (h − h0 )T ξ1
2
• μ = 0.5is the cosine of the polar angle ω − μτ
• μ0 = 0.8is the cosine of incident beam angle − S0 e 0 (20)
2
• τ0 = 1is the optical thickness
As the problem is linear, by imposing the losses to be equal zero
• ω = 0.9is the single scattering albedo
and re-arranging the terms we get the following system of linear
• S0 = 1is the source intensity
algebraic equations:
The detailed explanation of these parameters physically repre-  ω T ω T 
sent will be given in the next section. The unknowns are y1 (τ ) and μ c h  + ( h − h 0 ) − ( h − h 0 ) ξ1 − h − hf ξ2
y2 (τ ) that represent the forward and backward scattered photons 2 2
fluxes respectively. ω − μ0 τ
= S0 e (21)
The first step of the TFC framework is to build the constrained 2
expressions (CE) for the unknowns. According to previous explana-
tion, the CEs are:  ω     ω T
− (h − h0 )T ξ1 + −μ c h + h − h f − h − hf ξ2
y1 ( τ ) = ( h − h0 ) ξ1
T
(13) 2 2
ω − μτ
= S0 e (22)
 T
0
2
y2 ( τ ) = h − h f ξ2 (14)

3
66
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 1
ADO and TFC forward scattered flux y1 . The latter is obtained with M =
100and mop = 40. Mis the number of discretization points on τ and
mop is the number of basis used.

Slab Points ADO TFC

0 1.110223024625157 × 10−16 0
0.1 0.121476763490043 0.121476763490043
0.2 0.217584323416591 0.217584323416591
0.3 0.291745312795815 0.291745312795815
0.4 0.346937565918151 0.346937565918151
0.5 0.385747980285517 0.385747980285517
0.6 0.410419885421222 0.410419885421222
0.7 0.422894687247150 0.422894687247150
0.8 0.424848465764781 0.424848465764781
0.9 0.417724123526887 0.417724123526887
1 0.402759611584474 0.402759611584474

Table 2 Fig. 2. Geometry of the 1D slab for the considered radiative transfer problem.
ADO and TFC backward scattered flux y2 . The latter is obtained with
M = 100and mop = 40.

Slab points ADO TFC where the β l are the Legendre coefficients in the Lthorder expan-
0 0.504726885720671 0.504726885720671
sion of scattering law.
0.1 0.467993882643466 0.467993882643466 The problem is subject to the following constraints/boundary
0.2 0.427166877800790 0.427166877800790 conditions:
0.3 0.382783602872154 0.382783602872154 
0.4 0.335285178981115 0.335285178981115 I ( 0 , μ , φ ) = S 0 π δ ( μ − μ 0 ) δ ( φ − φ0 ) for μ>0
(27)
0.5 0.285027561759344 0.285027561759344 I ( τ0 , μ, φ ) = 0 for μ<0
0.6 0.232291508206183 0.232291508206183
0.7 0.177291238875862 0.177291238875862
where S0 π represents the flux carried by a source beam across a
0.8 0.120181947999640 0.120181947999640
0.9 0.061066295681211 0.061066295681211 plane normal to the direction of incidence, and μ0 and φ 0 deter-
1 −5.551115123125783 × 10−17 0 mine the inclination of the source beam.
The solution is defined as the sum of collided (or diffuse) term
and uncollided term [16]:
In matrix form the system becomes:
  T   τ 
μc h T + (h − h0 )T − ω2 (h − h0 )T − ω2 h − h f ξ1 ω S e− μ0
 T  T ξ2
= 2 0 τ(23)
ω S e− μ0
− ω2 (h − h0 ) −μc h + h − h f ω
T T
− 2
h − hf 2 0

A ξ b

As the problem is linear, we can solve for ξ using a least-squares.

That is: I ( τ , μ, φ ) = I ∗ ( τ , μ, φ ) + I o ( τ , μ, φ ) , (28)
 −1
ξ = AT A AT b (24) ∗
where the superscript refers to the collided beam, and the su-
Once ξ are computed, the final solutions are built by plugging ξ perscript o refers to the uncollided beam. The problem is therefore
into the CEs (13), as shown in Fig. 1. split into two parts. The uncollided term solves the following first
The results are reported in Tables 1 and 2, where they are com- order PDE:
pared to the results computed according to the ADO method. As ∂ o
μ I ( τ , μ, φ ) + I o ( τ , μ, φ ) = 0 (29)
we can see, the boundary conditions are analytically satisfied. ∂τ
3. Radiative transfer equation: problem formulation subject to:

I o ( 0 , μ , φ ) = S 0 π δ ( μ − μ 0 ) δ ( φ − φ0 ) for μ>0
As we previously mentioned, we consider Chandrasekhar’s basic (30)
I o ( 0 , μ, φ ) = 0 for μ<0
problem for Radiative Transfer for an incoming beam at x = 0,as
shown in Fig. 2, following the notation used in Siewert [16], Be-
where μ ∈ (0, 1].
nassi et al. [27] the radiative transfer equation is written as:
The problem (29) and (30) has the following analytical solu-
∂ tions:
μ I ( τ , μ, φ ) + I ( τ , μ, φ )
∂τ 
  I o ( τ , μ , φ ) = S 0 π δ ( μ − μ 0 ) δ ( φ − φ0 ) e − τ / μ μ>0
ω 1 2π for
= p(cos )I (τ , μ , φ  ) dφ  dμ (25) I o ( τ , μ, φ ) = 0 for μ<0
4π −1 0
where is the scattering angle, μ ∈ [−1, 1]is the cosine of the po- (31)
lar angle, τ ∈ [0, τ 0 ] is the optical variable, τ 0 is the optical thick-
Plugging into the Eq. (28), we obtain the following:
ness, φ ∈ [0, 2π ] is the azimuthal angle, and ω ∈ [0, 1] is the single
scattering albedo. According to [28], we use the addition theorem I ( τ , μ , φ ) = I ∗ ( τ , μ , φ ) + S 0 π δ ( μ − μ 0 ) δ ( φ − φ0 ) e − τ / μ (32)
to express the phase function in terms of Legendre polynomials:
The relation of the phase function and the scattering angle can

L
p(cos )= βl Pl (cos ) (26) be expressed using the Addition Theorem of the Spherical Harmonics
[28]:
l=0

4
67
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384


L 
L ∂
p(cos )= (2 − δ0,m ) βl Plm (μ )Plm (μ ) cos[m(φ  − φ )] −μi I (τ , −μi ) + Im (τ , −μi ) =
∂τ m
m=0 l=m
ω L N
(33) = βl Plm (−μi ) wk Plm (−μk )[(−1 )l−m Im (τ , μk )
2
l=m k=1
where the following expression is used to denote the normalized
ω 
L
function [29,30]: + Im (τ , −μk )] + e − τ /μ0 βl Plm (μ0 )Plm (−μi ) (42)
 2
l=m
( l − m )! d m
Plm (μ ) = (1 − μ2 )m/2 m Pl (μ ) (34) where the Gauss-Legendre quadrature rule is used to evaluate the
( l + m )! dμ
integral in the range [0,1].
Thus, we can express the diffuse component as a Fourier expansion Eqs. (41) and (42) with the constraints (39) are a set of N linear
with L terms. That is: first-order ODEs that we will solve via TFC.

1
L
4. TFC formulation of the problem
I ∗ ( τ , μ, φ ) = (2 − δ0,m )Im (τ , μ ) cos[m(φ − φ0 )] (35)
2
m=0 In our case, we assume to have one constraint in one point;
The mthFourier component satisfies the following linear integro - hence, according to (1), y(τ ) can be expressed as following:
1storder PDE: y ( τ ) = g ( τ ) + η p( τ ) (43)
∂ We set p(τ ) = 1,as suggested by Mortari [21], and the function
μ Im (τ , μ ) + Im (τ , μ )
∂t g(τ ) as an expansion of Chebyshev polynomials as follows:
 1
ω L
g( τ ) = h T ( τ ) ξ (44)
= βl Plm (μ ) Plm (μ )Im (τ , μ )dμ + Q m (τ , μ ), (36)
2 −1 Since Chebyshev Polynomials are used as basis functions, we need
l=m
to define a new variable x that ranges in [−1, 1]. The new xvariable
where Q is the inhomogeneous source term [16]: is defined as x = cτ − 1,where c is the mapping coefficient:
 x f − x0
ω L
c=
Q m (τ , μ ) = e − τ /μ0 βl Plm (μ0 )Plm (μ ) (37) τ f − τ0
2
l=m
Considering the new independent variable, we have:
Thus, the problem to solve is the following: 
Im (τ , μ ) = Im (x, μ )
∂ ∂ ∂ (45)
μ Im (τ , μ ) + Im (τ , μ ) I (τ , μ ) = c Im (x, μ )
∂t ∂τ m ∂x
 1 To simplify the notation, for the remainder of this manuscript, we
ω L
= βl Plm (μ ) Plm (μ )Im (τ , μ )dμ give for granted the x and μ dependency and write the terms for
2 −1 + −
l=m the forward and backward flux as Im and Im ,respectively. Thus, our
 problem becomes:
ω L
+ e − τ /μ0 βl Plm (μ0 )Plm (μ ) (38)
2 ∂ + ω L N
l=m c μi +
Im + Im = βl Plm (μi ) wk Plm (μk )[Im+ + (−1 )l−m Im− ]
∂x 2
l=m k=1
Subject to:
 ω 
L

Im (0, μ ) = 0 for μ>0 + e − τ /μ0 βl Plm (μ0 )Plm (μi ) (46)

(39) 2
Im (τ0 , μ ) = 0 for μ<0 l=m

Via discretizing the angle μ into N points in the interval (0,1] ac- ∂ − ω L N
−cμi −
Im + Im = βl Plm (−μi ) wk Plm (−μk )
cording to the Gauss-Legendre quadrature scheme: ∂x 2
l=m k=1
μ −→ μ = {μi }Ni=1 ω 
L
× [(−1 )l−m Im
+ −
+ Im ]+ e − τ /μ0 βl Plm (μ0 )Plm (−μi )
it follows that: 2
l=m

Im (τ , μ ) −→ Im (τ ) = {Im (τ , μi )}Ni=1 (40) (47)

Subject to:
Following the notation used by Siewert [16], suppressing some of +
the m indexes, and splitting the problem into two equations, for Im (0 ) = I0+ = 0
(48)
positive and negative values of μ, we get the following set of equa-
−
Im (τ0 ) = I−f = 0
tions:
Our constrained expressions, for forward and backward terms, are:
∂
μi Im (τ , μi ) + Im (τ , μi ) ±
(x ) = g± (x ) + η±
∂τ Im (49)
ωL N
Therefore, by using the boundary conditions to calculate η+ and
= βl Plm (μi ) wk Plm (μk )[Im (τ , μk ) η− ,we obtain:
2
l=m k=1
0 +η
I0+ = 0 = g+ η+ = −g+0
+
−→
ω 
L
− τ /μ0
+ (−1 ) l−m
Im (τ , −μk )] + e β
l Pl
m
(μ )
0 Pl
m
( μi ) (41)
2
l=m I−f = 0 = g−f + η− −→ η− = −g−f

5
68
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Fig. 3. Schematic of the theory of functional connections algorithm to solve Chandrasekhar’s basic problem in radiative transfer.

Table 3
Mie scattering—The Fourier component I∗ (τ , μ) for m = 0,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 4.76807e−2 4.41912e−2 4.06467e−2 3.37099e−2 1.58572e−2 5.45297e−3 0
−0.9 6.45644e−2 6.03742e−2 5.60139e−2 4.72899e−2 2.38121e−2 9.01083e−3 0
−0.8 8.45877e−2 7.96763e−2 7.44425e−2 6.37540e−2 3.38287e−2 1.36919e−2 0
−0.7 1.08350e−1 1.02719e−1 9.65678e−2 8.37488e−2 4.64918e−2 1.98845e−2 0
−0.6 1.36504e−1 1.30204e−1 1.23128e−1 1.08063e−1 6.26048e−2 2.81721e−2 0
−0.5 1.69677e−1 1.62852e−1 1.54919e−1 1.37618e−1 8.32921e−2 3.94792e−2 0
−0.4 2.08230e−1 2.01201e−1 1.92626e−1 1.73358e−1 1.10124e−1 5.53631e−2 0
−0.3 2.51677e−1 2.45065e−1 2.36310e−1 2.15806e−1 1.45144e−1 7.86298e−2 0
−0.2 2.97523e−1 2.92401e−1 2.84241e−1 2.63800e−1 1.90035e−1 1.14557e−1 0
−0.1 3.40126e−1 3.38476e−1 3.31984e−1 3.12888e−1 2.41217e−1 1.70697e−1 0
−0.0 3.59379e−1 3.74485e−1 3.73800e−1 3.59347e−1 2.88258e−1 2.25623e−1 0
0.0 0 3.74485e−1 3.73800e−1 3.59347e−1 2.88258e−1 2.25623e−1 1.51520e−1
0.1 0 1.55620e−1 2.51427e−1 3.38864e−1 3.30703e−1 2.69855e−1 2.03073e−1
0.2 0 9.24370e−2 1.65080e−1 2.61887e−1 3.35517e−1 3.01553e−1 2.44242e−1
0.3 0 6.70726e−2 1.24144e−1 2.10383e−1 3.14805e−1 3.09713e−1 2.70302e−1
0.4 0 5.30170e−2 9.98939e−2 1.75304e−1 2.88183e−1 3.02939e−1 2.81066e−1
0.5 0 4.37182e−2 8.32383e−2 1.49251e−1 2.61304e−1 2.88464e−1 2.80822e−1
0.6 0 3.67864e−2 7.05194e−2 1.28323e−1 2.35259e−1 2.69793e−1 2.73029e−1
0.7 0 3.11411e−2 5.99908e−2 1.10387e−1 2.09939e−1 2.48476e−1 2.59822e−1
0.8 0 2.62225e−2 5.07166e−2 9.42001e−2 1.84982e−1 2.25179e−1 2.42453e−1
0.9 0 2.17133e−2 4.21538e−2 7.90039e−2 1.60034e−1 2.00187e−1 2.21667e−1
1.0 0 1.74231e−2 3.39716e−2 6.43193e−2 1.34819e−1 1.73627e−1 1.97932e−1

and then: where ξ + and ξ − are the unknowns to be computed via LS. Plugging
(50) into our DEs, we get:
+
Im (x ) = g (x ) −
+
g+
0
cμi hT ξi+ + (h − h0 )T ξi+

−
(x ) = g− (x ) − g−f
ω
L 
N
Im = βl Plm (μi ) wk Plm (μk )[(h − h0 )T ξk+
2
l=m k=1
Finally, we get the CEs for the fluxes in the following form: 
ω L
+ (−1 ) l−m
(h − h f )T ξk− ] + e − τ /μ0 βl Plm (μ0 )Plm (μi ) (51)
+
Im = (h −
T
hT0 )ξ +
; −
Im = (h −
T
hTf )ξ ,
−
(50) 2
l=m

6
69
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 4
Mie scattering—The Fourier component I∗ (τ , μ) for m = 1,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 3.60181e−2 3.36746e−2 3.12064e−2 2.62787e−2 1.32903e−2 5.19802e−3 0
−0.8 6.16596e−2 5.78589e−2 5.38160e−2 4.56764e−2 2.38338e−2 9.72915e−3 0
−0.7 9.10359e−2 8.57207e−2 8.00113e−2 6.84223e−2 3.67820e−2 1.56203e−2 0
−0.6 1.26254e−1 1.19295e−1 1.11742e−1 9.62823e−2 5.33398e−2 2.35586e−2 0
−0.5 1.68911e−1 1.60192e−1 1.50609e−1 1.30818e−1 7.48561e−2 3.44878e−2 0
−0.4 2.20425e−1 2.09925e−1 1.98188e−1 1.73691e−1 1.03142e−1 4.99460e−2 0
−0.3 2.81715e−1 2.69635e−1 2.55778e−1 2.26467e−1 1.40633e−1 7.26822e−2 0
−0.2 3.52226e−1 3.39192e−1 3.23503e−1 2.89666e−1 1.89868e−1 1.07900e−1 0
−0.1 4.28566e−1 4.16266e−1 3.99406e−1 3.61374e−1 2.49443e−1 1.63513e−1 0
−0.0 4.90592e−1 4.93468e−1 4.79629e−1 4.40175e−1 3.12405e−1 2.22786e−1 0
0.0 0 4.93468e−1 4.79629e−1 4.40175e−1 3.12405e−1 2.22786e−1 1.40086e−1
0.1 0 2.12101e−1 3.36358e−1 4.36674e−1 3.78485e−1 2.79507e−1 1.92369e−1
0.2 0 1.27652e−1 2.24094e−1 3.44034e−1 3.98038e−1 3.26312e−1 2.41187e−1
0.3 0 9.31423e−2 1.69547e−1 2.78455e−1 3.79153e−1 3.42883e−1 2.74467e−1
0.4 0 7.34351e−2 1.36109e−1 2.31599e−1 3.47468e−1 3.37079e−1 2.87903e−1
0.5 0 5.97940e−2 1.11999e−1 1.94715e−1 3.11216e−1 3.17429e−1 2.84863e−1
0.6 0 4.89801e−2 9.23641e−2 1.62887e−1 2.72157e−1 2.88095e−1 2.68557e−1
0.7 0 3.94407e−2 7.47181e−2 1.33107e−1 2.29842e−1 2.50372e−1 2.40503e−1
0.8 0 3.01812e−2 5.73658e−2 1.02969e−1 1.82413e−1 2.03320e−1 2.00130e−1
0.9 0 2.00568e−2 3.82174e−2 6.90058e−2 1.24817e−1 1.41801e−1 1.42461e−1
1.0 0 0 0 0 0 0 0

Table 5
Mie scattering—The Fourier component I∗ (τ , μ) for m = 2,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 6.13369e−3 5.63258e−3 5.14146e−3 4.22728e−3 2.08109e−3 8.51138e−4 0
−0.8 1.40473e−2 1.29199e−2 1.18125e−2 9.74639e−3 4.86758e−3 2.02908e−3 0
−0.7 2.42374e−2 2.23272e−2 2.04470e−2 1.69330e−2 8.59010e−3 3.65704e−3 0
−0.6 3.72977e−2 3.44146e−2 3.15724e−2 2.62520e−2 1.35554e−2 5.91266e−3 0
−0.5 5.39364e−2 4.98590e−2 4.58321e−2 3.82838e−2 2.01851e−2 9.06805e−3 0
−0.4 7.49853e−2 6.94667e−2 6.40030e−2 5.37486e−2 2.90737e−2 1.35709e−2 0
−0.3 1.01366e−1 9.41479e−2 8.69734e−2 7.34927e−2 4.10537e−2 2.02184e−2 0
−0.2 1.33969e−1 1.24806e−1 1.15624e−1 9.83434e−2 5.71259e−2 3.05232e−2 0
−0.1 1.73539e−1 1.62321e−1 1.50803e−1 1.28934e−1 7.75407e−2 4.68764e−2 0
−0.0 2.17610e−1 2.07223e−1 1.93748e−1 1.66780e−1 1.01645e−1 6.54823e−2 0
0.0 0 2.07223e−1 1.93748e−1 1.66780e−1 1.01645e−1 6.54823e−2 3.94802e−2
0.1 0 9.13850e−2 1.41198e−1 1.74338e−1 1.30506e−1 8.62088e−2 5.45248e−2
0.2 0 5.47834e−2 9.40008e−2 1.38386e−1 1.41997e−1 1.05409e−1 7.12370e−2
0.3 0 3.94265e−2 7.02551e−2 1.11034e−1 1.35966e−1 1.12775e−1 8.32078e−2
0.4 0 3.03255e−2 5.50820e−2 9.03843e−2 1.22905e−1 1.10240e−1 8.74206e−2
0.5 0 2.37574e−2 4.36468e−2 7.32857e−2 1.06676e−1 1.01104e−1 8.46547e−2
0.6 0 1.83638e−2 3.39907e−2 5.79582e−2 8.84652e−2 8.72842e−2 7.61525e−2
0.7 0 1.35348e−2 2.51835e−2 4.34139e−2 6.86137e−2 6.97897e−2 6.28579e−2
0.8 0 8.96183e−3 1.67383e−2 2.90913e−2 4.72161e−2 4.91804e−2 4.54269e−2
0.9 0 4.47932e−3 8.39018e−3 1.46739e−2 2.43199e−2 2.58191e−2 2.43409e−2
1.0 0 0 0 0 0 0 0

ω
L 
N
(−cμi h + h − h f )T ξi− − βl Plm (−μi ) wk Plm (−μk )
−cμ ih ξ + (h −
T −
i
) ξ
h f T i− 2
l=m k=1

ω
L 
N × [(−1 )l−m (h − h0 )T ξk+ + (h − h f )T ξk− ]
= βl Plm (−μi ) wk Plm (−μk )[(−1 )l−m (h − h0 )T ξk+
2 ω 
L
k=1
e − τ /μ0 βl Plm (μ0 )Plm (−μi )
l=m
= (54)
 2
ω − τ /μ0
L
l=m
+ (h − ) ξ
h f T k− ] + e βl Pl
m
(μ )
0 Pl
m
(−μi ) (52)
2
l=m
For the sake of simplicity, we write the inhomogeneous term as:
Reorganizing all the terms we get:

ω
L 
N
ω 
L
(cμi h + h − h0 )T ξi+ − βl Plm (μi ) wk Plm (μk ) b+ = e − τ /μ0 βl Plm (μ0 )Plm (μi )
2 i 2
l=m k=1 l=m

× [(h − h0 )T ξk+ + (−1 )l−m (h − h f )T ξk− ] ω 

L
b−
i = e − τ /μ0 βl Plm (μ0 )Plm (−μi )
ω 
L 2
− τ /μ0
= e β
l Pl
m
(μ ) 0 Pl
m
( μi ) (53) l=m
2
l=m
So, the two problems become:

7
70
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 6
Mie scattering—The Fourier component I∗ (τ , μ) for m = 3,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 4.71882e−4 4.27922e−4 3.86504e−4 3.12323e−4 1.49303e−4 6.12974e−5 0
−0.8 1.63700e−3 1.48492e−3 1.34202e−3 1.e-08650e-3 5.24875e−4 2.18940e−4 0
−0.7 3.62794e−3 3.29256e−3 2.97807e−3 2.41643e−3 1.18117e−3 5.01304e−4 0
−0.6 6.65125e−3 6.04087e−3 5.46943e−3 4.44992e−3 2.20550e−3 9.55263e−4 0
−0.5 1.09558e−2 9.96046e−3 9.02968e−3 7.37068e−3 3.71563e−3 1.65092e−3 0
−0.4 1.68361e−2 1.53260e−2 1.39154e−2 1.14039e−2 5.87321e−3 2.70031e−3 0
−0.3 2.46361e−2 2.24603e−2 2.04291e−2 1.68191e−2 8.90196e−3 4.29905e−3 0
−0.2 3.47536e−2 3.17326e−2 2.89122e−2 2.39119e−2 1.30799e−2 6.81521e−3 0
−0.1 4.77051e−2 4.36203e−2 3.97892e−2 3.29968e−2 1.85468e−2 1.08405e−2 0
−0.0 6.40589e−2 5.89247e−2 5.38601e−2 4.47620e−2 2.52999e−2 1.55900e−2 0
0.0 0 5.89247e−2 5.38601e−2 4.47620e−2 2.52999e−2 1.55900e−2 9.40491e−3
0.1 0 2.65947e−2 4.04696e−2 4.85984e−2 3.37870e−2 2.12142e−2 1.30156e−2
0.2 0 1.59125e−2 2.69468e−2 3.87779e−2 3.75052e−2 2.66263e−2 1.73623e−2
0.3 0 1.12859e−2 1.98696e−2 3.07668e−2 3.58014e−2 2.85953e−2 2.04272e−2
0.4 0 8.43804e−3 1.51551e−2 2.44012e−2 3.16829e−2 2.74886e−2 2.11807e−2
0.5 0 6.31959e−3 1.14883e−2 1.89493e−2 2.64242e−2 2.42961e−2 1.98181e−2
0.6 0 4.56717e−3 8.36998e−3 1.40335e−2 2.05701e−2 1.97297e−2 1.67997e−2
0.7 0 3.04358e−3 5.61002e−3 9.51751e−3 1.44725e−2 1.43314e−2 1.26141e−2
0.8 0 1.71669e−3 3.17792e−3 5.43950e−3 8.50729e−3 8.63650e−3 7.80304e−3
0.9 0 6.32858e−4 1.17545e−3 2.02593e−3 3.23965e−3 3.35492e−3 3.09582e−3
1.0 0 0 0 0 0 0 0

Table 7
Mie scattering—The Fourier component I∗ (τ , μ) for m = 4,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 6.69258e−5 6.02239e−5 5.40838e−5 4.33436e−5 2.05304e−5 0.851494e−6 0
−0.8 3.02494e−4 2.72377e−4 2.44794e−4 1.96548e−4 9.39014e−5 3.93973e−5 0
−0.7 7.61720e−4 6.86418e−4 6.17472e−4 4.96877e−4 2.39851e−4 1.02062e−4 0
−0.6 1.50358e−3 1.35622e−3 1.22134e−3 9.85431e−4 4.81766e−4 2.08696e−4 0
−0.5 2.59147e−3 2.34013e−3 2.11014e−3 1.70803e−3 8.48455e−4 3.76292e−4 0
−0.4 4.09363e−3 3.70143e−3 3.34275e−3 2.71612e−3 1.37699e−3 6.30869e−4 0
−0.3 6.08324e−3 5.50815e−3 4.98269e−3 4.06626e−3 2.11599e−3 1.01667e−3 0
−0.2 8.63901e−3 7.83189e−3 7.09526e−3 5.81430e−3 3.12207e−3 1.61559e−3 0
−0.1 1.18659e−2 1.07645e−2 9.75898e−3 8.01357e−3 4.41270e−3 2.55497e−3 0
−0.0 1.59510e−2 1.44948e−2 1.31484e−2 1.08036e−2 5.96834e−3 3.63154e−3 0
0.0 0 1.44948e−2 1.31484e−2 1.08036e−2 5.96834e−3 3.63154e−3 2.19670e−3
0.1 0 6.42599e−3 9.72886e−3 1.15783e−2 7.86933e−3 4.86919e−3 2.96365e−3
0.2 0 3.72830e−3 6.28643e−3 8.98115e−3 8.53292e−3 5.98094e−3 3.86184e−3
0.3 0 2.53962e−3 4.45371e−3 6.85213e−3 7.85349e−3 6.20648e−3 4.39482e−3
0.4 0 1.80267e−3 3.22602e−3 5.16374e−3 6.61455e−3 5.68628e−3 4.34786e−3
0.5 0 1.26267e−3 2.28772e−3 3.75288e−3 5.16865e−3 4.71326e−3 3.81819e−3
0.6 0 8.35644e−4 1.52664e−3 2.54654e−3 3.68968e−3 3.51216e−3 2.97182e−3
0.7 0 4.93531e−4 9.07021e−4 1.53136e−3 2.30332e−3 2.26473e−3 1.98170e−3
0.8 0 2.32528e−4 4.29264e−4 7.31400e−4 1.13210e−3 1.14161e−3 1.02575e−3
0.9 0 6.20029e−5 1.14862e−4 1.97113e−4 3.12097e−4 3.21141e−4 2.94782e−4
1.0 0 0 0 0 0 0 0

ω
L 
N  i = ( c μi h  + h − h 0 ) T
(cμi h + h − h0 )T ξi+ − βl Plm (μi ) wk Plm (μk )
2
l=m k=1

[ (h − ) ξ + (−1 )
h0 T k+ l−m
(h − h f )T ξk− ] = b+i (55) ω 
L
♣ki = − wk ( h − h f )T βl Plm (μi )Pl (μk )(−1 )l−m
2
l=m

ω
L 
N
(−cμi h + h − h f )T ξi− − βl Plm (−μi ) wk Plm (−μk )

2 ω L
l=m k=1 ♠ki = − wk ( h − h0 )T βl Plm (−μi )Pl (−μk )(−1 )l−m
2
[(−1 )l−m (h − h0 )T ξk+ + (h − h f )T ξk− ] = b−
i (56) l=m

Expanding the summations, we get the following matrix form:

ω 
L
   ♦ki = − wk ( h − h0 )T βl Plm (μi )Pl (μk )
i + ♦ki ♣ki ξi+ b+ 2
= i (57) l=m
♠ki i + ♥ki ξi− b−
i

where: 
ω L

i = (−cμi h + h − h f )T ♥ki = − wk ( h − h f )T βl Plm (−μi )Pl (−μk )

2
l=m

8
71
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 8
Mie scattering—The Fourier component I∗ (τ , μ) for m = 5,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 6.28063e−6 5.64038e−6 5.05780e−6 4.04469e−6 1.91071e−6 7.93886e−7 0
−0.8 3.73124e−5 3.35331e−5 3.00940e−5 2.41116e−5 1.14857e−5 4.82442e−6 0
−0.7 1.07458e−4 9.66557e−5 8.68253e−5 6.97210e−5 3.35504e−5 1.42849e−5 0
−0.6 2.29470e−4 2.06608e−4 1.85802e−4 1.49599e−4 7.28951e−5 3.15824e−5 0
−0.5 4.15167e−4 3.74235e−4 3.36993e−4 2.72200e−4 1.34741e−4 5.97452e−5 0
−0.4 6.75372e−4 6.09580e−4 5.49751e−4 4.45739e−4 2.25137e−4 1.03090e−4 0
−0.3 1.01957e−3 9.21504e−4 8.32420e−4 6.77822e−4 3.51320e−4 1.68651e−4 0
−0.2 1.45558e−3 1.31706e−3 1.19142e−3 9.74059e−4 5.20758e−4 2.69137e−4 0
−0.1 1.99273e−3 1.80388e−3 1.63272e−3 1.33734e−3 7.32823e−4 4.23514e−4 0
−0.0 2.65403e−3 2.40328e−3 2.17551e−3 1.78221e−3 9.79104e−4 5.94123e−4 0
0.0 0 2.40328e−3 2.17551e−3 1.78221e−3 9.79104e−4 5.94123e−4 3.60119e−4
0.1 0 1.04360e−3 1.57774e−3 1.87309e−3 1.26582e−3 7.80603e−4 4.74525e−4
0.2 0 5.87087e−4 9.88700e−4 1.40973e−3 1.33332e−3 9.31756e−4 6.00443e−4
0.3 0 3.83898e−4 6.72490e−4 1.03283e−3 1.17920e−3 9.29569e−4 6.57017e−4
0.4 0 2.58469e−4 4.62075e−4 7.38429e−4 9.42637e−4 8.08592e−4 6.17257e−4
0.5 0 1.69091e−4 3.06064e−4 5.01326e−4 6.88265e−4 6.26405e−4 5.06701e−4
0.6 0 1.02291e−4 1.86706e−4 3.10999e−4 4.49276e−4 4.26898e−4 3.60733e−4
0.7 0 5.34205e−5 9.80933e−5 1.65394e−4 2.48078e−4 2.43517e−4 2.12817e−4
0.8 0 2.09675e−5 3.86763e−5 6.58150e−5 1.01604e−4 1.02297e−4 9.18069e−5
0.9 0 4.03113e−6 7.46208e−6 1.27901e−5 2.02003e−5 2.07549e−5 1.90300e−5
1.0 0 0 0 0 0 0 0

Table 9
Mie scattering—-The Fourier component I∗ (τ , μ) for m = 6,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 3.64612e−7 3.27338e−7 2.93459e−7 2.34597e−7 1.10774e−7 4.60360e−8 0
−0.8 2.90480e−6 2.60976e−6 2.34156e−6 1.87544e−6 8.92958e−7 3.75132e−7 0
−0.7 9.72738e−6 8.74687e−6 7.85546e−6 6.30582e−6 3.03290e−6 1.29146e−6 0
−0.6 2.27862e−5 2.05098e−5 1.84403e−5 1.48421e−6 7.22835e−6 3.13189e−6 0
−0.5 4.37887e−5 3.94595e−5 3.55246e−5 2.86845e−5 1.41912e−5 6.29250e−6 0
−0.4 7.40943e−5 6.68560e−5 6.02802e−5 4.88581e−5 2.46633e−5 1.12928e−5 0
−0.3 1.14597e−4 1.03542e−4 9.35097e−5 7.61152e−5 3.94263e−5 1.89249e−5 0
−0.2 1.65614e−4 1.49803e−4 1.35478e−4 1.10719e−4 5.91524e−5 3.05665e−5 0
−0.1 2.27191e−4 2.05580e−4 1.86022e−4 1.52303e−4 8.33921e−5 4.81822e−5 0
−0.0 3.00490e−4 2.71920e−4 2.46056e−4 2.01467e−4 1.10580e−4 6.70733e−5 0
0.0 0 2.71920e−4 2.46056e−4 2.01467e−4 1.10580e−4 6.70733e−5 4.06782e−5
0.1 0 1.16112e−4 1.75494e−4 2.08255e−4 1.40600e−4 8.66594e−5 5.26746e−5
0.2 0 6.35970e−5 1.07078e−4 1.52621e−4 1.44236e−4 1.00747e−4 6.49064e−5
0.3 0 4.00550e−5 7.01511e−5 1.07705e−4 1.22887e−4 9.68330e−5 6.84233e−5
0.4 0 2.56452e−5 4.58379e−5 7.32305e−5 9.34258e−5 8.01121e−5 6.11407e−5
0.5 0 1.56977e−5 2.84085e−5 4.65197e−5 6.38310e−5 5.80755e−5 4.69672e−5
0.6 0 8.68979e−6 1.58582e−5 2.64083e−5 3.81303e−5 3.62204e−5 3.06005e−5
0.7 0 4.01435e−6 7.37013e−6 1.24236e−5 1.86253e−5 1.82779e−5 1.59706e−5
0.8 0 1.31217e−6 2.42003e−6 4.11715e−6 6.35302e−6 6.39473e−6 5.73797e−6
0.9 0 1.81710e−7 3.36316e−7 5.76320e−7 9.09821e−7 9.34571e−7 8.56760e−7
1.0 0 0 0 0 0 0 0

Thus, the problem reduce to the following linear system of al-

gebraic equations:

which is a problem of the type:

Aξ = B (58)

9
72
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 10
Mie scattering—The Fourier component I∗ (τ , μ) for m = 7,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 1.27031e−8 1.14041e−8 1.02235e−8 8.17255e−9 3.85880e−9 1.60370e−9 0
−0.8 1.42973e−7 1.28447e−7 1.15244e−7 9.22992e−8 4.39445e−8 1.84614e−8 0
−0.7 5.79668e−7 5.21220e−7 4.68089e−7 3.75735e−7 1.80707e−7 7.69491e−8 0
−0.6 1.53649e−6 1.38294e−6 1.24336e−6 1.00071e−6 4.87335e−7 2.11154e−7 0
−0.5 3.21082e−6 2.89328e−6 2.60469e−6 2.10309e−6 1.04041e−6 4.61327e−7 0
−0.4 5.75018e−6 5.18825e−6 4.67782e−6 3.79130e−6 1.91370e−6 8.76244e−7 0
−0.3 9.22443e−6 8.33427e−6 7.52656e−6 6.12622e−6 3.17306e−6 1.52308e−6 0
−0.2 1.36060e−5 1.23065e−5 1.11294e−5 9.09502e−6 4.85872e−6 2.51067e−6 0
−0.1 1.87914e−5 1.70031e−5 1.53850e−5 1.25956e−5 6.89601e−6 3.98428e−6 0
−0.0 2.47201e−5 2.23679e−5 2.02394e−5 1.65707e−5 9.09428e−6 5.51598e−6 0
0.0 0 2.23679e−5 2.02394e−5 1.65707e−5 9.09428e−6 5.51598e−6 3.34558e−6
0.1 0 9.38991e−6 1.41917e−5 1.68401e−5 1.13680e−5 7.00631e−6 4.25867e−6
0.2 0 4.99787e−6 8.41462e−6 1.19931e−5 1.13331e−5 7.91561e−6 5.09952e−6
0.3 0 3.02141e−6 5.29147e−6 8.12382e−6 9.26819e−6 7.30285e−6 5.16014e−6
0.4 0 1.83088e−6 3.27241e−6 5.22780e−6 6.66903e−6 5.71842e−6 4.36413e−6
0.5 0 1.04255e−6 1.88668e−6 3.08937e−6 4.23874e−6 3.85640e−6 3.11870e−6
0.6 0 5.24606e−7 9.57346e−7 1.59419e−6 2.30167e−6 2.18631e−6 1.84705e−6
0.7 0 2.12804e−7 3.90688e−7 6.58548e−7 9.87231e−7 9.68785e−7 8.46476e−7
0.8 0 5.74705e−8 1.05990e−7 1.80314e−7 2.78220e−7 2.80038e−7 2.51273e−7
0.9 0 5.68457e−9 1.05210e−8 1.80285e−8 2.84596e−8 2.92330e−8 2.67986e−8
1.0 0 0 0 0 0 0 0

Table 11
Mie scattering—The Fourier component I∗ (τ , μ) for m = 8,with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 6.56254e−10 5.89143e−10 5.28152e−10 4.22197e−10 1.99346e−10 8.28473e−11 0
−0.8 9.16369e−9 8.23263e−9 7.38637e−9 5.91575e−9 2.81653e−9 1.18325e−9 0
−0.7 4.02493e−8 3.61908e−8 3.25016e−8 2.60890e−8 1.25472e−8 5.34291e−9 0
−0.6 1.09663e−7 9.87034e−8 8.87413e−8 7.14229e−8 3.47818e−8 1.50704e−8 0
−0.5 2.29190e−7 2.06523e−7 1.85923e−7 1.50118e−7 7.42637e−8 3.29292e−8 0
−0.4 4.03647e−7 3.64199e−7 3.28368e−7 2.66136e−7 1.34335e−7 6.15090e−8 0
−0.3 6.29436e−7 5.68693e−7 5.13578e−7 4.18023e−7 2.16513e−7 1.03927e−7 0
−0.2 8.94476e−7 8.09046e−7 7.31658e−7 5.97914e−7 3.19414e−7 1.65053e−7 0
−0.1 1.18131e−6 1.06889e−6 9.67164e−7 7.91809e−7 4.33507e−7 2.50465e−7 0
−0.0 1.47573e−6 1.33530e−6 1.20823e−6 9.89214e−7 5.42893e−7 3.29281e−7 0
0.0 0 1.33530e−6 1.20823e−6 9.89214e−7 5.42893e−7 3.29281e−7 1.99719e−7
0.1 0 5.28593e−7 7.98899e−7 9.47982e−7 6.39936e−7 3.94403e−7 2.39731e−7
0.2 0 2.63292e−7 4.43289e−7 6.31803e−7 5.97029e−7 4.16994e−7 2.68642e−7
0.3 0 1.47638e−7 2.58560e−7 3.96958e−7 4.52873e−7 3.56839e−7 2.52139e−7
0.4 0 8.20707e−8 1.46688e−7 2.34338e−7 2.9894e−7 2.56328e−7 1.95622e−7
0.5 0 4.22497e−8 7.64582e−8 1.25197e−7 1.71775e−7 1.56280e−7 1.26385e−7
0.6 0 1.88253e−8 3.43539e−8 5.72066e−8 8.25936e−8 7.84537e−8 6.62795e−8
0.7 0 6.54561e−9 1.20171e−8 2.02561e−8 3.03658e−8 2.97984e−8 2.60363e−8
0.8 0 1.42836e−9 2.63425e−9 4.48146e−9 6.91474e−9 6.95991e−9 6.24498e−9
0.9 0 9.88569e−11 1.82964e−10 3.13522e−10 4.94919e−10 5.08367e−10 4.66032e−10
1.0 0 0 0 0 0 0 0

whose the dimensions are: That is, the original Linear BVP has been reformulated as an un-
constrained optimization problem. In this case, ξ is calculated with
ξi±[mop ×1]
only one direct pass via LS methods, as the original problem is lin-
ear [21]. That is:
ξ[2mop N×1]  −1
ξ = AT A AT B (59)

b±
i [M×1]
As can be seen from (58) the unknowns are computed using a
classic LS method. However, the authors have tried all the LS meth-
ods suggested in Appendix A of reference [22], find the classic LS
B[2MN×1] method the most convenient in terms of accuracy and computa-
tional time.
Once ξ are computed, the final solutions are built according to
i [M×mop ] (50).
(same dimension for the other terms of the matrix A) After finding the solutions for all mth Fourier components (with
m = 0, 1, . . . , L), we can compute the diffuse terms expressed as:

1
A[(2MN )×(2mop N )] L
I ( τ , μ, φ ) = (2 − δ0,m )Im (τ , μ ) cos[m(φ − φ0 )] (60)
where M is the number of discretization points of τ , and mop is 2
m=0
the number of basis functions used.

10
73
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 12
Mie scattering—The Fourier component I∗ (τ , μ) for m = 0,according to Ganapol [32], with a computational time of approximately 2.6 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 4.7680739e−2 4.4191232e−2 4.0646720e−2 3.3709854e−2 1.5857241e−2 5.4529708e−3 0
−0.9 6.4564440e−2 6.0374289e−2 5.6013853e−2 4.7289947e−2 2.3812095e−2 9.0108349e−3 0
−0.8 8.4587655e−2 7.9676269e−2 7.4442518e−2 6.3753956e−2 3.3828695e−2 1.3691858e−2 0
−0.7 1.0834976e−1 1.0271869e−1 9.6567817e−2 8.3748795e−2 4.6491789e−2 1.9884482e−2 0
−0.6 1.3650449e−1 1.3020361e−1 1.2312760e−1 1.0806281e−1 6.2604846e−2 2.8172141e−2 0
−0.5 1.6967721e−1 1.6285165e−1 1.5491892e−1 1.3761804e−1 8.3292056e−2 3.9479159e−2 0
−0.4 2.0822991e−1 2.0120060e−1 1.9262576e−1 1.7335796e−1 1.1012425e−1 5.5363056e−2 0
−0.3 2.5167744e−1 2.4506518e−1 2.3630972e−1 2.1580644e−1 1.4514429e−1 7.8629773e−2 0
−0.2 2.9752321e−1 2.9240107e−1 2.8424081e−1 2.6379979e−1 1.9003527e−1 1.1455737e−1 0
−0.1 3.4012557e−1 3.3847566e−1 3.3198375e−1 3.1288756e−1 2.4121715e−1 1.7069703e−1 0
−0.0 3.5937904e−1 3.7448490e−1 3.7380021e−1 3.5934745e−1 2.8825763e−1 2.2562269e−1 0
0.0 0 3.7448490e−1 3.7380021e−1 3.5934745e−1 2.8825763e−1 2.2562269e−1 1.5152044e−1
0.1 0 1.5561969e−1 2.5142670e−1 3.3886425e−1 3.3070258e−1 2.6985545e−1 2.0307340e−1
0.2 0 9.2437025e−2 1.6508021e−1 2.6188673e−1 3.3551731e−1 3.0155334e−1 2.4424153e−1
0.3 0 6.7072588e−2 1.2414425e−1 2.1038294e−1 3.1480528e−1 3.0971262e−1 2.7030221e−1
0.4 0 5.3016958e−2 9.9893927e−2 1.7530448e−1 2.8818268e−1 3.0293863e−1 2.8106600e−1
0.5 0 4.3718197e−2 8.3238337e−2 1.4925136e−1 2.6130432e−1 2.8846360e−1 2.8082243e−1
0.6 0 3.6786430e−2 7.0519358e−2 1.2832314e−1 2.3525903e−1 2.6979273e−1 2.7302917e−1
0.7 0 3.1141106e−2 5.9990823e−2 1.1038662e−1 2.0993897e−1 2.4847557e−1 2.5982225e−1
0.8 0 2.6222496e−2 5.0716634e−2 9.4200130e−2 1.8498212e−1 2.2517903e−1 2.4245347e−1
0.9 0 2.1713290e−2 4.2153804e−2 7.9003938e−2 1.6003444e−1 2.0018674e−1 2.2166696e−1
1.0 0 1.7423141e−2 3.3971636e−2 6.4319263e−2 1.3481854e−1 1.7362739e−1 1.9793246e−1

Table 13
Mie scattering—The Fourier component I∗ (τ , μ) for m = 8,according to Ganapol [32], with a computational time of approximately 0.026 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 0 0 0 0 0 0 0
−0.9 6.5625391e−10 5.8914304e−10 5.2815237e−10 4.2219714e−10 1.9934608e−10 8.2847308e−11 0
−0.8 9.1636878e−9 8.2326319e−9 7.3863711e−9 5.9157513e−9 2.8165373e−9 1.1832473e−9 0
−0.7 4.0249326e−8 3.6190815e−8 3.2501619e−8 2.6088956e−8 1.2547246e−8 5.3429052e−9 0
−0.6 1.0966293e−7 9.8703386e−8 8.8741262e−8 7.1422850e−8 3.4781849e−8 1.5070373e−8 0
−0.5 2.2918971e−7 2.0652282e−7 1.8592286e−7 1.5011771e−7 7.4263736e−8 3.2929240e−8 0
−0.4 4.0364654e−7 3.6419928e−7 3.2836806e−7 2.6613595e−7 1.3433461e−7 6.1509004e−8 0
−0.3 6.2943646e−7 5.6869337e−7 5.1357757e−7 4.1802334e−7 2.1651285e−7 1.0392686e−7 0
−0.2 8.9447618e−7 8.0904612e−7 7.3165782e−7 5.9791370e−7 3.1941422e−7 1.6505266e−7 0
−0.1 1.1813127e−6 1.0688908e−6 9.6716367e−7 7.9180936e−7 4.3350657e−7 2.5046511e−7 0
−0.0 1.4757328e−6 1.3352990e−6 1.2082288e−6 9.8921442e−7 5.4289266e−7 3.2928109e−7 0
0.0 0 1.3352990e−6 1.2082288e−6 9.8921442e−7 5.4289266e−7 3.2928109e−7 1.9971892e−7
0.1 0 5.2859344e−7 7.9889942e−7 9.4798249e−7 6.3993602e−7 3.9440254e−7 2.3973118e−7
0.2 0 2.6329242e−7 4.4328927e−7 6.3180326e−7 5.9702947e−7 4.1699385e−7 2.6864210e−7
0.3 0 1.4763750e−7 2.5856043e−7 3.9695812e−7 4.5287275e−7 3.5683886e−7 2.5213926e−7
0.4 0 8.2070715e−8 1.4668783e−7 2.3433850e−7 2.9894033e−7 2.5632829e−7 1.9562189e−7
0.5 0 4.2249686e−8 7.6458207e−8 1.2519740e−7 1.7177455e−7 1.5627982e−7 1.2638468e−7
0.6 0 1.8825271e−8 3.4353900e−8 5.7206557e−8 8.2593552e−8 7.8453748e−8 6.6279496e−8
0.7 0 6.5456093e−9 1.2017087e−8 2.0256111e−8 3.0365796e−8 2.9798350e−8 2.6036280e−8
0.8 0 1.4283575e−9 2.6342575e−9 4.4814591e−9 6.9147374e−9 6.9599115e−9 6.2449834e−9
0.9 0 9.8856917e−11 1.8296411e−10 3.1352189e−10 4.9491896e−10 5.0836713e−10 4.6603207e−10
1.0 0 0 0 0 0 0 0

1 ω 
L L N
I (τ , −μ, φ ) = (2 − δ0,m )Im (τ , −μ ) cos[m(φ − φ0 )] (61) (cγ j h + h − h0 )T ζ +j = βl Plm (γ j ) wk Plm (μk )[(h − h0 )T ξk+
2 2
m=0 l=m k=1

+ (−1 )l−m (h − h f )T ξk− ] + b+

i
(62)
For the convenience of the reader, the all process described in
this section is summarized in the schematic of Fig. 3.

ω
L

5. Post-processing
(−cγ j h + h − h f )T ζ −j = βl Plm (−γ j )
2
l=m

Via the discretization of the cosine of the polar angle μ, the 

N

solutions obtained are for specific angles μ that are the Gauss–
× wk Plm (−μk )[(−1 ) l−m
(h − h0 )T ξk+ + (h − h f )T ξk− ] + b−i
k=1
Legendre quadrature nodes. However, in many applications, it is
necessary to compute the solutions for any arbitrary angles, that (63)
can be different than the Gauss–Legendre quadrature nodes. To
compute the solutions at any arbitrary query points μ a post– where γ j are the query angles, and ζ ±
j
are the vectors of the new
processing is needed. The post–processing is also performed via unknowns corresponding to the angle γ j , and the right–hand sides
TFC. Introducing some changes to the Eqs. (55) and (56), we get: of the equations are known. Hence, we obtain the following linear

11
74
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 14
Haze L problem—The intensity I∗ (τ , μ) for the Haze L phase function with ω = 0.9and μ0 = 1,with a computational time
of approximately 1.6 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 2.79717e−2 2.65834e−2 2.51795e−2 2.23421e−2 1.37519e−2 6.70439e−3 0
−0.9 3.01802e−2 2.87427e−2 2.72763e−2 2.42824e−2 1.50370e−2 7.32793e−3 0
−0.8 3.14478e−2 3.00707e−2 2.86411e−2 2.56621e−2 1.60962e−2 7.85504e−3 0
−0.7 3.43839e−2 3.30557e−2 3.16407e−2 2.86060e−2 1.83142e−2 9.00263e−3 0
−0.6 3.91308e−2 3.78910e−2 3.65135e−2 3.34278e−2 2.20977e−2 1.10619e−2 0
−0.5 4.56379e−2 4.46171e−2 4.33840e−2 4.04032e−2 2.80086e−2 1.45143e−2 0
−0.4 5.35113e−2 5.29854-2 5.21410e−2 4.96862e−2 3.68540e−2 2.02308e−2 0
−0.3 6.15420e−2 6.19914e−2 6.19786e−2 6.08863e−2 4.96165e−2 2.98277e−2 0
−0.2 6.69562e−2 6.90564e−2 7.04996e−2 7.20372e−2 6.66970e−2 4.63006e−2 0
−0.1 6.55296e−2 7.00041e−2 7.34324e−2 7.85904e−2 8.44628e−2 7.36110e−2 0
−0.0 5.17485e−2 6.17096e−2 6.80163e−2 7.74665e−2 9.39979e−2 9.74837e−2 0
0.0 0 6.17096e−2 6.80163e−2 7.74665e−2 9.39978e−2 9.74837e−2 7.93126e−2
0.1 0 2.24949e−2 3.95181e−2 6.26530e−2 9.62543e−2 1.08718e−1 1.08189e−1
0.2 0 1.31007e−2 2.53401e−2 4.67729e−2 9.15916e−2 1.13815e−1 1.24212e−1
0.3 0 1.01943e−2 2.02703e−2 3.94722e−2 8.74675e−2 1.16620e−1 1.35712e−1
0.4 0 9.52906e−3 1.90677e−2 3.77703e−2 8.83323e−2 1.22503e−1 1.48268e−1
0.5 0 1.02637e−2 2.05023e−2 4.06492e−2 9.64183e−2 1.35817e−1 1.67516e−1
0.6 0 1.25293e−2 2.49095e−2 4.90656e−2 1.15336e−1 1.62230e−1 2.00701e−1
0.7 0 1.74171e−2 3.44152e−2 6.70811e−2 1.53982e−1 2.13563e−1 2.61672e−1
0.8 0 2.85622e−2 5.60204e−2 1.07697e−2 2.38486e−1 3.22550e−1 3.86921e−1
0.9 0 6.16331e−2 1.19763e−1 2.26124e−1 4.76103e−1 6.19703e−1 7.17745e−1
1.0 0 3.28124e−1 6.29065e−1 1.15632 2.24839 2.74147 2.97766

Table 15
Haze L problem—The intensity I∗ (τ , μ) for the Haze L phase function with ω = 1and μ0 = 1,with a computational time
of approximately 1.6 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 3.61452e−2 3.43394e−2 3.25109e−2 2.88122e−2 1.76286e−2 8.52589e−3 0
−0.9 3.97819e−2 3.78723e−2 3.59207e−2 3.19303e−2 1.96202e−2 9.45731e−3 0
−0.8 4.27313e−2 4.08406e−2 3.88734e−2 3.47677e−2 2.16019e−2 1.03959e−2 0
−0.7 4.80051e−2 4.61319e−2 4.41307e−2 3.98292e−2 2.52479e−2 1.22171e−2 0
−0.6 5.58214e−2 5.40432e−2 5.20594e−2 4.75986e−2 3.11837e−2 1.53618e−2 0
−0.5 6.60942e−2 6.46296e−2 6.28449e−2 5.84971e−2 4.02740e−2 2.05621e−2 0
−0.4 7.81481e−2 7.74403e−2 7.62508e−2 7.27049e−2 5.37300e−2 2.91285e−2 0
−0.3 8.99682e−2 9.07706e−2 9.08784e−2 8.94711e−2 7.29643e−2 4.34688e−2 0
−0.2 9.70815e−2 1.00421e−1 1.02789e−1 1.05506e−1 9.83777e−2 6.79949e−2 0
−0.1 9.29328e−2 9.98187e−2 1.05195e−1 1.13497e−1 1.24037e−1 1.08399e−2 0
−0.0 6.98774e−2 8.46673e−2 9.41663e−1 1.08727e−1 1.35762e−1 1.42779e−1 0
0.0 0 8.46673e−2 9.41663e−1 1.08727e−1 1.35762e−1 1.42779e−1 1.14808e−1
0.1 0 2.95418e−2 5.24346e−2 8.45649e−2 1.35096e−1 1.56106e−1 1.56976-1
0.2 0 1.64907e−2 3.22817e−2 6.07527e−2 1.24350e−1 1.58925e−1 1.76818e−1
0.3 0 1.23421e−2 2.48488e−2 4.93968e−2 1.14811e−1 1.57937e−1 1.88301e−1
0.4 0 1.11879e−2 2.26450e−2 4.57547e−2 1.12269e−1 1.60862e−1 2.00019e−1
0.5 0 1.17959e−2 2.37910e−2 4.80003e−2 1.19079e−1 1.73191e−1 2.19633e−1
0.6 0 1.42049e−2 2.84584e−2 5.68731e−2 1.39051e−1 2.01445e−1 2.55983e−1
0.7 0 1.95833e−2 3.89248e−2 7.67454e−2 1.82004e−1 2.58986e−1 3.25125e−1
0.8 0 3.19532e−2 6.29430e−2 1.22045e−1 2.77182e−1 3.82767e−1 4.68658e−1
0.9 0 6.87267e−2 1.33917e−1 2.54259e−1 5.44601e−1 7.19447e−1 8.46084e−1
1.0 0 3.64940e−1 7.00266e−1 1.28955 2.52255 3.09319 3.38091

system: After finding the solutions for all mth Fourier components (with
m = 0, 1, . . . , L) we can compute the diffuse terms as follows:
 T ω
L 
N
1
L
c γ j h + h − h0 ζ +j = βl Plm (γ j ) wk Plm (μk ) I + (τ , γ , φ ) = (2 − δ0,m )Im+ (τ , γ ) cos[m(φ − φ0 )] (66)
2 2
l=m k=1
  m=0
× (h − h0 )T ξk+ + (−1 )l−m (h − h f )T ξk + b+
i
−
(64)
1
L
I− (τ , −γ , φ ) = (2 − δ0,m )Im− (τ , −γ ) cos[m(φ − φ0 )] (67)
2
m=0

 T 6. Results and discussions

ω
L 
N
−cγ j h + h − h f ζ −j = βl Plm (γ j ) wk Plm (−μk )
2 The algorithm described above has been implemented in both
l=m k=1
  Matlab and Python platforms. To validate the accuracy of our
× (−1 )l−m (h − h0 )T ξk+ + (h − h f )T ξk− + b−
i (65)
framework, we compared our results with some of typical bench-
marks for the Radiative Transfer problems, such as Mie Scattering
This is solved for ζ ±
j
via LS. Once ζ ±
j
are computed, the final so- and Haze L Problem. We ran all our simulations with an Intel Core
lutions are again built according to (50). i7 - 9700 CPU PC with 64 GB of RAM. The results reported below

12
75
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 16
Haze L problem—The intensity I∗ (τ , μ, φ ) for the Haze L phase function with ω = 0.9, μ0 = 0.5,and φ − φ0 = 0,with a
computational time of approximately 132 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 2.28190e−2 2.14170e−2 1.99920e−2 1.71574e−2 9.34719e−3 4.02513e−3 0
−0.9 4.11125e−2 3.86276e−2 3.60559e−2 3.08703e−2 1.64542e−2 6.81049e−3 0
−0.8 6.49983e−2 6.12972e−2 5.73958e−2 4.94000e−2 2.66078e−2 1.10213e−2 0
−0.7 9.99446e−2 9.47422e−2 8.91210e−2 7.73509e−2 4.26132e−2 1.79328e−2 0
−0.6 1.50993e−1 1.44083e−1 1.36343e−1 1.19657e−1 6.81169e−2 2.95065e−2 0
−0.5 2.24768e−1 2.16263e−1 2.06169e−1 1.83479e−1 1.09068e−1 4.93125e−2 0
−0.4 3.29336e−1 3.20182e−1 3.08045e−1 2.78889e−1 1.75210e−1 8.41056e−2 0
−0.3 4.72536e−1 4.65520e−1 4.52954e−1 4.18719e−1 2.82082e−1 1.47239e−1 0
−0.2 6.56834e−1 6.58390e−1 6.49495e−1 6.15206e−1 4.51466e−1 2.66183e−1 0
−0.1 8.70325e−1 8.94525e-1 8.97446e−1 8.72172e−1 6.97514e−1 4.91555e−1 0
−0.0 1.03177 1.14775 1.18797 1.19440 1.00873 7.95748e−1 0
0.0 0 1.14775 1.18797 1.19440 1.00873 7.95748e−1 5.24167e−1
0.1 0 6.07548e−1 9.98592e−1 1.38473 1.41181 1.15806 8.76470e−1
0.2 0 5.08883e−1 9.07223e−1 1.43790 1.82630 1.61273 1.29144
0.3 0 5.49994e−1 9.99898e−1 1.64825 2.30787 2.15587 1.80549
0.4 0 6.38615e−1 1.16925 1.95778 2.87301 2.77822 2.40104
0.5 0 6.34271e−1 1.16869 1.98321 3.03333 3.03426 2.71055
0.6 0 4.18260e−1 7.78087e−1 1.34620 2.18395 2.29506 2.15369
0.7 0 2.04883e−1 3.85810e−1 6.83932e−1 1.19334 1.33200 1.32682
0.8 0 8.64752e−2 1.65150e−1 3.01013e−1 5.69583e−1 6.78608e−1 7.19934e−1
0.9 0 3.13664e−2 6.09010e−2 1.14612e−1 2.37335e−1 3.03207e−1 3.43478e−1
1.0 0 5.07113e−3 1.01191e−2 2.00435e−2 4.76083e−2 6.73492e−2 8.37579e−2

Table 17
Haze L problem—The intensity I∗ (τ , μ, φ ) for the Haze L phase function with ω = 0.9, μ0 = 0.5,and φ − φ0 = π /2,with a
computational time of approximately 132 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 2.28190e−2 2.14170e−2 1.99920e−2 1.71574e−2 9.34719e−3 4.02513e−3 0
−0.9 2.69861e−2 2.54001e−2 2.3770e−2 2.04885e−2 1.12507e−2 4.83998e−3 0
−0.8 3.23251e−2 3.05433e−2 2.86841e−2 2.48816e−2 1.38576e−2 5.98687e−3 0
−0.7 3.90915e−2 3.71288e−2 3.50364e−2 3.06624e−2 1.74617e−2 7.63435e−3 0
−0.6 4.75194e−2 4.54446e−2 4.31587e−2 3.82274e−2 2.24929e−2 1.00585e−2 0
−0.5 5.76960e−2 5.56800e−2 5.33274e−2 4.79966e−2 2.95696e−2 1.37243e−2 0
−0.4 6.92921e−2 6.76843e−2 6.55506e−2 6.02592e−2 3.95485e−2 1.94423e−2 0
−0.3 8.09723e−2 8.04082e−2 7.90373e−2 7.47154e−2 5.34553e−2 2.86762e−2 0
−0.2 8.94088e−2 9.08993e−2 9.11597e−2 8.93864e−2 7.18225e−2 4.41114e−2 0
−0.1 8.86327e−2 9.36078e−2 9.65669e−2 9.91642e−2 9.15413e−2 6.94491e−2 0
−0.0 6.76014e−2 8.16018e−2 8.92220e−2 9.83762e−2 1.03484e−1 9.32369e−2 0
0.0 0 8.16018e−2 8.92220e−2 9.83762e−2 1.03484e−1 9.32371e−2 6.29164e−2
0.1 0 2.74475e−2 4.83619e−2 7.57571e−2 1.04622e−1 1.04387e−1 8.95907e−2
0.2 0 1.41330e−2 2.75945e−2 5.09061e−2 9.28868e−2 1.04678e−1 1.01178e−1
0.3 0 9.26644e−3 1.87294e−2 3.68737e−2 7.85470e−2 9.74004e−2 1.02990e−1
0.4 0 6.96644e−3 1.42411e−2 2.88244e−2 6.68034e−2 8.82947e−2 9.95180e−2
0.5 0 5.75720e−3 1.17847e−2 2.40718e−2 5.82338e−2 8.01154e−2 9.43192e−2
0.6 0 5.11030e−3 1.04251e−2 2.12819e−2 5.23831e−2 7.37544e−2 8.93298e−2
0.7 0 4.79703e−3 9.73440e−3 1.97631e−2 4.87196e−2 6.93677e−2 8.54626e−2
0.8 0 4.71115e−3 9.50506e−3 1.91501e−2 4.68396e−2 6.68825e−2 8.31200e−2
0.9 0 4.80640e−3 9.64244e−3 1.92635e−2 4.64998e−2 6.62130e−2 8.24990e−2
1.0 0 5.07113e−3 1.01191e−2 2.00435e−2 4.76083e−2 6.73492e−2 8.37579e−2

are obtained via Matlab, although the ones computed with Python racy published by Ganapol [32] for m = 0. Our results are pre-
achieved the same order of accuracy. sented in Table 12. For those simulations, the following parame-
The first benchmark of interest is the Radiative Transfer prob- ters have been employed: N = 35, M = 300,and m = 90. TFC has
lem with scattering modeled via the Mie Scattering theory for achieved the reported solutions with a computational time of 2.6 s.
spherical particles. This problem is defined by a L = 8phase func- Conversely, for m = 8Fourier component, the 7-digit accuracy of
tion. The results obtained using the TFC approach are com- Ganapol have been achieved without the need to increase eitheer
pared with the benchmarks reported by Garcia and Siewert [31]. the angle or spatial discretization. Indeed, as in the previous case,
The latter employed a discretization of N = 20 − 50μ directions. the following parameters have been used: N = 15, M = 100,and
Our results are reported in Tables 3–11, where thee follow- m = 40. Results are shown in Table 13.
ing the following parameters have been employed: N = 15 , M = For the Haze L problem, which was posed as a basic test prob-
100 , mop = 40 , τ0 = 1 , S0 = 1 , μ0 = 0.5 , ω = 0.95. The TFC algo- lem by the Radiation Commission of the International Associa-
rithm achieved the reported results with a computational time of tion of Meteorology and Atmospheric Physics (IAMAP) [33], we
0.026 s. Importantly, the TFC method achieved exactly the same compared our results with the ones reported in Benassi et al.
results as reported by Garcia and Siewert [31]. [27] and [31]. In such cases, the authors used an angular dis-
Furthermore, we ran the TFC algorithm by increasing the cretization of N = 100 − 150. For all our simulations, we ran the
angle and spatial discretizations to achieve the 7-digit accu- TFC algorithm with a discretization of N = 30. The same parame-

13
76
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 18
Haze L problem—The intensity I∗ (τ , μ, φ ) for the Haze L phase function with ω = 0.9, μ0 = 0.5,and φ − φ0 = π ,with a
computational time of approximately 132 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 2.28190e−2 2.14170e−2 1.99920e−2 1.71574e−2 9.34719e−3 4.02513e−3 0
−0.9 2.61852e−2 2.46217e−2 2.30415e−2 1.99042e−2 1.11589e−2 4.97196e−3 0
−0.8 3.20368e−2 3.01730e−2 2.82831e−2 2.45151e−2 1.38928e−2 6.24448e−3 0
−0.7 3.74817e−2 3.54358e−2 3.33331e−2 2.90811e−2 1.67591e−2 7.61010e−3 0
−0.6 4.12139e−2 3.92700e−2 3.72011e−2 3.28685e−2 1.95280e−2 9.02300e−3 0
−0.5 4.99464e−2 4.77799e−2 4.54343e−2 4.04517e−2 2.46595e−2 1.16636e−2 0
−0.4 5.50134e−2 5.32394e−2 5.11567e−2 4.64342e−2 2.98815e−2 1.47743e−2 0
−0.3 6.44934e−2 6.30132e−2 6.10801e−2 5.64129e−2 3.85985e−2 2.04385e−2 0
−0.2 7.08868e−2 7.03673e−2 6.91450e−2 6.55321e−2 4.91145e−2 2.92042e−2 0
−0.1 6.95600e−2 7.10895e−2 7.13778e−2 7.01215e−2 5.90302e−2 4.25269e−2 0
−0.0 5.47761e−2 6.23325e−2 6.55280e−2 6.81201e−2 6.39396e−2 5.39412e−2 0
0.0 0 6.23325e−2 6.55280e−2 6.81201e−2 6.39396e−2 5.39412e−2 3.42189e−2
0.1 0 1.99925e−2 3.42555e−2 5.10066e−2 6.25009e−2 5.80166e−2 4.66801e−2
0.2 0 9.41145e−3 1.79861e−2 3.18930e−2 5.27234e−2 5.56537e−2 5.06958e−2
0.3 0 5.52988e−3 1.10120e−2 2.10538e−2 4.15040e−2 4.87980e−2 4.91257e−2
0.4 0 3.67310e−3 7.45174e−3 1.48051e−2 3.24697e−2 4.12526e−2 4.48014e−2
0.5 0 2.67314e−3 5.46407e−3 1.10690e−2 2.59261e−2 3.47946e−2 3.99910e−2
0.6 0 2.11016e−3 4.31933e−3 8.82241e−3 2.14934e−2 2.99722e−2 3.59328e−2
0.7 0 1.81107e−3 3.69988e−3 7.56877e−3 1.88391e−2 2.69599e−2 3.33330e−2
0.8 0 1.72244e−3 3.50577e−3 7.15364e−3 1.79613e−2 2.61129e−2 3.29719e−2
0.9 0 1.93240e−3 3.91318e−3 7.93848e−3 1.98733e−2 2.90534e−2 3.70748e−2
1.0 0 5.07113e−3 1.01191e−2 2.00435e−2 4.76083e−2 6.73492e−2 8.37579e−2

Table 19
Haze L problem—The intensity I∗ (τ , μ) for the Haze L phase function with ω = 0.9and μ0 = 1,according to Ganapol [32], with a computa-
tional time of approximately 4.4 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 2.7971665e−2 2.6583431e−2 2.5179489e−2 2.2342144e−2 1.3751918e−2 6.7043863e−3 0
−0.9 3.0180197e−2 2.8742728e−2 2.7276328e−2 2.4282402e−2 1.5036989e−2 7.3279307e−3 0
−0.8 3.1447755e−2 3.0070750e−2 2.8641094e−2 2.5662054e−2 1.6096218e−2 7.8550379e−3 0
−0.7 3.4383906e−2 3.3055747e−2 3.1640694e−2 2.8605980e−2 1.8314210e−2 9.0026264e−3 0
−0.6 3.9130810e−2 3.7890987e−2 3.6513506e−2 3.3427829e−2 2.2097749e−2 1.1061916e−2 0
−0.5 4.5637920e−2 4.4617111e−2 4.3383966e−2 4.0403191e−2 2.8008565e−2 1.4514343e−2 0
−0.4 5.3511337e−2 5.2985449e−2 5.2140980e−2 4.9686294e−2 3.6854018e−2 2.0230769e−2 0
−0.3 6.1542012e−2 6.1991418e−2 6.1978635e−2 6.0886294e−2 4.9616521e−2 2.9827680e−2 0
−0.2 6.6956243e−2 6.9056402e−2 7.0499635e−2 7.2037240e−2 6.6696988e−2 4.6300639e−2 0
−0.1 6.5529583e−2 7.0004105e−2 7.3432378e−2 7.8590386e−2 8.4462845e−2 7.3611021e−2 0
−0.0 5.1748534e−2 6.1709609e−2 6.8016336e−2 7.7466538e−2 9.3997859e−2 9.7483668e−2 0
0.0 0 6.1709609e−2 6.8016336e−2 7.7466538e−2 9.3997859e−2 9.7483668e−2 7.9312594e−2
0.1 0 2.2494931e−2 3.9518122e−2 6.2652973e−2 9.6254337e−2 1.0871841e−1 1.0818930e−1
0.2 0 1.3100677e−2 2.5340076e−2 4.6772905e−2 9.1591615e−2 1.1381468e−1 1.2421167e−1
0.3 0 1.0194331e−2 2.0270341e−2 3.9472225e−2 8.7467481e−2 1.1662007e−1 1.3571209e−1
0.4 0 9.5290644e−3 1.9067650e−2 3.7770256e−2 8.8332315e−2 1.2250259e−1 1.4826767e−1
0.5 0 1.0263750e−2 2.0502330e−2 4.0649220e−2 9.6418345e−2 1.3581653e−1 1.6751584e−1
0.6 0 1.2529327e−2 2.4909477e−2 4.9065634e−2 1.1533634e−1 1.6223048e−1 2.0070062e−1
0.7 0 1.7417124e−2 3.4415206e−2 6.7081148e−2 1.5398186e−1 2.1356335e−1 2.6167192e−1
0.8 0 2.8562211e−2 5.6020429e−2 1.0769702e−2 2.3848565e−1 3.2254956e−1 3.8692070e−1
0.9 0 6.1633112e−2 1.1976311e−1 2.2612375e−1 4.7610276e−1 6.1970346e−1 7.1774509e−1
1.0 0 3.2812354e−1 6.2906510e−1 1.1563161 2.2483946 2.7414726 2.9776602

ters, as reported below, are used in all cases. An exception is repre- (1) φ − φ0 = 0,
sented by the parameter m, which is varied across cases. The one– (2) φ − φ0 = π /2,
angle case, when the incident beam direction is μ0 = 1,requires (3) φ − φ0 = π .
to compute only the 1stFourier component m = 0,i.e. without con-
sidering the azimuthal angle. In the two–angle cases, we calcu- The set of parameters used was: N = 30 , M = 250 , mop =
lated all the L + 1Fourier components. The TFC results are re- 90 , τ0 = 1 , S0 = 1 , μ0 = 0.5 , ω = 0.9. The TFC computational
ported in Tables 14–18. For the normal incidence case, the follow- time is reported to be approximately 132 s. Even in this case
ing parameters have been used: N = 30 , M = 250 , mop = 90 , τ0 = we were able to obtain all the digits published by Siewert et al.
1 , S0 = 1 , μ0 = 1 , ω = 0.9. For the conservative case, the follow- [27,31].
ing parameters have been used: N = 30 , M = 250 , mop = 90 , τ0 = Again, we increased the TFC angular and spatial discretiza-
1 , S0 = 1 , μ0 = 1 , ω = 1. For both simulations, the computational tions to improve the accuracy of the solutions, and to achieve the
times are approximately 1.6 s. 7-digit accuracy as published by Ganapol for the Haze L prob-
For the two-angle case, the incident beam is not μ0 = 1,so lem. We computed the incident cases (μ0 = 1), with both conser-
we needed to compute all the L + 1Fourier components. For vative single scattering albedo ω = 1and non-conservative single
the Haze L problem, 83 components are required. However, scattering albedo ω = 0.9. We found that the optimal parameters
we found out that it was not necessary to modify the pa- are N = 35,M = 400,and mop = 100. The TFC computational time
rameters for each case. The three different cases are reported is reported to be approximately 4.4 s. The results are reported in
below: Tables 19and 20.

14
77
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

Table 20
Haze L problem—The intensity I∗ (τ , μ) for the Haze L phase function with ω = 1and μ0 = 1,according to Ganapol [32], with a computa-
tional time of approximately 4.4 s.

μ τ =0 τ = 0.05τ0 τ = 0 . 1 τ0 τ = 0 . 2 τ0 τ = 0 . 5 τ0 τ = 0.75τ0 τ = τ0
−1.0 3.6145156e−2 3.4339396e−2 3.2510866e−2 2.8812216e−2 1.7628611e−2 8.5258908e−3 0
−0.9 3.9781870e−2 3.7872320e−2 3.5920682e−2 3.1930313e−2 1.9620173e−2 9.4573134e−3 0
−0.8 4.2731263e−2 4.0840607e−2 3.8873442e−2 3.4767734e−2 2.1601856e−2 1.0395857e−2 0
−0.7 4.8005147e−2 4.6131929e−2 4.4130697e−2 3.9829198e−2 2.5247889e−2 1.2217079e−2 0
−0.6 5.5821353e−2 5.4043177e−2 5.2059351e−2 4.7598583e−2 3.1183660e−2 1.5361836e−2 0
−0.5 6.6094221e−2 6.4629636e−2 6.2844874e−2 5.8497071e−2 4.0273976e−2 2.0562127e−2 0
−0.4 7.8148081e−2 7.7440255e−2 7.6250769e−2 7.2704873e−2 5.3729974e−2 2.9128534e−2 0
−0.3 8.9968154e−2 9.0770642e−2 9.0878384e−2 8.9471128e−2 7.2964349e−2 4.3468799e−2 0
−0.2 9.7081540e−2 1.0042085e−1 1.0278927e−1 1.0550594e−1 9.8377715e−2 6.7994924e−2 0
−0.1 9.2932814e−2 9.9818714e−2 1.0519502e−1 1.1349749e−1 1.2403692e−1 1.0839912e−2 0
−0.0 6.9877391e−2 8.4667310e−2 9.4166299e−1 1.0872694e−1 1.3576248e−1 1.4277947e−1 0
0.0 0 8.4667310e−2 9.4166299e−1 1.0872694e−1 1.3576248e−1 1.4277947e−1 1.1480771e−1
0.1 0 2.9541820e−2 5.2434564e−2 8.4564915e−2 1.3509602e−1 1.5610649e−1 1.5697621-1
0.2 0 1.6490681e−2 3.2281653e−2 6.0752687e−2 1.2435036e−1 1.5892546e−1 1.7681766e−1
0.3 0 1.2342100e−2 2.4848764e−2 4.9396789e−2 1.1481121e−1 1.5793652e−1 1.8830112e−1
0.4 0 1.1187938e−2 2.2644991e−2 4.5754673e−2 1.1226864e−1 1.6086203e−1 2.0001870e−1
0.5 0 1.1795943e−2 2.3790964e−2 4.8000306e−2 1.1907946e−1 1.7319074e−1 2.1963289e−1
0.6 0 1.4204907e−2 2.8458379e−2 5.6873102e−2 1.3905102e−1 2.0144487e−1 2.5598334e−1
0.7 0 1.9583294e−2 3.8924848e−2 7.6745368e−2 1.8200357e−1 2.5898644e−1 3.2512495e−1
0.8 0 3.1953231e−2 6.2942983e−2 1.2204484e−1 2.7718191e−1 3.8276705e−1 4.6865779e−1
0.9 0 6.8726703e−2 1.3391677e−1 2.5425935e−1 5.4460066e−1 7.1944669e−1 8.4608373e−1
1.0 0 3.6493954e−1 7.0026634e−1 1.2895497 2.5225517 3.0931861 3.3809098

7. Conclusions and future outlooks to tackle some of the most challenging Radiative Transfer prob-
lems such as multi-slab case [40], time-dependent integrodiffer-
This paper highlights the potential of the Theory of Functional ential Boltzmann Equations, and, subsequently, 3D time-dependent
Connections to solve one-dimensional, anisotropic Radiative Trans- Radiative Transfer Problems. Moreover, the application of X-TFC
fer Problems in homogeneous media, with high accuracy for both to time-dependent radiative transfer problems would aim to of-
one angle and two angles incident source beam, at least up to the fer an alternative to the existing frameworks based on spectral
7thfigure. and collocation methods such as [41–43]. We are currently ap-
The advantage of the proposed method lies in its elegant and plying the X-TFC physics-informed framework in solving a series
concise implementation, in the analytically satisfaction of bound- of Rarefied-Gas Dynamics problems such as Poiseuille Flow and
ary conditions, as well as in the absence of numerical issues for Thermal Creep Flow in a plane channel. Extension of the X-TFC
the conservative case (ω = 1), and in the ability to compute the to multi-dimensional RTEs is expected to be relatively straightfor-
photon flux in any exact query spatial and directional points with- ward.
out further manipulations.
Moreover, it has been shown that via TFC, one is able to directly Declaration of Competing Interest
solve the Radiative Transfer Equation in the form of Eq. (38), with-
out further transformations on the equation itself as usually neces- The authors declare that they have no known competing finan-
sary in other methods [16,32]. Future works will focus on the ap- cial interests or personal relationships that could have appeared to
plication of the TFC to the Matrix Riccati Equation (MRE) solution influence the work reported in this paper.
of the 1D RTE arising from the application of the interaction prin-
ciple of particle transport to Eq. (38) [34]. Indeed, using the MRE CRediT authorship contribution statement
would allow us to tackle the problem via recursively applying the
TFC method as presented in Mortari et al. [22] for problems with Mario De Florio: Methodology, Investigation, Software, Writ-
a long integration space (e.g., τ for our case). Furthermore, we be- ing - original draft. Enrico Schiassi: Methodology, Investigation,
lieve that this approach will allow us to combine the TFC method- Software. Roberto Furfaro: Supervision, Conceptualization, Inves-
ology with the adding-doubling method proposed by Ganapol in tigation, Writing - review & editing. Barry D. Ganapol: Resources,
Ganapol [35–38], in synergy with the Wynn-Epsilon Convergence Writing - review & editing. Domiziano Mostacci: Supervision.
Acceleration to further improve the performance of the TFC frame-
References
work in solving even more challenging Radiative Transfer prob-
lems, such as the Cloud C1 problem, posed by IAMAP [33]. This [1] Chandrasekhar S. Radiative transfer. Courier Corporation; 2013.
benchmark requires a large optical depth (τ0 = 64). This is causing [2] Wick GC. Über ebene diffusionsprobleme. Z Phys 1943;121(11–12):702–18.
the matrix to invert to be large. Thus the problem becomes too [3] Carlson B, Bell G. Solution of the transport equation by the Sn method. Tech.
Rep.. Los Alamos Scientific Lab, N Mex; 1958.
computationally expensive both in terms of memory and compu-
[4] Vilhena M, Segatto C, Barichello L. A particular solution for the Sn radiative
tational time. Hence, at this stage, large optical depth is still the transfer problems. J Quant Spectrosc Radiat Transf 1995;53(4):467–9.
limitation of our proposed method. Indeed, while our TFC based [5] Chien K-Y. Application of the S, method to spherically symmetric radiative–
method for solving the RTE is comparable with the other state of transfer problems. AIAA J 1972;10(1):55–9.
[6] Simch M, Segatto C, Vilhena M. An analytical solution for the Sn radiative
the art methods in tackling problems with small optical depths transfer equation with polarization in a slab by the LTSN method. J Quant
(e.g., Mie Scattering and Haze L problems), it appears to suffer Spectrosc Radiat Transf 2006;97(3):424–35.
when solving problems with large ones. [7] Benoist P, Kavenoky A. A new method of approximation of the Boltzmann
equation. Nucl Sci Eng 1968;32(2):225–32.
Additionally, future works will focus on the application of [8] Siewert C, Benoist P. The FN method in neutron-transport theory. Part I: theory
the newly developed Extreme Theory of Functional Connections (X- and applications. Nucl Sci Eng 1979;69(2):156–60.
TFC) [39], that is a Physics-Informed Neural Network framework, [9] Devaux C, Siewert C. The FN method for radiative transfer problems without
azimuthal symmetry. Z Angew Math Phys ZAMP 1980;31(5):592–604.

15
78
M. De Florio, E. Schiassi, R. Furfaro et al. Journal of Quantitative Spectroscopy & Radiative Transfer 259 (2021) 107384

[10] Garcia R, Siewert C. The FN method for radiative transfer models that include [27] Benassi M, Garcia R, Karp A, Siewert C. A high-order spherical harmon-
polarization effects. J Quant Spectrosc Radiat Transf 1989;41(2):117–45. ics solution to the standard problem in radiative transfer. Astrophys J
[11] Ganapol BD, Myneni R. The FN method for the one-angle radiative transfer 1984;280:853–64.
equation applied to plant canopies. Remote Sens Environ 1992;39(3):213–31. [28] Gradshteyn IS, Ryzhik IM. Table of integrals, series, and products. Academic
[12] Benassi M, Cotta R, Siewert C. The PN method for radiative transfer prob- Press; 2014.
lems with reflective boundary conditions. J Quant Spectrosc Radiat Transf [29] Chalhoub E, Garcia R. On the solution of azimuthally dependent transport
1983;30(6):547–53. problems with the ANISN code. Ann Nucl Energy 1997;24(13):1069–84.
[13] Benassi M, Garcia R, Karp A, Siewert C. A high-order spherical harmon- [30] Chalhoub E, Garcia R. A new quadrature scheme for solving azimuthally de-
ics solution to the standard problem in radiative transfer. Astrophys J pendent transport problems. Transp Theory Stat Phys 1998;27(5–7):607–24.
1984;280:853–64. [31] Garcia R, Siewert C. Benchmark results in radiative transfer. Transp Theory Stat
[14] Siewert C, Thomas J Jr.. A particular solution for the PN method in radiative Phys 1985;14(4):437–83.
transfer. J Quant Spectrosc Radiat Transf 1990;43(6):433–6. [32] Ganapol BD. The response matrix discrete ordinates solution to the 1D radia-
[15] Barichello L, Siewert CE. A discrete-ordinates solution for a non-grey model tive transfer equation. J Quant Spectrosc Radiat Transf 2015;154:72–90.
with complete frequency redistribution. J Quant Spectrosc Radiat Transf [33] Lenoble J. Standard procedures to compute atmospheric radiative transfer in a
1999;62(6):665–75. scattering atmosphere-volume I. International association of meteorology and
[16] Siewert C. A concise and accurate solution to Chandrasekhar’s basic problem atmospheric physics (IAMAP), Boulder, Colorado, USA; 1977.
in radiative transfer. J Quant Spectrosc Radiat Transf 20 0 0;64(2):109–30. [34] Ganapol B.D.. Matrix riccati equation solution of the 1d radiative transfer equa-
[17] Picca P, Furfaro R. Analytical discrete ordinate method for radiative transfer in tion. arXiv:200702437 2020.
dense vegetation canopies. J Quant Spectrosc Radiat Transf 2013;118:60–9. [35] Ganapol BD. 1D thermal creep channel flow in the BGK approximation by
[18] Ganapol B. Radiative transfer with internal reflection via the converged dis- adding and doubling. Ann Nucl Energy 2019;134:441–51.
crete ordinates method. J Quant Spectrosc Radiat Transf 2011;112(4):693–713. [36] Ganapol BD. Particle transport in a 3d duct by adding and doubling. J Comput
[19] Picca P, Furfaro R, Ganapol BD. An efficient multiproblem strategy for accurate Theor Transp 2017;46(3):202–28.
solutions of linear particle transport problems in spherical geometry. Nucl Sci [37] Ganapol BD. Poiseuille channel flow by adding and doubling. In: AIP confer-
Eng 2012;170(2):103–24. ence proceedings, 1786. AIP Publishing LLC; 2016. p. 070 0 09.
[20] Mortari D. The theory of connections: connecting points. Mathematics [38] Ganapol BD. A 1d radiative transfer benchmark with polarization via doubling
2017;5(4):57. and adding. J Quant Spectrosc Radiat Transf 2017;201:236–50.
[21] Mortari D. Least-squares solution of linear differential equations. Mathematics [39] Schiassi E., Leake C., De Florio M., Johnston H., Furfaro R., Mortari D.. Extreme
2017;5(4):48. Theory of Functional Connections: A Physics-Informed Neural Network Method
[22] Mortari D, Johnston H, Smith L. High accuracy least-squares solutions of non- For Solving Parametric Differential Equations. arXiv2020b.
linear differential equations. J Comput Appl Math 2019;352:293–307. [40] Previti A, Furfaro R, Picca P, Ganapol BD, Mostacci D. Solving radiative trans-
[23] Furfaro R, Mortari D. Least-squares solution of a class of optimal space guid- fer problems in highly heterogeneous media via domain decomposition and
ance problems via theory of connections. Acta Astronaut 2019. doi:10.1016/j. convergence acceleration techniques. Appl Radiat Isot 2011;69(8):1146–50.
actaastro.2019.05.050. [41] Chen S-S, Li B-W, Sun Y-S. Chebyshev collocation spectral method for solving
[24] Johnston H., Schiassi E., Furfaro R., Mortari D.. Fuel-Efficient Powered Descent radiative transfer with the modified discrete ordinates formulations. Int J Heat
Guidance on Large Planetary Bodies via Theory of Functional Connections. Mass Transf 2015;88:388–97.
arXiv:200103572 2020. [42] Kim AD, Moscoso M. Chebyshev spectral methods for radiative transfer. SIAM
[25] Schiassi E., D’Ambrosio A., Johnston H., Furfaro R., Curti F., Mortari D.. Complete J Sci Comput 2002;23(6):2074–94.
Energy Optimal Landing on Planetary Bodies via Theory of Functional Connec- [43] Sun Y-S, Li B-W. Chebyshev collocation spectral method for one-dimen-
tions. AAS 20-557, AAS/AIAA Astrodynamics Specialist Conference, Lake Tahoe, sional radiative heat transfer in graded index media. Int J Therm Sci
CA, August 9-13, 2020. 2009;48(4):691–8.
[26] Meador W, Weaver W. Two-stream approximations to radiative transfer in
planetary atmospheres: a unified description of existing methods and a new
improvement. J Atmos Sci 1980;37(3):630–43.

16
79
Appendix C

De Florio et al. (2021) Thermal Creep

Flow

Reproduced from De Florio, M., Schiassi, E., Ganapol, B. D., & Furfaro, R. (2021). Physics-
informed neural networks for rarefied-gas dynamics: Thermal creep flow in the Bhatna-
gar–Gross–Krook approximation. Physics of Fluids, 33(4), 047110. https://doi.org/10.1063/5.0046181
[9], with the permission of AIP Publishing.

80
Physics-informed neural networks for
rarefied-gas dynamics: Thermal creep flow in
the Bhatnagar–Gross–Krook approximation
Cite as: Phys. Fluids 33, 047110 (2021); https://doi.org/10.1063/5.0046181
Submitted: 01 February 2021 • Accepted: 27 March 2021 • Published Online: 21 April 2021

Mario De Florio, Enrico Schiassi, Barry D. Ganapol, et al.

ARTICLES YOU MAY BE INTERESTED IN

Uncovering near-wall blood flow from sparse data with physics-informed neural networks
Physics of Fluids 33, 071905 (2021); https://doi.org/10.1063/5.0055600

Simulation of multi-species flow and heat transfer using physics-informed neural networks
Physics of Fluids 33, 087101 (2021); https://doi.org/10.1063/5.0058529

Data-driven physics-informed constitutive metamodeling of complex fluids: A multifidelity

neural network (MFNN) framework
Journal of Rheology 65, 179 (2021); https://doi.org/10.1122/8.0000138

81
Phys. Fluids 33, 047110 (2021); https://doi.org/10.1063/5.0046181 33, 047110

© 2021 Author(s).
 Physics of Fluids ARTICLE scitation.org/journal/phf

Physics-informed neural networks for rarefied-gas

dynamics: Thermal creep flow in the
Bhatnagar–Gross–Krook approximation
Cite as: Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181
Submitted: 1 February 2021 . Accepted: 27 March 2021 .
Published Online: 21 April 2021

Mario De Florio,1 Enrico Schiassi,1 Barry D. Ganapol,2 and Roberto Furfaro1,2,a)

AFFILIATIONS
1
Department of Systems and Industrial Engineering, The University of Arizona, 1127 James E. Rogers Way, Tucson, Arizona 85719,
USA
2
Department of Aerospace and Mechanical Engineering, The University of Arizona, 1130 N Mountain Ave, Tucson, Arizona 85721,
USA

a)
Author to whom correspondence should be addressed: robertof@email.arizona.edu

ABSTRACT
This work aims at accurately solve a thermal creep flow in a plane channel problem, as a class of rarefied-gas dynamics problems, using
Physics-Informed Neural Networks (PINNs). We develop a particular PINN framework where the solution of the problem is represented by
the Constrained Expressions (CE) prescribed by the recently introduced Theory of Functional Connections (TFC). CEs are represented by a
sum of a free-function and a functional (e.g., function of functions) that analytically satisfies the problem constraints regardless to the choice
of the free-function. The latter is represented by a shallow Neural Network (NN). Here, the resulting PINN-TFC approach is employed to
solve the Boltzmann equation in the Bhatnagar–Gross–Krook approximation modeling the Thermal Creep Flow in a plane channel. We test
three different types of shallow NNs, i.e., standard shallow NN, Chebyshev NN (ChNN), and Legendre NN (LeNN). For all the three cases
the unknown solutions are computed via the extreme learning machine algorithm. We show that with all these networks we can achieve
accurate solutions with a fast training time. In particular, with ChNN and LeNN we are able to match all the available benchmarks.
Published under license by AIP Publishing. https://doi.org/10.1063/5.0046181

I. INTRODUCTION Boltzmann equation,13 or its approximations such as Bhatnagar,

Accurate calculations of rarefied gas flows at an arbitrary Gross, and Krook (BGK)14 and Shakhov,15 which have been exten-
Knudsen number is extremely challenging. Rarefied Gas Dynamics sively used in the literature to find their numerical solutions with sev-
(RGD) describes the fluid dynamics of a gas where the mean free path eral methods.16–26
ðkÞ of the molecules is larger than the size (d) of the chamber under In the last decay, the thermal creep flow has been widely studied
test. Generally, a gas flow is considered rarefied when the Knudsen based on the Direct Simulation Monte Carlo (DSMC),27 for both
number Kn ¼ dk > 1. In particular, a thermal creep flow, also known monatomic28–31 and polyatomic gases.17,32–34 Over the years, several
as Stokes flow, is a flow of a slightly rarefied gas caused by the temper- methods have been developed and improved to solve Boltzmann inte-
ature gradient along a wall, and when in the fluid the viscous forces grodifferential transport equation applied to problems of physics, such
dominate over inertial forces. as radiative transfer, neutron transport, and RGD. The first family of
The phenomenon of thermal creep effect has been initially stud- methods developed for the transport theory dates back to the 1940s,
ied independently by Reynolds,1 Maxwell,2 and Knudsen.3 named Discrete Ordinates Method (DOM).35,36 Subsequently, the
Subsequently, many researchers have addressed the problem with an DOM has undergone improvements and changes to achieve ever
experimental approach.4–9 From a theoretical point of view, the ther- higher accuracy. Among these, we count the Sn method, which uses
mal creep problem in the field of RGD has been investigated to achieve line segments to approximate the ldependence of the angular
trustworthy solutions for a large interval of Knudsen number, with the flux,37–39 and the CN method, where the angular flux is approximated
fundamentals of kinetic theory.10–12 The fundamental mathematical by an expansion of orthogonal polynomials.40 During the 1980s the
model generally employed to describe thermal creep flows is the FN method and the PN method were developed, extended, and applied

82
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-1
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

to solve problems in neutron and photon transport.41–46 Around The goal of this paper is to develop and evaluate a PINNs-based
20 years ago, Barichello and Siewert presented the first version of the methodology to solve the 1D thermal creep channel flow in a plane
Analytical Discrete Ordinates (ADO) method, which achieved some channel problem. Following the formulation presented by Ganapol,54
of the most accurate benchmark in the field of radiative transfer.47,48 we demonstrate that the proposed PINN-based approach is accurate
Many other methods and applications for solving problems aris- in solving this class of problems arising from the Boltzmann integro-
ing from the Boltzmann equation have been published in the footsteps differential equation. Specifically, to solve this problem, we develop a
of the previous ones to date,49–51 and as many new ones were born. particular PINN framework that brings together NNs and the Theory
In particular, recently, new methods to solve differential equa- of Functional Connections (TFC).55,56 Recently developed by
tions (DE) were developed within the Machine Learning community. Mortari,57 TFC enables functional interpolation for a large class of
Named Physics-Informed Neural Networks (PINNs), such methods mathematical objects. According to TFC, any solution of a mathemati-
train Neural Networks (NN) to learn the solution of Differential cal problem can be represented via the so-called Constrained
Equations (DE). However, PINNs are not only employed to solve DEs Expressions (CEs). The CEs are a sum of a free-function and a func-
and can be generally considered as machine learning algorithms that tional that analytically satisfies the problem constraints, regardless to
embed physics into data-driven representation of functional relation- the choice of the free-function.57 Among its many applications, TFC
ships underlying collections of input–output pairs. The term PINN, as has been applied to solve ODEs with generally excellent results, leading
described by Raissi et al.,52 defines NNs that use the physics as regular- to machine-level accuracy solutions in a short computational time.58,59
izer in the loss function. As an example, assume that one wants to per- The authors already demonstrated that TFC can be successfully
form a regression of a set of experimental data using a NN, and that employed to solve the Chandrasekhar’s basic problem in radiative
such collected data represent a physical phenomena modeled via a set transfer,60 as well as optimal control problems for aerospace applica-
of DEs. In standard regression, one could approximate the data using tions.61–63 Here, we consider a PINN TFC-based approach where shal-
a NN (either shallow or deep) trained to minimize a Mean Squared low NN are employed to solve DEs while analytically satisfying IC/
Error (MSE) as loss function. However, doing so, there is no guarantee BCs via TFC prescriptions. Such approach aims at overcoming the
that the physics phenomena governing the data would not be violated. main limitations of the standard PINN and TFC methods (e.g., solu-
More precisely, especially when the data are noisy, one could tion accuracy and curse of dimensionality, respectively). We solve the
experience the issue of over-fitting. That is, the data would be accu- Thermal Creep Flow problem using, as free function in the CE, three
rately fitted (e.g., negligible error between the NN approximation and different types of shallow NNs: standard shallow NN,56 Chebyshev
the experimental data), but the relation input–output, governed by the NN (ChNN),64 and Legendre NN (LeNN).65 For all the three cases the
physics phenomena, could be poorly learned. For instance, imagine to unknown solutions are then computed via the Extreme Learning
kick a ball, and to measure its position (which represent the output of Machine (ELM) algorithm.66 According to the ELM algorithm, input
the dataset) at certain times (which represent the input of the dataset), weights and bias are randomly selected and not tuned during the
from when the ball is kicked till it lands on the ground, to learn, from training, leaving the outputs weight to be the only trainable parame-
the measurements, what the trajectory (which represents the relation ters. Thus, the training is reduced to a fast least squares.
input–output) is. In this easy example, it is known, from physics prin- The paper is organized as follows: in Sec. II the PINN methodol-
ciples, that the trajectory is a parabola. However, if one wants to ogy and how to apply it to solve a generic first order linear ODE are
understand what the trajectory is by fitting the experimental data (e.g., presented. In Sec. III how to apply PINN TFC-based methods to tackle
the position-time pairs), if the data are noisy and the over-fitting generic first order linear ODEs is explained. The PINN formulation of
occurs the trajectory may not be learned correctly. For example, the the Thermal creep flow in a plane channel problem is showed in Secs.
trajectory could be approximated as a periodic function, instead of as a IV and V. Numerical results are presented and discussed in Sec. VI.
parabola, and this would violate the physics phenomena governing the
collected experimental data. II. PHYSICS-INFORMED NEURAL NETWORKS
PINN are therefore introduced to ensure that DEs, coming from PINN have been proposed by Raissi et al.52 to solve two main
first principles and modeling the physics of the experimental dataset, classes of problems: data-driven solution and data-driven discovery of
are added as a penalty to the loss function. This additional term acts as PDEs, validating the accuracy with benchmark comparisons.
a regularizer that penalizes the training when the DE and the Initial Although in Ref. 52 PINN were introduced to tackle problem gov-
Conditions (ICs) and/or Boundary Conditions (BCs) are violated. erned by PDEs, the same framework can be used to solve problem
Overall, one ensures that the physics underlying the process is not vio- involving ODEs as well. The most known mesh-based methods [such
lated. This approach is called data-driven solution of DEs. Conversely, as Finite Difference Method (FDM) and Finite Element Method
when the goal is to estimate parameters governing some physical phe- (FEM)67] use piece-wise polynomial functions as basis functions.
nomena modeled through a DE (e.g., the thermal conductivity in the Conversely, PINN employs a single deep neural network (DNN) to
heat equation) one conventionally talks about data-driven parameters fully represent the solution. In particular, the training set consists on
discovery of DEs (e.g., inverse problems).52,53 When data are not avail- randomly parsed data points from the full high-resolution dataset, and
able, and therefore the loss function contains only the residual of the randomly sampled collocation points for enforcing BCs and to enforce
DEs, as well as the corresponding ICs and/or BCs, PINNs are used for the PDE, on the boundary and inside the solution domain, respec-
approximating the solutions of problems involving ordinary differen- tively. Once the output of DNN is computed, it is plugged in the resid-
tial equations (ODEs) and is Partial Differential Equations (PDEs). To ual of the DE and in the residual of the BCs. The training of the NN is
this end PINNs can be also viewed as methods that approximate solu- used to learn the weights and biases parameters that minimize the
tions of DEs using NNs. physics-based cost function, which can be DE and BCs least squares

83
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-2
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

solved with gradient-based optimization methods. Thus, the key fea- The unknown solution is approximated via the constrained expres-
ture of the PINN is the embedding of the information of the physics of sion, defined within the TFC framework. That is,
the problem, derived from the DE and the BCs, in their cost functions.
X
n
To better clarify how a PINN works, the mathematical formulation to yðtÞ ¼ gðtÞ þ gk pk ðtÞ ¼ gðtÞ þ gT pðtÞ; (10)
solve a first order linear ODE is reported below. k¼1

where, pk ðtÞ are n assigned linearly independent functions and the g(t)
A. PINN approach to solving first order linear ODEs
is the free function, which must be linearly independent from the
Consider solving the following first-order ODE: pk ðtÞ functions. The values of the gk coefficients are calculated by
Fðt; y; y_ Þ ¼ 0 subject to : yðt0 Þ ¼ y0 : (1) imposing a set of n linear constraints in y(t). By doing that, the con-
straints for the differential equations are satisfied for any chosen g(t).
The unknown solution is approximated by using a NN (that can be For a first order ODE the CE becomes
either deep or shallow). That is,
yðtÞ ¼ gðtÞ þ g pðtÞ; (11)
yðtÞ  yNN ðt; HÞ; (2)
where p(t) ¼ 1 (see Refs. 58 and 59 for the details of this decision).
where H are the NN parameters (e.g., weights and biases). The deriva- Thus, the CE becomes
tive is
yðtÞ ¼ gðtÞ þ g: (12)
d
y_ ðtÞ  y_ NN ðt; HÞ ¼ yNN ðt; HÞ; (3) By imposing the constraint, we get
dt
which is computed using automatic differentiation.68,69 Thus, the yðt0 Þ ¼ y0 ¼ g0 þ g ) g ¼ y0  g 0 ; (13)
approximated solution and its derivative defined by Eqs. (2) and (3) which is plugged into Eq. (12) to obtain the CE in its final form
are plugged into (1). Doing so, we have the following residuals for
both ODE and BC yðtÞ ¼ gðtÞ þ ðy0  g0 Þ; (14)
RF ðtÞ ¼ Fðt; yNN ; y_ NN Þ; (4) which represents all possible functions satisfying the boundary value
RBC ðtÞ ¼ yNN ðt0 ; HÞ  y0 ; (5) constraint. The derivative then follows:

where yNN ðt0 ; HÞ is the NN comptued in t ¼ t0. Now, we need to y_ ðtÞ ¼ g_ ðtÞ: (15)
build the MSE loss as The CE defined by Eq. (14) and its derivative are plugged into our
MSE ¼ MSEF þ MSEBC ; (6) ODE which then becomes

where, F ðt; y; y_ Þ ¼ 0; (16)

that is now a boundary condition free. By substituting Eqs. (14) into
1 X NF
MSEF ¼ R2 ðti Þ (7) (16), the differential equation F is transformed into a new differential
NF i¼1 F equation F~ , which is only a function of the independent variable t and
the free-function g(t), that is,
and
F~ ðt; g; g_ Þ ¼ 0: (17)
MSEBC ¼ R2BC ðt0 Þ: (8)
This differential equation is now unconstrained and will always satisfy
Here, NF are the training points and ti is the independent variable the boundary-value due to the way Eq. (12) is built. In this work, we
computed in the ith training point. choose the free function g(t) to be a shallow NN, such that
Now, the NN parameters H are learned by minimizing the MSE
loss with gradient-based optimization methods, such as stochastic gra- 3T 2
r1
dient descent, and plugged into (2) to obtain the approximated solu- X
L
 
6 .. 7
tion of the ODE. gðtÞ ¼ bj r wj t þ bj ¼ 4 . 5 b ¼ rT b; (18)
j¼1
rL
III. PINN TFC-BASED APPROACH TO SOLVING FIRST
ORDER LINEAR ODEs where L is the number of hidden neurons, wj 2 R is the input weights
As previously mentioned, in this work we develop an alternative vector connecting the jth hidden neuron and the input nodes, bj 2 R
PINN framework. In the proposed approach, we merge the standard with j ¼ 1; …; L is the jth output weight connecting the jth hidden
PINN with the TFC method.55,56 neuron and the output node, and bj is the bias of the jth hidden neu-
Here, we present the step-by-step procedure for solving boundary ron, and rj ðÞ are activation functions. Once the free-function is
value linear ODEs via the proposed PINN-TFC based method. defined, the unknown solution y(t), defined by Eq. (14), is computed
Consider solving the following first-order ODE via the ELM algorithm. According to the ELM algorithm Ref. 66, input
weights and biases are randomly selected and not tuned during the
Fðt; y; y_ Þ ¼ 0 subject to : yðt0 Þ ¼ y0 : (9) training, thus they are known parameters. The activation functions,

84
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-3
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

rj ðÞ, are chosen by the user, so they are also known. Therefore, the For the convenience of the reader, a schematic that summarizes how
only unknowns to compute are the output weights b ¼ ½b1 ; …; bL T . the PINN-TFC based method works for first order linear ODEs is
The interested reader can find more details about the ELM and its reported in Fig. 2
convergence proofs in Ref. 66. In this work we use three different types IV. THERMAL CREEP FLOW IN A PLANE CHANNEL:
of shallow NNs: standard shallow NN, ChNN, and LeNN. From previ- PROBLEM FORMULATION
ous works,70 we noticed that for ChNN and LeNN, the choice of input
weights and biases that guarantees the best accuracy, is to set wj ¼ 1 Three basic problems defined by flow in a finite plane-parallel
and bj ¼ 0 ; 8j ¼ 1; …; L. For the convenience of the reader, in Fig. 1 channel are considered important when evaluating the effectiveness of
we show a representation of a general shallow NN and the ChNN, and a new kinetic model of the linearized Boltzmann equation or a new
LeNN that we use to solve the problem presented in this research. It computational method. These problems are the Couette flow, the
may happen that the activation functions are defined on an inconsis- Poiseuille flow, and the thermal creep flow, and they are widely studied
by Cercignani.71
tent domain z 2 ½z0 ; zf  (for instance, Chebyshev and Legendre poly-
In the Couette flow, a rarefied gas is confined between two paral-
nomials are defined on z 2 ½1; þ1). Thus, our independent variable
lel walls, one of which moves in its own plane with respect to the other.
t 2 ½t0 ; tf  needs to be mapped in the z domain. In general to have bet-
The Poiseuille flow is a particular case of flow in a plane channel
ter training performance, regardless to the activation function used, it
driven by a pressure gradient in a direction parallel to the walls that
is convenient to map the independent variable t 2 ½t0 ; tf  in the z
enclose the rarefied gas. A thermal creep flow, unlike the Poiseuille
domain (usually z 2 ½0; þ1 or z 2 ½1; þ1). This can be done using
flow, is driven by a wall temperature gradient. It occurs between two
the following linear transformation:
long parallel plates at a very short distance (in the scale of micro/nano-
1 dimension), and the momentum of the flow varies from the cooler to
z ¼ z0 þ cðt  t0 Þ $ t ¼ t0 þ ðz  z0 Þ; the warmer molecules, until an equilibrium temperature is reached.
c
Only then will the differences in molecular velocities and the shear
where c is the mapping coefficient
stress through the wall equal zero.
zf  z0 As explained by Siewert in Ref. 72, the Poiseuille flow and the
c¼ :
tf  t0 thermal creep flow have much in common, and their problems present
By the derivative chain rule, the nth derivative of Eq. (18) is a similar formulation. However, for the purpose of this work, we
report only the formulation of the thermal creep problem in BGK
dn gðtÞ n
n T d rðzÞ approximation. The latter significantly simplifies the mathematics of
¼ c b ; (19)
dt n dz n the problem by replacing the collision integral with a simple relaxation
model. It has proved to be a useful qualitative model, because it retains
which defines all mappings of the free-function. Finally, the domain t
most of the qualitative characteristics of the Boltzmann equation. The
must be discretized by M points, and then, the DE reduces to the
validity of the BGK model has been evaluated by comparing its solu-
following:
tion with the Boltzmann solutions for several relaxation problems,73
F~ ðbÞ ¼ 0: (20) showing satisfactory accuracy and history of the velocity distribution
function during the relaxation, when the flow is not far from a
Equation (20) is an unconstrained optimization problem that can be Maxwellian distribution. Also, there is no direct connection with the
solved via different optimization schemes. As the problem is linear, relaxation problem to the Knudsen number, thus the BGK model can
rearranging the term in Eq. (20), it reduces to a linear system of alge- be applied to flow of highly rarefied gas, as long as degree of non-
braic equation: Ab ¼ b. Therefore, equilibrium is not strong in velocity distribution. One of its main dis-
1 advantages is that it uses a Prandtl number Pr ¼ 1, while for a mon-
b ¼ ðAT AÞ AT b: (21) atomic gas a value around 2/3 would be more appropriate.10,74–76

FIG. 1. Standard shallow neural network (left) and shallow Chebyshev/Legendre neural network (right).

85
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-4
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

FIG. 2. Schematic of the general PINN-TFC based framework to solve linear ODEs with one constraint in one point.

Thus, the BGK model cannot match the viscosity and the thermal con- n1 ðx; z Þ ¼ n0 ð1 þ Rx x þ Rz z Þ;
ductivity of a real gas simultaneously. Therefore, when heat transfer is
T1 ðx; z Þ ¼ T0 ð1 þ Kx x þ Kz z Þ;
involved in the flow calculations, particular precautions must be taken.
To bypass this problem, many modified models. such as the u ðxÞ ¼ Kx;
ellipsoidal-statistical BGK model77 and the Shakhov model15 have here, Ri and Ki are the relative density and temperature gradients in
been proposed. In this paper, we consider the creep flow in the 1D the ith direction, respectively. Let’s consider the non-dimension
micro channel shown in Fig. 3. velocity
The full non-linear Boltzmann equation for the gas atom space  
and velocity distribution function f ðr; vÞ with the collision operator J m 1=2
c¼v :
can be written as 2kT0
v  rr f ðr; vÞ ¼ Jðf ; f 0 Þ; (22) Now, if we substitute the Eq. (23) into Eq. (22), we obtain the
78 following:
and by following the formulation proposed by Williams, we can
    
introduce a perturbation and express f as 3 3
cx c2  Kx þ Rx þ 2cz K0 þ cz c2  Kx þ cz Rz
f ðr; vÞ ¼ f0 ðr; vÞ½1 þ hðr; vÞ; (23) 2 2
ð
@hðx; cÞ
where h is the noise caused by the presence of the walls to the local þcx þ VðcÞhðx; cÞ ¼ Kðc; c Þhðx; c0 Þdc0 ;
0
(25)
@x
Maxwellian f0 ðr; vÞ, which has the following form:
 3=2 with
m
f0 ðr; vÞ ¼ n1 ðx; zÞ  1=2  1=2
2pkT1 ðx; zÞ m m
 K0 ¼ K ; V ðcÞ ¼ K V ðvÞ;
m n o 2kT0 2kT0
 exp   ðxÞ2 ; (24)
vx2 þ vy2 þ ½vz  u
2pkT1 ðx; zÞ where V(v) represents the collision frequency of the atoms. Now we
where, can take use of the classical BGK model, which assumes that the colli-
sion frequency VðcÞ ¼ k is constant and independent of velocity.
Usually, this value is chosen to be equal to the viscosity of the modeled
gas.79 Thus, the scattering kernel of Eq. (25) is
   
0 k0 0 2 2 3 02 3
Kðc; c Þ ¼ 3=2 1 þ 2c  c þ c  c  ; (26)
p 3 2 2
where,
 1=2
m
k0 ¼ k:
2kT0
FIG. 3. Creep microchannel flow. Then, Eq. (25) becomes

86
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-5
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

    
3 3 where u is set to cx, subject to
cx c2  Kx þ Rx þ 2cz K0 þ cz c2  Kx þ cz Rz
2 2 (
ð Z ða; uÞ ¼ ð1  aÞZ ða; uÞ;
@hðx; cÞ (35)
þ cx þ k0 hðx; cÞ ¼ Kðc; c Þhðx; c0 Þdc0 :
0
(27) Z ða; uÞ ¼ ð1  aÞZ ða; uÞ;
@x
By considering the upper and lower channel plates at x ¼ a and a, the for u 2 ð0; 1Þ. By using
most general form of the gas-wall boundary condition at the plates is  
ð Kz 1
Zðx; uÞ ¼ Yðx; uÞ  u2  ; (36)
cx f ð6a; cÞ ¼ c0 <0 c0x Cðc; c0 Þf ð6a; c0 Þdc0 : (28) 2k0 2
x
c0x >0
and plugging it into (34) and (35), we find that Yðs; uÞ satisfies the fol-
More details regarding the derivation of Eq. (27) are reported in lowing linear integrodifferential equation:
Ref. 78. We assume a local Maxwellian that characterizes a diffusely ð1
and specularly reflecting wall @
u Yðs; uÞ þ Yðs; uÞ ¼ WðuÞYðs; uÞdu; (37)
 3=2  o @s
m m n 2 1
Fðv;Tw ;uw Þ ¼ N exp  vx þ vy2 þ ðvz  uw Þ2 ; where s 2 ð0; aÞ replaces x by taking advantage of spatial symmetry,
2pkTw 2kTw
and u 2 ð1; 1Þ; subject to the following boundary conditions:
(29)
(
where N is the molecular density and Tw and uw are the temperature Y ð0; uÞ ¼ Y ð0; uÞ;
and the velocity at the wall, respectively. By using the perturbation of (38)
Y ða; uÞ ¼ ð1  aÞY ða; uÞ þ g ðuÞ;
Eq. (23) with F instead of f0 into Eq. (28) gives
  
hð6a; cÞ ¼ a 2cz uw þ dðc2  2Þ þ ð1  aÞhð6a; cðcx ; cy ; cz ÞÞ for u 2 ð0; 1Þ, where gðuÞ ¼ a2 u2  12 . Also, we used the normali-
ð zation Kk0z  1, and WðuÞ is a weight function defined as follows:
2a c02
þ e hðx 6 a; c0 Þdc0 ; (30)
p 2
WðuÞ ¼ p1=2 eu : (39)
for cx < 0 and cx > 0, where d ¼ TwTT 0
0
. The accommodation coeffi-
cient a 2 ð0; 1 represents the fraction of diffuse reflection. The problem (37) and (38) is the one to be solved via the PINN TFC-
Following the formulation proposed by Ganapol,54 the problem based method.
to be solved is the following:
  V. PINN TFC-BASED FORMULATION OF THE PROBLEM
1 1 3 @
cx K0 þ Rz þ Kz c2x þ þ cx Zðx; cx Þ þ k0 Zðx; cx Þ As we deal with a Boundary Value Problem (BVP), according to
2 2 2 @x
ð (10), yðsÞ can be expressed as following:
2
¼ k0 p1=2 ecx Zðx; cx Þdcx ; (31)
yðsÞ ¼ gðsÞ þ g pðsÞ: (40)
for x 2 ½a; a and cx 2 ð1; 1Þ and subject to
( We choose pðsÞ ¼ 1, as suggested by Ref. 58, and the free function
Z ða; cx Þ ¼ auw þ ð1  aÞZ ða; cx Þ; gðsÞ as a shallow NN
(32)
Z ða; cx Þ ¼ auw þ ð1  aÞZ ða; cx Þ;
gðsÞ ¼ rT b: (41)
for cx 2 ð0; 1Þ, where a is the channel half-thickness, Rz and Kz are We map the independent variable s to a new independent variable x
gradients in the flow direction z, K is the scattering kernel, K0 is pro-
that ranges in ½1; 1. The new independent variable x is defined as
portional to K, k0 is proportional to the frequency of collisions
x ¼ cðs þ aÞ  1, where c is the following mapping constant:
between the atoms, x is the spatial variable, a 2 ð0; 1 is the accommo-
dation coefficient, and the moment Zðx; cx Þ is xf  x0 1
ð1 ð1 c¼ ¼ :
2 2 sf  s0 a
Zðx; cx Þ ¼ p1 eðcy þcz Þ cz hðx; cx ; cy ; cz Þdcy dcz ; (33)
1 1 Considering the new independent variable, we have
where ðcx ; cy ; cz Þ are the three components of the molecular velocity 8
and h is a perturbation from Maxwell distribution, according to >
< Y ðs; uÞ ¼ Y ðx; uÞ;
Cercignani.80 @ @ (42)
>
: Y ðs; uÞ ¼ c Y ðx; uÞ:
For our case of interest, the creep flow, we can make some @s @x
assumptions, such as K0 and uw are equal to 0 (no-slip velocity condi-
tion), and Rz ¼ Kz , obtaining Then the problem becomes
  ð1
1 1 @ @
Kz u2  þ u Zðx; uÞ þ k0 Zðx; uÞ cu Yðx; uÞ þ Yðx; uÞ ¼ WðuÞYðx; uÞdu: (43)
2 2 @x @x 1
ð1
¼ k0 p1=2
2
eu Zðx; uÞdu; (34) In order to use a Gauss–Legendre quadrature in the range ½1; 1, we
1 can use a further change of variable

87
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-6
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

1 2
Replacing them in the previous system of equations, we obtain
WðlÞ ¼ p2 eðlogðjljÞÞ ;
1
u ¼ logðjljÞ; du ¼  dl;
l (
rT0 bþ þ T  
i þ gi ¼ r0 bi þ gi ;
where l 2 ½1; 1. We can then rewrite the problem in the following
form: rTf b  T þ þ
i þ gi ¼ ð1  aÞrf bi þ ð1  aÞgi þ gi ;

ð1 and then
@ 1
c log ðlÞ Yðx; lÞ þ Yðx; lÞ ¼  WðlÞYðx; lÞdl: (44) (
@x l gþ  T  T þ
i  gi ¼ r0 bi  r0 bi ;
1

Via discretizing the velocity l into N points in the interval ð0; 1 ð1  aÞgi  gi ¼ rf b
þ  T T þ
i  ð1  aÞrf bi  gi ;
according to the Gauss–Legendre quadrature scheme we have
where in matrix form is
l ! l ¼ fli gNi¼1 ; " #" # " #
1 1 gþ i rT0 b T þ
i  r0 bi
it follows that: ¼ :
1  a 1 g i
rTf b T þ
i  ð1  aÞrf bi  gi
Yðx; lÞ ! YðxÞ ¼ fYðx; li ÞgNi¼1 : (45)
Keeping the gs vector on the LHS, we get
The problem can be split for both positive and negative molecular " # " #" #
velocity following Ref. 54, where the Gauss–Legendre quadrature rule gþi 1 1 1 rT0 b T þ
i  r0 bi
¼ ;
is used to evaluate the integral in the range ½0; 1: gi a ð1  aÞ 1 rTf b T þ
i  ð1  aÞrf bi  gi

@ that is,
c log ðli Þ Yðx; li Þ þ Yðx; li Þ
@x 2 3
XN
1 " # 1 1 " #
¼ wk Wðlk Þ½Yðx; lk Þ þ Yðx; lk Þ ; (46) gþ 6 a  7 rT0 b  rT0 bþ
l k
i
¼6 a 7 i i
:
k¼1
g 41  a 1 5 rTf b T þ
i  i  ð1  aÞrf bi  gi
@ a a
c log ðli Þ Yðx; li Þ þ Yðx; li Þ
@x
XN
1 For the sake of simplicity, we introduce new parameters
¼ wk Wðlk Þ½Yðx; lk Þ þ Yðx; lk Þ ; (47)
k¼1
l k 1a 1
u¼ and h ¼ ;
subject to: a a
(
Y ð0; li Þ ¼ Y ð0; li Þ; and the system of equations becomes
(48) " # " #" #
Y ða; li Þ ¼ ð1  aÞY ða; li Þ þ gi ;
gþ
i h h rT0 b T þ
i  r0 bi
¼ ;
where gi ¼ a2 log2 li  12 . To simplify the notation, we suppress the g
i u h rTf b T þ
i  ð1  aÞrf bi  gi

x and l dependency and write the terms for the positive and negative
and thus
molecular velocity as Yþ and Y, respectively. Thus, our problem
becomes: (  T þ T 
gþ
i ¼ urf  hr0 bi þ hðr0  rf Þ bi þ hgi ;
XN  T  (53)
@ 1    T þ
gi ¼ uðrf  r0 Þ bi þ ur0  hrf bi þ hgi :
c log ðli Þ Yiþ þ Yiþ ¼ wk Wðlk Þ Ykþ þ Yk ; (49)
@x k¼1
lk
Replacing them in the constrained expressions, we get
@ XN
1  
c log ðli Þ Yi þ Yi ¼ wk Wðlk Þ Ykþ þ Yk ; (50)  T
@x k¼1
lk Yiþ ¼ r þ urf  hr0 bþ T 
i þ hðr0  rf Þ bi þ hgi ;
  T 
(54)
subject to Yi ¼ uðrf  r0 ÞT bþ
i þ r þ ur0  hrf bi þ hgi ;
(
Y0þ ¼ Y0 ; and replacing the constrained expressions in the equations of our
(51)
Yf ¼ ð1  aÞYfþ þ gi : problem, we obtain

Our constrained expressions, for positive and negative terms, are  T

c log ðli Þ r0 þ r þ urf  hr0 bþ T 
i þ hðr0  rf Þ bi
Y 6 ðxÞ ¼ g 6 ðxÞ þ g6 ; (52) X N
1 h T
and according to the boundary conditions, we have  wk Wðlk Þ r þ 2urf  hur0 bþ k
k¼1
l k

Y0þ ¼ rT0 bþ þ gþ ; Yf ¼ rTf b þ g ;  T i X

N
þ r þ hur0  2hrf bk ¼ h gi þ 2wk Wk hgk ; (55)
Yfþ ¼ rTf bþ þ gþ ; Y0 ¼ rT0 b þ g : k¼1

88
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-7
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

  
uðrf  r0 ÞT bþ 0
i þ c log ðli Þ r þ r þ ur0  hrf bi A½ð2MNÞð2LNÞ ;
XN
1 h T where M is the discretization order for s, and L is the number of neu-
 wk Wðlk Þ r þ 2urf  hur0 bþ k rons of the shallow NN.
k¼1
lk
That is, the original BVP has been reformulated as an uncon-
 T i X
N
strained optimization problem. In this case, according to the ELM
þ r þ hur0  2hrf bk ¼ h gi þ 2wk Wk hgk : (56)
algorithm, the output weights of the network b’s are calculated with
k¼1
only one direct pass via least squares methods, as the original problem
For the sake of simplicity, we write the inhomogeneous terms as is linear.58 That is,
X
N 1
bþ  b ¼ ðAT AÞ AT B: (59)
i ¼ bi ¼ hgi þ 2wk Wk hgk :
k¼1 As can be seen from Eq. (59) the unknowns are computed using a clas-
Expanding the summations, we get the following matrix form: sic least-square method. The authors have tried all the least squares
" #" # " # methods suggested in Appendix A of Ref. 59, finding the classic one
䉬k þ ?i “k þ |i bþ bþ the most convenient in terms of accuracy and computational time.
i
¼ i (57)
䉬k þ €i “k þ 䉲i bi
bi Once the output weights b’s are computed, the final solutions are built
according to (52). To be noticed that, although the output weights are
where,
evaluated on specific values of s (e.g., training points), once the b’s are
?i ¼ clogðli Þr0 T þ rT þ urf T  hr0 T known the solution can be computed in any desired query points with-
out further manipulation (e.g., post-processing or interpolation). That
䉲i ¼ clogðli Þr0 T þ rT þ ur0 T  hrf T is, once the solution is learned on the training points, we have an ana-
€k ¼ uðrf  r0 ÞT lytical representation (by NNs) of it. Therefore, it can be evaluated
|k ¼ hðr0  rf ÞT everywhere else in the channel with no additional computational
efforts. For example, we do not have to retrain the network and/or use
1
䉬k ¼ wk Wðlk Þðr þ 2urf  hur0 ÞT a separate interpolation routine. We simply evaluate the solution
lk learned on the training points in the desired new point in the channel.
1 For the convenience of the reader, the all process described in
“k ¼ wk Wðlk Þðr þ hur0  2hrf ÞT
lk this section is summarized in the schematic of Fig. 5.
Thus, the problem reduce to the system in Fig. 4, which is a problem VI. RESULTS AND DISCUSSIONS
of the type To generate the results, we coded the problem in the Matlab
Ab ¼ B; (58) R2019a platform and ran all our simulations with an Intel Core i7–9700
whose the dimensions are CPU PC with 64 GB of RAM. To demonstrate the accuracy of the pro-
posed PINN TFC-based method in solving the thermal creep problem,
we report the macroscopic velocity profile in a inverse Knudsen number
b6 i ½L1 b½2LN1
range of ½0:05; 9:00, that according to Refs. 20, 54, and 81 is given by
b6 i ½M1 B½2MN1 ð1
qðsÞ ¼ WðlÞYðs; lÞdl ¼ Y0 ðsÞ for s 2 ½0; a; (60)
?i ½ML ðsame dimension for the other terms of AÞ 1

FIG. 4. Linear system solved via least-squares.

89
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-8
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

FIG. 5. Schematic of the PINN TFC-based algorithm to solve thermal creep flow problem in BGK approximation.

where Y0 ðsÞ is given by81 Figure 6 shows the qualitative behavior of the macroscopic velocity in
ð1 a half channel for different values of a, computed with all the three dif-
Y0 ðsÞ ¼ WðlÞYðs; lÞdl; (61) ferent types of NNs tested. The accommodation coefficient quantifies
1 the interaction between a fluid and a solid (in this case the wall). It
which it can be computed as indicates the degree to which molecules are accommodated to the sur-
X
N   face.82 Thus, the macroscopic velocity increases for increasing accom-
Y0 ¼ wi Wðli Þ Yiþ þ Yi : (62) modation coefficient a. Hence, Fig. 6 proves that all the NNs tested are
i¼1 able to capture the qualitative behavior of the macroscopic velocity.
In Table I we reported the values of the macroscopic velocity pro-
file q, at s ¼ 0 and s ¼ a, with accommodation coefficient a ¼ 1. The
results were computed with different type of shallow NNs as free-func-
tion: standard shallow NNs (using hyperbolic tangent as activation
function), ChNN, and LeNN (we reported in bold the digits that differ
from the benchmarks published by Ganapol54). The results we
obtained using the following parameters: N ¼ 30, M ¼ 200, 20 neurons
for the standard shallow NN, and 74 for ChNN and LeNN. The train-
ing time was on the order of seconds for all the networks. Although
we achieved good results (both qualitatively and quantitatively) with
all the networks, with ChNN and LeNN we were able to match all the
digits of the benchmarks published by Ganapol.54

TABLE I. Macroscopic velocity profile q computed with different type of shallow NNs
as free-function, for a channel half-thickness a ¼ 0.5.

Neural network qðs ¼ 0Þ qðs ¼ aÞ

1
Standard 1.974 216 9  10 1.468 959 0  101
ChNN 1.974 209 2  101 1.468 959 4  101
LeNN 1.974 209 2  101 1.468 959 4  101
FIG. 6. Macroscopic velocity profiles in a half channel for different values of a.

90
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-9
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

ða ða
In Tables II and III, the results computed with ChNN and LeNN, 1 1
Q¼ qðsÞds ¼  Y0 ðsÞds; (63)
for different values of the accommodation number at center of channel 2a2 a a2 0
s ¼ 0 and at the wall s ¼ a, are reported. All our digits match with the
benchmarks reported in Ref. 54. Furthermore, we computed the flow where the results, computed with ChNN and LeNN, are reported in
rate Table IV. The chosen parameters and the computational time for each

TABLE II. The macroscopic velocity profile qða; aÞ for s ¼ 0. All the digits match the benchmark published by Ganapol.54

2a a ¼ 0:50 a ¼ 0:80 a ¼ 0:88 a ¼ 0:96 a ¼ 1:00

0.05 4.222 500 3  102 2.822 852 1  102 2.572 389 8  102 2.352 962 1  102 2.252 976 7  102
0.1 6.509 553 8  102 4.571 936 2  102 4.214 646 6  102 3.898 974 2  102 3.754 310 5  102
0.3 1.187 663 9  101 9.267 451 1  102 8.752 494 4e  102 8.288 705 5  102 8.073 408 3  102
0.5 1.504 850 8  101 1.248 201 3  101 1.195 421 8  101 1.147 314 3  101 1.124 801 4  101
0.7 1.729 512 0  101 1.498 255 6  101 1.449 292 7  101 1.404 273 7  101 1.383 081 9  101
0.9 1.901 655 4  101 1.703 262 7  101 1.660 349 2  101 1.620 635 1  101 1.601 858 0  101
1.0 1.974 209 2  101 1.793 256 3  101 1.753 784 8  101 1.717 162 8  101 1.699 817 9  101
2.0 2.439 083 9  101 2.420 532 0  101 2.417 017 5  101 2.413 990 6  101 2.412 644 8  101
5.0 2.931 138 2  101 3.168 530 9  101 3.229 393 6  101 3.289 278 1  101 3.318 861 4  101
7.0 3.047 934 0  101 3.354 423 3  101 3.433 559 0  101 3.511 647 6  101 3.550 306 2  101
9.0 3.108 636 2  101 3.451 505 4  101 3.540 327 0  101 3.628 093 0  101 3.671 587 0  101

TABLE III. The macroscopic velocity profile qða; aÞ for s ¼ a. All the digits match the benchmark published by Ganapol.54

2a a ¼ 0:50 a ¼ 0:80 a ¼ 0:88 a ¼ 0:96 a ¼ 1:00

0.05 3.911 603 2  102 2.390 993 9  102 2.110 893 4  102 1.862 664 4  102 1.748 562 7  102
0.1 5.880 243 5  102 3.700 835 1  102 3.284 860 8  102 2.912 348 4  102 2.739 890 6  102
0.3 1.004 132 6  101 6.702 653 1  102 6.013 395 6  102 5.381 871 5  102 5.084 945 4  102
0.5 1.214 816 5  101 8.373 539 5  102 7.557 561 3  102 6.799 631 4  102 6.439 969 8  102
0.7 1.345 641 9  101 9.479 089 9  102 8.591 182 3  102 7.758 311 5  102 7.360 456 6  102
0.9 1.434 574 8  101 1.026 763 3  101 9.335 172 6  102 8.453 946 8  102 8.030 847 2  102
1.0 1.468 959 4  101 1.058 190 1  101 9.633 441 4  102 8.734 303 1  102 8.301 685 5  102
2.0 1.643 019 4  101 1.227 565 4  101 1.126 192 6  101 1.028 313 2  101 9.806 182 8  102
5.0 1.737 460 7  101 1.330 060 5  101 1.227 130 6  101 1.126 479 4  101 1.076 988 9  101
7.0 1.746 701 8  101 1.340 824 0  101 1.237 910 6  101 1.137 133 7  101 1.087 530 3  101
9.0 1.749 563 0  101 1.344 208 9  101 1.241 314 3  101 1.140 511 0  101 1.090 878 3  101

TABLE IV. The flow rate Qða; aÞ. All the digits match the benchmark published by Ganapol.54

2a a ¼ 0:50 a ¼ 0:80 a ¼ 0:88 a ¼ 0:96 a ¼ 1:00

0.05 1.653 688 8 1.080 865 1 9.775 525 3  101 8.867 589 5  101 8.452 892 6  101
0.1 1.266 441 6 8.659 804 7  101 7.914 281 9  101 7.253 123 2  101 6.949 271 6  101
0.3 7.580 823 6  101 5.712 072 1  101 5.338 214 3  101 4.999 735 7  101 4.841 992 5  101
0.5 5.705 722 9  101 4.551 663 9  101 4.310 382 9  101 4.089 057 3  101 3.984 992 8  101
0.7 4.649 665 6  101 3.864 581 3  101 3.695 099 4  101 3.538 098 0  101 3.463 780 9  101
0.9 3.953 836 9  101 3.392 428 5  101 3.268 193 4  101 3.152 202 1  101 3.097 001 1  101
1.0 3.685 434 6  101 3.205 049 0  101 3.097 629 6  101 2.997 000 5  101 2.948 999 2  101
2.0 2.245 046 2  101 2.129 203 2  101 2.101 613 5  101 2.075 211 6  101 2.062 428 8  101
5.0 1.075 322 0  101 1.116 569 5  101 1.127 116 1  101 1.137 481 4  101 1.142 597 5  101
7.0 8.036 290 7  102 8.533 480 4  102 8.661 542 2  102 8.787 780 4  102 8.850 228 2  102
9.0 6.421 908 1  102 6.907 720 3  102 7.033 297 6  102 7.157 269 6  102 7.218 664 1  102

91
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-10
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

TABLE V. Parameters used to compute the flow rate Q(a) and the macroscopic VII. CONCLUSIONS
velocity profile qðsÞ for each channel width.
We have developed and successfully applied a PINN TFC-based
2a N M m Comp. time (s) method to solving the thermal creep flow in a plane channel problem.
The proposed method is proved to be accurate and efficient. To check
0.05 69 100 34 1.35 the accuracy of our solutions, we have compared them with the bench-
0.10 59 120 54 2.39 marks published by Loyalka et al.,20 Barichello et al.,81 and Ganapol,54
0.30 35 140 60 0.78 matching their results up to the fifth and seventh digits.
0.50 30 140 60 0.51 The results of this paper show the feasibility of the proposed
methodology and set the stage for the application of PINNs to obtain
0.70 30 140 60 0.52
accurate solutions for a large variety of problems arising from the
0.90 30 200 67 0.81 Boltzmann linear integrodifferential equation. We are currently work-
1.00 30 200 74 0.92 ing on applying the proposed PINN-TFC based method to solving
2.00 24 400 90 1.30 more RGD problems such as Poiseuille flow in a plane channel83 as
5.00 24 700 130 4.31 formulated by Barichello et al.84,85 Moreover, we are currently working
7.00 24 900 150 6.92 on solving high-dimensional problems with the X-TFC algorithm,56
9.00 24 1400 150 10.12 which has been proved being a significant improvement for the PINN
frameworks for estimating solutions of differential equations,52 in par-
ticular, for PDEs.
channel width are reported in Table V. The parameters reported in the Future works then will focus on the application of this new
Table V are also plotted in Fig. 7 to better appreciate their variations method to solve 3D time-dependent problems arising from the
with respect the channel thickness 2a. As it can be seen from Tables Boltzmann linear integrodifferential equation, in rarefied gas dynam-
II–IV, our results match all the digits reported in the published ics, radiative transfer, and neutron transport problems.
benchmarks.54
ACKNOWLEDGMENTS
The authors would like to acknowledge Mr. Andrea
D’Ambrosio for his precious advise that helped to improve this
manuscript.
DATA AVAILABILITY
The data that support the findings of this study are available
from the corresponding author upon reasonable request.

REFERENCES
1
O. Reynolds, “Xviii. On certain dimensional properties of matter in the gaseous
state.-Part I. Experimental researches on thermal transpiration of gases through
porous plates and on the laws of transpiration and impulsion, including an
experimental proof that gas is not a continuous plenum.-Part II. On an exten-
sion of the dynamical theory of gas, which includes the stresses, tangential and
normal, caused by a varying condition of gas, and affords an explanation of the
phenomena of transpiration and impulsion,” Philos. Trans. R. Soc. London
170, 727–845 (1879).
2
J. C. Maxwell, “III. On stresses in rarefied gases arising from inequalities of
temperature,” Proc. R. Soc. London 27(185-189), 304–308 (1878).
3
M. Knudsen, “Thermischer molekulardruck der gase in r€ohren,” Ann. Phys.
338(16), 1435–1448 (1910).
4
T. Edmonds and J. Hobson, “A study of thermal transpiration using ultrahigh-
vacuum techniques,” J. Vac. Sci. Technol. 2(4), 182–197 (1965).
5
J. Hobson, “Surface smoothness in thermal transpiration at very low pressur-
es,” J. Vac. Sci. Technol. 6(1), 257–259 (1969).
6
R. Healy and T. Storvick, “Rotational collision number and Eucken factors
from thermal transpiration measurements,” J. Chem. Phys. 50(3), 1419–1427
(1969).
7
A. D. Gupta and T. Storvick, “Analysis of the heat conductivity data for polar
and nonpolar gases using thermal transpiration measurements,” J. Chem.
Phys. 52(2), 742–749 (1970).
8
B. Annis, “Thermal creep in gases,” J. Chem. Phys. 57(7), 2898–2905 (1972).
9
H. Niimi, “Thermal creep flow of rarefied gas between two parallel plates,”
J. Phys. Soc. Jpn. 30(2), 572–574 (1971).
10
FIG. 7. Variations of the chosen parameters and computational times in function of C. Cercignani, “The Boltzmann equation,” The Boltzmann Equation and Its
the channel width. Applications (Springer, 1988), pp. 40–103.

92
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-11
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

11 38
J. Ferziger and H. Kaper, “Mathematical theory of transport processes in gases,” K.-Y. Chien, “Application of the s, method to spherically symmetric radiative-
Am. J. Phys. 41(4), 601–603 (1973). transfer problems,” AIAA J. 10(1), 55–59 (1972).
12 39
F. Sharipov, Rarefied Gas Dynamics: Fundamentals for Research and Practice M. Simch, C. Segatto, and M. Vilhena, “An analytical solution for the Sn radia-
(John Wiley & Sons, 2015). tive transfer equation with polarization in a slab by the LTSN method,”
13
C. Cercignani, Theory and Application of the Boltzmann Equation (Scottish J. Quant. Spectrosc. Radiat. Transfer 97(3), 424–435 (2006).
40
Academic Press, 1975). P. Benoist and A. Kavenoky, “A new method of approximation of the
14
P. L. Bhatnagar, E. P. Gross, and M. Krook, “A model for collision processes in Boltzmann equation,” Nucl. Sci. Eng. 32(2), 225–232 (1968).
gases. I. Small amplitude processes in charged and neutral one-component sys- 41
C. Siewert and P. Benoist, “The FN method in neutron-transport theory. Part I:
tems,” Phys. Rev. 94(3), 511 (1954). Theory and applications,” Nucl. Sci. Eng. 69(2), 156–160 (1979).
15 42
E. Shakhov, “Generalization of the Krook kinetic relaxation equation,” Fluid C. Devaux and C. Siewert, “The FN method for radiative transfer problems
Dyn. 3(5), 95–96 (1972). without azimuthal symmetry,” Z. Angew. Math. Phys. 31(5), 592–604
16
S. Loyalka, “Thermal transpiration in a cylindrical tube,” Phys. Fluids 12(11), (1980).
2301–2305 (1969). 43
R. Garcia and C. Siewert, “The FN method for radiative transfer models that
17
S. Loyalka and T. Storvick, “Kinetic theory of thermal transpiration and mecha- include polarization effects,” J. Quant. Spectrosc. Radiat. Transfer 41(2),
nocaloric effect. III. Flow of a polyatomic gas between parallel plates,” J. Chem. 117–145 (1989).
Phys. 71(1), 339–350 (1979). 44
B. D. Ganapol and R. Myneni, “The FN method for the one-angle radiative
18
S. Loyalka, “Comments on ‘Poiseuille flow and thermal creep of a rarefied gas transfer equation applied to plant canopies,” Remote Sens. Environ. 39(3),
between parallel plates’,” Phys. Fluids 17(5), 1053–1055 (1974). 213–231 (1992).
19
V. G. Chernyak, B. T. Porodnov, and P. E. Suetin, “Application of the varia- 45
M. Benassi, R. Garcia, A. Karp, and C. Siewert, “A high-order spherical har-
tional method to the problem of thermomolecular pressure difference in a monics solution to the standard problem in radiative transfer,” Astrophys. J.
cylindrical capillary,” J. Eng. Phys. 26(3), 309–312 (1974). 280, 853–864 (1984).
20
S. Loyalka, N. Petrellis, and T. Storvick, “Some exact numerical results for the 46
C. Siewert and J. Thomas, Jr., “A particular solution for the PN method in
BGK model: Couette, Poiseuille and thermal creep flow between parallel radiative transfer,” J. Quant. Spectrosc. Radiat. Transfer 43(6), 433–436
plates,” Z. Angew. Math. Phys. 30(3), 514–521 (1979). (1990).
21
D. Valougeorgis and J. Thomas, Jr., “Exact numerical results for Poiseuille and 47
L. Barichello and C. E. Siewert, “A discrete-ordinates solution for a non-grey
thermal creep flow in a cylindrical tube,” Phys. Fluids 29(2), 423–429 (1986). model with complete frequency redistribution,” J. Quant. Spectrosc. Radiat.
22
S. Loyalka and K. Hickey, “Kinetic theory of thermal transpiration and the Transfer 62(6), 665–675 (1999).
mechanocaloric effect: Planar flow of a rigid sphere gas with arbitrary accom- 48
C. Siewert, “A concise and accurate solution to Chandrasekhar’s basic problem
modation at the surface,” J. Vac. Sci. Technol. A 9(1), 158–163 (1991). in radiative transfer,” J. Quant. Spectrosc. Radiat. Transfer 64(2), 109–130
23
F. Sharipov, “Rarefied gas flow through a long tube at any temperature ratio,”
(2000).
J. Vac. Sci. Technol. A 14(4), 2627–2635 (1996). 49
24 P. Picca and R. Furfaro, “Analytical discrete ordinate method for radiative
K. Ritos, Y. Lihnaropoulos, S. Naris, and D. Valougeorgis, “Pressure-and
transfer in dense vegetation canopies,” J. Quant. Spectrosc. Radiat. Transfer
temperature-driven flow through triangular and trapezoidal microchannels,”
118, 60–69 (2013).
Heat Transfer Eng. 32(13-14), 1101–1107 (2011). 50
25 B. Ganapol, “Radiative transfer with internal reflection via the converged dis-
T. Ohwada, Y. Sone, and K. Aoki, “Numerical analysis of the shear and thermal
crete ordinates method,” J. Quant. Spectrosc. Radiat. Transfer 112(4), 693–713
creep flows of a rarefied gas over a plane wall on the basis of the linearized
(2011).
Boltzmann equation for hard-sphere molecules,” Phys. Fluids A 1(9), 51
P. Picca, R. Furfaro, and B. D. Ganapol, “An efficient multiproblem strategy for
1588–1599 (1989).
26 accurate solutions of linear particle transport problems in spherical geometry,”
T. Kanki and S. Iuchi, “Poiseuille flow and thermal creep of a rarefied gas
Nucl. Sci. Eng. 170(2), 103–124 (2012).
between parallel plates,” Phys. Fluids 16(5), 594–599 (1973). 52
27 M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural net-
G. A. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows
(Clarendon Press, 1994). works: A deep learning framework for solving forward and inverse problems
28
H. Akhlaghi, M. Balaj, and E. Roohi, “Hydrodynamic behaviour of micro/ involving nonlinear partial differential equations,” J. Comput. Phys. 378,
nanoscale Poiseuille flow under thermal creep condition,” Appl. Phys. Lett. 686–707 (2019).
53
103(7), 073108 (2013). E. Schiassi, A. D’Ambrosio, M. De Florio, R. Furfaro, and F. Curti, “Physics-
29
H. Akhlaghi, E. Roohi, and M. Balaj, “A thorough study on thermal mass flux informed extreme theory of functional connections applied to data-driven
of rarefied flow through micro/nanochannels,” Appl. Phys. Lett. 104(7), parameters discovery of epidemiological compartmental models,”
073109 (2014). arXiv:2008.05554 (2020).
54
30
H. Akhlaghi and E. Roohi, “Mass flow rate prediction of B. D. Ganapol, “1D thermal creep channel flow in the BGK approximation by
pressure–temperature-driven gas flows through micro/nanoscale channels,” adding and doubling,” Ann. Nucl. Energy 134, 441–451 (2019).
55
Continuum Mech. Thermodyn. 26(1), 67–78 (2014). C. Leake and D. Mortari, “Deep theory of functional connections: A new
31
V. Shahabi, T. Baier, E. Roohi, and S. Hardt, “Thermally induced gas flows in method for estimating the solutions of partial differential equations,” Mach.
ratchet channels with diffuse and specular boundaries,” Sci. Rep. 7(1), 41412 Learn. Knowl. Extr. 2(1), 37–55 (2020).
56
(2017). E. Schiassi, C. Leake, M. De Florio, H. Johnston, R. Furfaro, and D. Mortari,
32 “Extreme theory of functional connections: A physics-informed method for
C. Tantos, “Polyatomic thermal creep flows through long microchannels at
large temperature ratios,” J. Vac. Sci. Technol. A 37(5), 051602 (2019). solving parametric differential equations,” arXiv:2005.10632 (2020).
33 57
F. Hanson and T. Morse, “Kinetic models for a gas with internal structure,” D. Mortari, “The theory of connections: Connecting points,” Mathematics
Phys. Fluids 10(2), 345–353 (1967). 5(4), 57 (2017).
34 58
I. Chemyaninov, V. Chernysk, and G. Fomyagin, “Heat and mass transfer pro- D. Mortari, “Least-squares solution of linear differential equations,”
cesses of a polyatomic gas in a flat channel for arbitrary Knudsen numbers,” Mathematics 5(4), 48 (2017).
59
J. Appl. Mech. Tech. Phys. 25(6), 881–888 (1984). D. Mortari, H. Johnston, and L. Smith, “High accuracy least-squares solutions
35 of nonlinear differential equations,” J. Comput. Appl. Math. 352, 293–307
B. Carlson and G. Bell, “Solution of the transport equation by the Sn method,”
Technical Report No. 15/P/2386 (Los Alamos Scientific Lab, New Mexico, 1958). (2019).
36 60
S. Chandrasekhar, Radiative Transfer (Courier Corporation, 2013). M. De Florio, E. Schiassi, R. Furfaro, B. D. Ganapol, and D. Mostacci,
37
M. Vilhena, C. Segatto, and L. Barichello, “A particular solution for the Sn radi- “Solutions of Chandrasekhar’s basic problem in radiative transfer via theory of
ative transfer problems,” J. Quant. Spectrosc. Radiat. Transfer 53(4), 467–469 functional connections,” J. Quant. Spectrosc. Radiat. Transfer 259, 107384
(1995). (2021).

93
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-12
Published under license by AIP Publishing
 Physics of Fluids ARTICLE scitation.org/journal/phf

61 73
R. Furfaro and D. Mortari, “Least-squares solution of a class of optimal space Q.-H. Sun, C.-P. Cai, and W. Gao, “On the validity of the Boltzmann-BGK
guidance problems via theory of connections,” Acta Astronaut. 168, 92 (2020). model through relaxation evaluation,” Acta Mech. Sin. 30, 133–143 (2014).
62 74
H. Johnston, E. Schiassi, R. Furfaro, and D. Mortari, “Fuel-efficient powered C. H. Kruger and W. Vincenti, Introduction to Physical Gas Dynamics (John
descent guidance on large planetary bodies via theory of functional con- Wiley & Sons, 1965).
75
nections,” arXiv:2001.03572 (2020). S. Chapman and T. Cowling, The Mathematical Theory of Non-Uniform Gases
63
E. Schiassi, A. D’Ambrosio, H. Johnston, R. Furfaro, F. Curti, and D. Mortari, (Cambridge University Press, Cambridge, England, 1970).
76
“Complete energy optimal landing on planetary bodies via theory of functional Y. Sone, Kinetic Theory and Fluid Dynamics (Springer Science & Business
connections,” in Proceedings of the Astrodynamics Specialist Conference, Media, 2002).
77
AAS, pp. 20–557. L. H. Holway, Jr., “Kinetic theory of shock structure using an ellipsoidal distri-
64
M. Liu, M. Hou, J. Wang, and Y. Cheng, “Solving two-dimensional linear par- bution function,” Rarefied Gas Dyn. 1, 193 (1965).
78
tial differential equations based on Chebyshev neural network with extreme M. Williams, “A review of the rarefied gas dynamics theory associated with
learning machine algorithm,” Eng. Comput. 38, 874 (2020). some classical problems in flow and heat transfer,” Z. Angew. Math. Phys.
65
Y. Yang, M. Hou, and J. Luo, “A novel improved extreme learning machine 52(3), 500–516 (2001).
79
algorithm in solving ordinary differential equations by Legendre neural net- N. G. Hadjiconstantinou, “The limits of Navier-Stokes theory and kinetic
work methods,” Adv. Differ. Equations 2018(1), 469. extensions for describing small-scale gaseous hydrodynamics,” Phys. Fluids
66
G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: Theory 18(11), 111301 (2006).
80
and applications,” Neurocomputing 70, 489–501 (2006). C. Cercignani, “Plane Poiseuille flow according to the method of elementary
67
J. N. Reddy, “An introduction to the finite element method,” J. Pressure Vessel solutions,” J. Math. Anal. Appl. 12(2), 254–262 (1965).
81
Technol. 111, 348–349 (1989). L. Barichello, M. Camargo, P. Rodrigues, and C. Siewert, “Unified solutions to
68
A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, “Automatic differ- classical flow problems based on the BGK model,” Z. Angew. Math. Phys.
entiation in machine learning: A survey,” J. Mach. Learn. Res. 18 (2018). 52(3), 517–534 (2001).
69 82
L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis, “DeepXDE: A deep learning S. K. Prabha and S. P. Sathian, “Molecular-dynamics study of Poiseuille flow in
library for solving differential equations,” arXiv:1907.04502 (2019). a nanochannel and calculation of energy and momentum accommodation
70
M. De Florio, E. Schiassi, B. D. Ganapol, and R. Furfaro, “An accurate solution coefficients,” Phys. Rev. E 85(4), 041201 (2012).
83
for Poiseuille flow in a plane channel via theory of functional connections” F. Sharipov, “Application of the Cercignani–Lampis scattering kernel to calcu-
(unpublished). lations of rarefied gas flows. I. Plane flow between two parallel plates,” Eur. J.
71
C. Cercignani, Rarefied Gas Dynamics: From Basic Concepts to Actual Mech.-B 21(1), 113–123 (2002).
84
Calculations (Cambridge University Press, 2000), Vol. 21. L. Barichello and C. Siewert, “A discrete-ordinates solution for Poiseuille flow
72
C. Siewert, “Poiseuille, thermal creep and Couette flow: Results based on the in a plane channel,” Z. Angew. Math. Phys. 50(6), 972–981 (1999).
85
CES model of the linearized Boltzmann equation,” Eur. J. Mech.-B 21(5), C. Siewert, R. Garcia, and P. Grandjean, “A concise and accurate solution for
579–597 (2002). Poiseuille flow in a plane channel,” J. Math. Phys. 21(12), 2760–2763 (1980).

94
Phys. Fluids 33, 047110 (2021); doi: 10.1063/5.0046181 33, 047110-13
Published under license by AIP Publishing
Appendix D

De Florio et al. (2022) Poiseuille Flow

Reproduced from De Florio, M., Schiassi, E., Ganapol, B. D., & Furfaro, R. (2022). Physics-
Informed Neural Networks for rarefied-gas dynamics: Poiseuille flow in the BGK approxima-
tion. Zeitschrift für angewandte Mathematik und Physik, 73(3), 1-18. https://doi.org/10.1007/s00033-
022-01767-z [10], with the permission of Springer Nature.

95
Z. Angew. Math. Phys. (2022) 73:126
c 2022 The Author(s), under exclusive licence to Springer Nature
Switzerland AG Zeitschrift für angewandte
https://doi.org/10.1007/s00033-022-01767-z Mathematik und Physik ZAMP

Physics-Informed Neural Networks for rarefied-gas dynamics: Poiseuille flow

in the BGK approximation
Mario De Florio , Enrico Schiassi , Barry D. Ganapol and Roberto Furfaro

Abstract. We present a new accurate approach to solving a class of problems in the theory of rarefied–gas dynamics using
a Physics-Informed Neural Networks framework, where the solution of the problem is approximated by the constrained
expressions introduced by the Theory of Functional Connections. The constrained expressions are made by a sum of a free
function and a functional that always analytically satisfies the equation constraints. The free function used in this work is a
Chebyshev neural network trained via the extreme learning machine algorithm. The method is designed to accurately and
efficiently solve the linear one-point boundary value problem that arises from the Bhatnagar–Gross–Krook model of the
Poiseuille flow between two parallel plates for a wide range of Knudsen numbers. The accuracy of our results is validated
via the comparison with the published benchmarks.

Mathematics Subject Classification. 76P05, 68T07, 35Q20.

Keywords. Physics-Informed Neural Networks, Extreme learning machine, Functional interpolation, Rarefied gas dynamics,
Poiseuille flow, Boltzmann equation.

1. Introduction

Rarefied–gas dynamics (RGD) has been widely studied in fluid dynamics, both from theoretical and
experimental perspectives. Gas is considered in a rarefied state when its Knudsen number, defined by the
ratio of the mean free path to the chamber under test, is greater than one. The behavior of the particles of
a rarefied gas can be described through the Boltzmann equation. In particular, a Poiseuille flow is a case
of motion of a gas induced by a pressure gradient along the walls of the channel. The Poiseuille problem
in the RGD has been initially theoretically studied by Reynolds [1] and Maxwell [2]. Experimental studies
[3,4] followed, where a minimum in the flow rate has been observed in the transition region (as a function
of the pressure Knudsen number). Many researchers applied several methods to solve the class of linear
one-point boundary value problems arising from the Boltzmann integrodifferential equation for a rarefied
gas, such as [5–7], and [8]. The fundamentals of kinetic theory [9–11] have been used to investigate
the Poiseuille flow problem in the field of RGD from a theoretical point of view to find solutions for
a large interval of Knudsen number. The Boltzmann equation has been the most used fundamental
mathematical model to describe the Poiseuille flow [12], and different models and approximations have
been developed, such as the Bhatnagar–Gross–Krook (BGK) model [13] and the Shakhov model [14].
In the years, many numerical methods have been employed to solve such problems in RGD [5,6,15–23].
One of the state-of-the-art methods to tackle these problems is the variation of the discrete-ordinates
method (DOM) of Chandrasekhar [24,25]. During the last decays, this family of methods has been
widely improved and used to solve the Boltzmann integrodifferential transport equation in many fields,
such as RGD and transport of neutrons and photons. We can mention the Sn [26–28], the CN [29],
PN , and FN methods [30–35]. Barichello et al. [36,37] used it to tackle the Poiseuille flow in a plane
channel problem, generating accurate results used as benchmarks. Siewert [38] applied the ADO method
in solving the basic Radiative transfer equation. Recently, Ganapol [39–41] has extended the benchmarks

0123456789().: V,-vol 96
126 Page 2 of 18 M. Florio et al. ZAMP

to additional digits. A new category of numerical methods in the machine learning community has been
developed, called Physics-Informed Neural Networks (PINNs). Such methods train neural networks (NNs)
in learning the solutions of differential equations (DEs). Nevertheless, PINNs can generally be considered
machine learning methods that embed physics into a data-driven representation of functional relationships
underlying collections of input–output pairs. As described by Raissi et al. [42], the term PINN defines
NNs that utilize physics as a regularizer in the loss function. For example, assume that one wants to
perform a regression of a set of experimental data using NNs and that such collected data represent some
physical phenomena represented by a set of DEs. One would approximate the data using NNs trained to
minimize a mean squared error as a loss function in standard regression. However, there is no guarantee
that the physics phenomena governing the data would not be violated. More precisely, especially when the
data is noisy, one could experience the issue of over-fitting. That is, the data would be accurately fitted
(e.g., with a negligible error between the NN approximation and the experimental data), but the relation
input–output, governed by the physics phenomena, could be poorly learned. To this matter, PINNs are
introduced to ensure that DEs, modeling the physics of the experimental dataset, are added as a penalty
to the loss function. This additional term acts as a regularizer that penalizes the training when the DE
and its constraints (e.g., the initial conditions ICs and/or boundary conditions BCs) are violated. Overall,
one guarantees that the physics governing the data is not violated. This approach is defined data-driven
solution of DEs. Conversely, when the goal is to estimate parameters governing some physical phenomena
modeled through a DE (e.g., the thermal conductivity in the heat equation), one refers to data-driven
parameters discovery of DEs (e.g., inverse problems) [42,43]. When data is not available, and hence the
loss function contains only the residual of the DEs (and its constraints), PINNs are trained for learning
the solutions of problems involving DEs. When data is not available, PINNs model a NN representation
of functions for which the residuals of the DEs are as close to zero as possible, i.e., PINNs model the
solutions of the DEs. To this end, PINNs can also be viewed as methods that approximate solutions of
DEs using NNs. In this paper, we consider the linearized BGK model suggested by Bhatnagar, Gross,
and Krook [13] to describe the physical problem of the Poiseuille flow [44,45] between two parallel plates
following the formulation presented by Williams [7]. Our task is to develop and evaluate a PINNs-based
framework to solve the Poiseuille flow between two parallel plates. Following the formulation proposed
by Ganapol [40], we demonstrate that our PINN-based approach is accurate in learning the solutions
of this class of problems arising from the Boltzmann integrodifferential equation. In particular, to solve
this problem, we develop a PINN framework that brings together NNs and the Theory of Functional
Connections (TFC), named Extreme Theory of Functional Connections (X-TFC) [46]. TFC, developed
by Mortari [47], is a functional interpolation technique for a large class of mathematical objects. According
to TFC, any mathematical problem’s solution can be represented via constrained expressions (CEs). The
CEs are made of a free function and a functional which analytically satisfies the problem constraints,
regardless of the choice of the free function. It can be expressed as

f (x)  fCE (x, g(x)) = A(x) + B(x, g(x)) (1.1)

where x ∈ Rn , f (x) is the unknown function to be interpolated, fCE (x, g(x)) is the CE, A(x) is the
functional that analytical satisfies any given linear constraint, and B(x, g(x)) projects the free function
g(x), which is a real function that needs to be defined on the constraints, onto the space of functions that
vanish at the constraints [48]. Among its many applications, TFC has been applied to solve ODEs with
usually excellent results, leading to machine-level accuracy solutions in a short computational time [49–
51]. The authors already employed PINN TFC-based methods to solve Chandrasekhar’s basic problem in
radiative transfer [52], thermal creep flow in the BGK approximation [53], optimal control problems for
aerospace applications [54–58]. Here, based on the results of De Florio et al. [53], we consider a X-TFC
approach where Chebyshev NNs (ChNNs) are employed as free function in the CE. ChNNs have been
initially proposed by Namatame [59] and successively adopted by Mall and Chakraverty [60] for solving
elliptic PDEs. They are a class of functional link artificial neural networks in which the hidden layer is

97
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 3 of 18 126

replaced by an expansion of Chebyshev polynomials as a functional expansion block for enhancement of

the input patterns. In this particular architecture of NN, the input weights and bias are selected to be
equal to one and zero, respectively. With this choice of input weights and bias, ChNN become equivalent
to a linear combination of Chebyshev polynomials, where the unknowns are the coefficients, similarly to
a collocation method [61]. Indeed, under the machine learning community prospective, according to the
definition of PINNs, collocation methods using orthogonal polynomials can be considered as a particular
family of PINNs. The ChNNs are trained via the extreme learning machine (ELM) algorithm [62,63]. In
the ELM algorithm, input weights and bias are randomly selected and not tuned during the training,
leaving the output weights the only trainable parameters. Hence, the training is reduced to a fast least
squares. The paper is organized as follows. In Sect. 2, the standard PINN methodology and how to apply
it to solve a generic first-order linear ODE are presented. In Sect. 3, how to apply PINN TFC-based
methods to tackle generic first-order linear ODEs is explained. The PINN TFC-based formulation of the
Poiseuille flow in a plane channel problem is shown in Sects. 4 and 5. Numerical results are presented
and discussed in Sect. 6. Conclusions are drawn in Sect. 7.

2. Physics-Informed Neural Networks

PINNs have been introduced by Raissi et al. [42], following the original idea proposed by Lagaris et al. [64],
to solve two main classes of problems: data-driven solution and data-driven discovery of PDEs. Although
in [42] PINNs were presented to face problems governed by PDEs, the same framework can be used to
tackle problems involving ODEs as well. PINNs use a single deep neural network (DNN) to approximate
the equation solution fully. In particular, the training set is made of randomly parsed data points from
the entire high-resolution dataset and randomly sampled collocation points for enforcing the equation
constraints (e.g., initial and/or boundary conditions) and the differential equation (DE) on the boundary
and inside the solution domain, respectively. Once the output of DNN is computed, it is plugged in the
residual of the DE and into the residual of the equation constraints. The NN is then trained to learn the
weights and biases parameters that minimize the physics-based cost function, which can be DE and its
constraints least squares solved with gradient-based optimization techniques. Thus, the key feature of the
PINN is the embedding of the information of the physics of the problem, derived from the DE and its
constraints, in their cost functions. To better clarify how a PINN works, the mathematical formulation
to solve a first-order linear ODE is reported in detail in Ref. [53]. PINNs, as described above, are the
standard PINN frameworks [42]. One of the main shortcomings of the standard PINN frameworks is that
the constraints of the equation are not analytically satisfied. Thus, there are competing objectives during
the PINN training: learning the DE solution within the domain and learning its constraints. This process
leads to have unbalanced gradients during the network training via gradient-based techniques that causes
PINNs to have often difficulties approximating the DE solution accurately [65]. Indeed, gradient-based
optimization techniques may get stuck in limit cycles or diverge if multiple competing objectives are
present [66,67]. In Ref. [65], to surmount this limitation, the authors proposed a learning rate annealing
algorithm that employs gradient statistics to adaptively assign proper weights to different terms (e.g.,
DE residuals within the domain and on the boundaries) in the PINNs loss function while the network is
training. The approach employed here is different and is based on the TFC [47] which allows satisfying the
equation constraints analytically, hence completely removing the competing objective in the NN training.
The following section provides an overview of our PINN TFC-based approach along with the proposed
learning algorithm.

98
126 Page 4 of 18 M. Florio et al. ZAMP

3. X-TFC approach to solving first-order linear ODEs

As previously mentioned, we develop an improved PINN framework, that we have already successfully
applied to solve the thermal creep flow in the BGK approximation [53]. In the proposed approach, we
merge the standard PINN with the TFC method [46,48].Here, we present the step-by-step procedure for
solving boundary value linear ODEs via the proposed X-TFC method. Consider solving the following
first-order ODE:
F (t, y, ẏ) = 0 subject to: y(t0 ) = y0 (3.1)
The unknown solution is approximated via the constrained expression, defined within the TFC framework.
The general form of the constrained expression of Eq. (1.1) can be rewritten for a univariate problem
(e.g., an ODE) in the following simpler form:
n
y(t) = g(t) + ηk pk (t) = g(t) + η T p(t) (3.2)
k=1
where pk (t) are n assigned linearly independent functions and the g(t) is the free function, which must
be linearly independent from the pk (t) functions. The values of the ηk coefficients are calculated by
imposing a set of n linear constraints in y(t). By doing that, the constraints for the differential equations
are satisfied for any chosen g(t). For a first-order ODE, the CE becomes:
y(t) = g(t) + η p(t) (3.3)
where p(t) = 1 (see ref. [49,50] for the details of this decision). Thus, the CE becomes:
y(t) = g(t) + η (3.4)
By imposing the constraint, we get:
y(t0 ) = y0 = g0 + η =⇒ η = y 0 − g0 (3.5)
which is plugged into Eq. (3.4) to obtain the CE in its final form:
y(t) = g(t) + (y0 − g0 ) (3.6)
which represents all possible functions satisfying the boundary value constraint. The derivative then
follows:
ẏ(t) = ġ(t) (3.7)
The CE defined by Eq. (3.6) and its derivative are plugged into our ODE which then becomes:
F (t, y, ẏ) = 0 (3.8)
that is now a boundary condition free. By substituting Eq. (3.6) into Eq. (3.8), the differential equation
F is transformed into a new differential equation F̃ , which is only a function of the independent variable
t and the free function g(t), that is:
F̃ (t, g, ġ) = 0 (3.9)
This differential equation is now unconstrained and will always satisfy the boundary-value due to the way
Eq. (3.4) is built. In this work, we choose the free function g(t) to be a shallow NN, such that:
⎡ ⎤T
L σ(w1 t + b1 )
 ⎢ .. ⎥ T
g(t) = βj σ (wj t + bj ) = ⎣ . ⎦ β=σ β (3.10)
j=1 σ(wL t + bL )
where L is the number of hidden neurons, wj ∈ R is the input weights vector connecting the jth hidden
neuron and the input nodes, βj ∈ R with j = 1, ..., L is the jth output weight connecting the j th
hidden neuron and the output node, and bj is the bias of the jth hidden neuron, and σj (·) are activation

99
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 5 of 18 126

Fig. 1. Standard shallow neural network (left) and shallow Chebyshev neural network (right)

functions. Once the free function is defined, the unknown solution y(t), defined by Eq. (3.6), is computed
via the ELM algorithm. According to the ELM algorithm Ref. [62], input weights and biases are randomly
selected and not tuned during the training, thus they are known parameters. The activation functions,
σj (·), are chosen by the user, so they are also known. Therefore, the only unknowns to compute are
T
the output weights β = [β1 , ..., βL ] . The interested reader can find more details about the ELM and
its convergence proofs in [62]. In this work, we use ChNN as, based on the results presented in [53] is
the best choice to achieve the highest accuracy for these kinds of problems. When using ChNN, the free
function becomes,
⎡ ⎤T
L P1 (w1 t + b1 )
 ⎢ .. ⎥ T
g(t) = βj Pj (wj t + bj ) = ⎣ . ⎦ β=σ β (3.11)
j=1 PL (wL t + bL )
where Pj for j + 1, ..., L is the Chebyshev polynomial of order j. We noticed that for ChNN the choice
of input weights and biases that guarantees the best accuracy, is to set wj = 1 and bj = 0 , ∀j = 1, ..., L.
Thus, using ChNNs with this choice on input weights and bias, Eq. (3.10) modifies as follows:
⎡ ⎤T
L P1 (t)
 ⎢ ⎥
g(t) = βj Pj (t) = ⎣ ... ⎦ β = P T β (3.12)
j=1 PL (t)
With this choice of input weights and bias, ChNNs become equivalent to a linear combination of Cheby-
shev polynomials, where the unknowns are the coefficients, similarly to a collocation method. For the
convenience of the reader, in Fig. (1) we show a representation of a general shallow NN and the ChNN
that we use to solve the problem presented in this manuscript.
It may happen that the activation functions are defined on an inconsistent domain z ∈ [z0 , zf ] (for
instance, Chebyshev and Legendre polynomials are defined on z ∈ [−1, +1]). Thus, our independent
variable t ∈ [t0 , tf ] needs to be mapped in the z domain. In general to have better training performance,
regardless to the activation function used, it is convenient to map the independent variable t ∈ [t0 , tf ]

100
 126 Page 6 of 18 M. Florio et al. ZAMP

Fig. 2. Schematic of the X-TFC framework to solve linear ODEs with one constraint in one point

in the z domain (usually z ∈ [0, +1] or z ∈ [−1, +1]). This can be done using the following linear
transformation:
1
z = z0 + c(t − t0 ) ←→ t = t0 + (z − z0 )
c
where c is the mapping coefficient:
zf − z0
c=
tf − t0
By the derivative chain rule, the nth derivative of Eq. (3.10) is,
dn g(t) n
n T d σ(z)
= c β , (3.13)
dtn dz n
which defines all mappings of the free function. Finally, the domain t must be discretized by M points,
and then, the DE reduces to the following:
F̄ (β) = 0 (3.14)
Equation (3.14) is an unconstrained optimization problem that can be solved via different optimization
schemes. As the problem is linear, rearranging the term in Eq. (3.14), it reduces to a linear system of
algebraic equation: Aβ = b. Therefore,
−1
β = AT A AT b (3.15)
For the convenience of the reader, a schematic that summarizes how the PINN TFC-based method works
for first-order linear ODEs is reported in Fig. 2

4. Poiseuille flow in a plane channel: problem formulation

In the underlying physical problem, a rarefied gas flows between two parallel plates, separated by a
distance d, in the z direction, as shown in Fig. 3.
A pressure gradient causes a density gradient kρ0 (where ρ0 is the density at the boundary z = 0, and
k is proportional to the pressure gradient), which in turn causes the flowing of the gas. The temperature

101
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 7 of 18 126

Fig. 3. Poiseuille microchannel flow

T is assumed to be constant on the walls, and therefore also in the gas because of the linearization. The
problem to be solved follows the formulation presented by Siewert et al. [68]
∞
1 ∂ 2
kθ + θcx Z(x, cx ) + Z(x, cx ) = π −1/2 e−u Z(x, u)du (4.1)
2 ∂x
−∞

for x ∈ (−d/2, d/2) and cx ∈ (−∞, ∞), and subject to:

Z(−d/2, cx ) = (1 − α) Z(−d/2, −cx )Z(d/2, −cx ) = (1 − α) Z(d/2, cx ) (4.2)

for cx ∈ (0, ∞), where α ∈ (0, 1] is the accommodation coefficient, θ is the mean-free time. Following Ref.
[36], we introduce new variables: τ = x/θ , δ = d/θ (Knudsen number) and u = cx , obtaining:
∞
1 ∂ 2
kθ + u Z(τ, u) + Z(τ, u) = π −1/2 e−u Z(τ, u)du (4.3)
2 ∂τ
−∞

for τ ∈ (δ/2, δ/2) and u ∈ (0, ∞), and the boundary conditions

Z(−δ/2, u) = (1 − α)Z(−δ/2, −u)

(4.4)
Z(δ/2, −u) = (1 − α)Z(δ/2, u)

for u ∈ (0, ∞). Now, by using a particular solution that accounts for the inhomogeneous term in Eq. (4.3),
we can obtain the equation in a homogeneous form. Using
1  
Z (τ, u) = kθ τ 2 − 2τ u + 2u2 − a2 − 2Y (τ, u) (4.5)
2

102
126 Page 8 of 18 M. Florio et al. ZAMP

where 2a = δ, and plugging it into (4.1) and (4.2), we find that Y (τ, u) satisfies the following linear
integrodifferential equation:
∞
∂
u Y (τ, u) + Y (τ, u) = Ψ(u)Y (τ, u)du (4.6)
∂τ
−∞

for τ ∈ (−a, a) and u ∈ (−∞, ∞), subject to the following boundary conditions:
Y (−a, u) = (1 − α)Y (−a, −u) + αu2 + au(2 − α)
(4.7)
Y (a, −u) = (1 − α)Y (a, u) + αu2 + au(2 − α)
for u ∈ (0, ∞). Here, Ψ(u) is a weight function defined as follows:

2
Ψ(u) = π −1/2 e−u (4.8)
We can take advantage of spatial symmetry to reduce the computational cost. Therefore, the boundary
conditions become
Y (0, u) = Y (0, −u)
(4.9)
Y (a, −u) = (1 − α)Y (a, u) + αu2 + au(2 − α)
For simplicity, we let K(u) = αu2 + au(2 − α), then:
Y (0, u) = Y (0, −u)
(4.10)
Y (a, −u) = (1 − α)Y (a, u) + K(u)
The problem (4.6)–(4.10) is the one to be solved via the TFC.

5. X-TFC formulation of the problem

Here, we are dealing a boundary value problem (BVP). Thus, according to (3.2), y(τ ) can be expressed
as following:
y(τ ) = g(τ ) + ηp(τ ). (5.1)
We choose p(τ ) = 1, as suggested by [49], and the free function g(τ ) as a Chebyshev NN:
g(τ ) = σ T (τ )β (5.2)
We need to map the independent variable τ to a new independent variable x that ranges in [−1, 1]. The
new independent variable χ is defined as χ = c(τ + a) − 1, where c is the following mapping constant:
χf − χ0 1
c= =
τf − τ0 a
Considering the new independent variable, we have:
⎧
⎨Y (τ, u) = Y (χ, u)
∂ ∂ (5.3)
⎩ Y (τ, u) = c Y (χ, u)
∂τ ∂χ
Then, the problem becomes:
∞
∂
cu Y (χ, u) + Y (χ, u) = Ψ(u )Y (χ, u )du (5.4)
∂χ
−∞

103
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 9 of 18 126

In order to use a Gauss–Legendre quadrature in the range [−1, 1], we can use a further change of variable:
1 2
u = − log(|μ|); du = − dμ; Ψ(μ) = π −1/2 e−(− log(|μ|)) (5.5)
μ
where μ ∈ [−1; 1]. We can then rewrite the problem in the following form:
1
∂ 1
− c log(μ) Y (χ, μ) + Y (χ, μ) = − Ψ(μ )Y (χ, μ )dμ (5.6)
∂χ μ
−1

Via discretizing the velocity μ into N points in the interval (0, 1] according to the Gauss–Legendre
quadrature scheme, we have:
μ −→ μ = {μi }N
i=1 ; μ ∈ (N × 1)
it follows that:
Y (χ, μ) −→ Y(χ) = {Y (χ, μi )}N
i=1 ; Y(χ) ∈ (N × 1) (5.7)
The problem can be split for both positive and negative molecular velocity following Ref. [36], where the
Gauss–Legendre quadrature rule is used to evaluate the integral in the range [0, 1]:
N
∂ 1
− c log(μi ) Y (χ, μi ) + Y (χ, μi ) = wk Ψ(μk ) [Y (χ, μk ) + Y (χ, −μk )] (5.8)
∂χ μk
k=1
 N
∂ 1
c log(μi ) Y (χ, −μi ) + Y (χ, −μi ) = wk Ψ(μk ) [Y (χ, μk ) + Y (χ, −μk )] (5.9)
∂χ μk
k=1

subject to:
Y (0, μi ) = Y (0, −μi )
(5.10)
Y (a, −μi ) = (1 − α)Y (a, μi ) + Ki

where Ki = α log2 (μi ) − a log(μi )(2 − α). To simplify the notation, we suppress the χ and μ dependency
and write the terms for the positive and negative molecular velocity as Y + and Y − , respectively. Thus,
our problem becomes:
 N
∂ 1
− c log(μi ) Yi+ + Yi+ = wk Ψ(μk ) Yk+ + Yk− (5.11)
∂χ μk
k=1
 N
∂ − 1
c log(μi ) Yi + Yi− = wk Ψ(μk ) Yk+ + Yk− (5.12)
∂χ μk
k=1

subject to:
Y0+ = Y0−
(5.13)
Yf− = (1 − α)Yf+ + Ki
Our constrained expressions, for positive and negative terms, are:
Y ± (χ) = g ± (χ) + η ± (5.14)
and according to the boundary conditions, we have:
Y0+ = σ0 T β + + η + Yf− = σf T β − + η −
Yf+ = σf T β + + η + Y0− = σ0 T β − + η −

104
126 Page 10 of 18 M. Florio et al. ZAMP

Replacing them in the previous system of equations, we obtain:

σ0 T βi + + ηi+ = σ0 T βi − + ηi−
(5.15)
σf T βi − + ηi− = (1 − α) σf T βi + + (1 − α) ηi+ + Ki
and then
ηi+ − ηi− = σ0 T βi − − σ0 T βi +
(1 − α) ηi+ − ηi− = σf T βi − − (1 − α)σf T βi + − Ki
that in matrix form becomes:
    
1 −1 ηi+ σ0 T βi − − σ0 T βi +
=
1 − α −1 ηi− σf T βi − − (1 − α)σf T βi + − Ki
And therefore
    
ηi+ 1 −1 1 σ0 T βi − − σ0 T βi +
= −
ηi− − +
α −(1 − α) 1 σf T βi − (1 − α)σf T βi − Ki
that is
   1  
ηi+ α − α1 σ0 T βi − − σ0 T βi +
= 1−α 1
ηi− α −α σf T βi − − (1 − α)σf T βi + − Ki
1−α
By calling ϕ = α and γ = α1 , we can rewrite:
 +   
ηi γ −γ σ0 T βi − − σ0 T βi +
=
ηi− ϕ −γ σf T βi − − (1 − α)σf T βi + − Ki
and thus
T T
ηi+ = (ϕσf − γσ0 ) βi + + γ (σ0 − σf ) βi − + γKi
T T (5.16)
ηi− = ϕ (σf − σ0 ) βi + + (ϕσ0 − γσf ) βi − + γKi
Replacing them in the constrained expressions, we have:
T T
Yi+ = (σ + ϕσf − γσ0 ) βi + + γ (σ0 − σf ) βi − + γKi
T T
(5.17)
Yi− = ϕ (σf − σ0 ) βi + + (σ + ϕσ0 − γσf ) βi − + γKi
and replacing the constrained expressions in the equations of our problem, we obtain:
T T
(−c log(μi ) σ  + σ + ϕσf − γσ0 ) βi + + γ (σ0 − σf ) βi −
N
1  
T T
− wk Ψ(μk ) (σ + 2ϕσf − (γ + ϕ)σ0 ) βk + + (σ + (γ + ϕ)σ0 − 2γσf ) βk −
μk
k=1
N

= −γ Ki + 2wk Ψk γKk (5.18)
k=1
T
ϕ (σf − + (c log(μi ) σ  + σ + ϕσ0 − γσf ) βi −
σ0 ) βi +
N
 1  
T T
− wk Ψ(μk ) (σ + 2ϕσf − (γ + ϕ)σ0 ) βk + + (σ + (γ + ϕ)σ0 − 2γσf ) βk −
μk
k=1
N

= −γ Ki + 2wk Ψk γKk (5.19)
k=1

105
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 11 of 18 126

For the sake of simplicity, we write the inhomogeneous terms as:

N

−
b+
i = bi = −γKi + 2wk Ψk γKk
k=1

Expanding the summations, we get the following matrix form:

     +
k + i k + ♣ βi+ bi
− = (5.20)
k + ♠ k + i βi b−
i

where:
T T
i = −c log (μi ) σ  + σ T + ϕσf T − γσ0 T i = c log (μi ) σ  + σ T + ϕσ0 T − γσf T
T T
♠ = ϕ (σf − σ0 ) ♣ = γ (σ0 − σf )
1 T 1 T
k = −wk Ψ (μk ) (σ + 2ϕσf − (γ + ϕ)σ0 ) k = −wk Ψ (μk ) (σ + (γ + ϕ)σ0 − 2γσf )
μk μk
Thus, the problem reduces to the following linear system:

which is a problem of the type:

Aβ = B (5.21)
whose the dimensions are:
β ± i L×1
β (2LN )×1
b± i M ×1
B (2M N )×1
i (M ×L) (same dimension for the other terms of the matrix A)
A(2M N )×(2LN )
where M is the discretization order for τ , and L is the number of neurons used. That is, the original
BVP has been reformulated as an unconstrained optimization problem. In this case, β is calculated via
LS, and then the final solutions are built according to (5.17).

106
126 Page 12 of 18 M. Florio et al. ZAMP

6. Results and discussions

To demonstrate the precision of the X-TFC in solving the problem, we report the macroscopic velocity
profile, that according to Loyalka et al. [6] is given by:
∞
1
q(τ ) = Ψ(u)Y (τ, u)du (6.1)
kθ
−∞

By plugging Eq. (4.5) into Eq. (6.1), we obtain:

1
q(τ ) = 1 − a2 + τ 2 − Y0 (τ ) (6.2)
2
where Y0 (τ ) is given by [36]:
∞

Y0 (τ ) = Ψ(u)Y (τ, u)du (6.3)

−∞

that is
1
1
Y0 (τ ) = Ψ(μ)Y (τ, μ)dμ (6.4)
μ
−1

which it can be computed as:

N
 1
Y0 = wi Ψ(μi ) Yi + + Yi − (6.5)
i=1
μi
Figure 4 shows the qualitative behavior of the macroscopic velocity for different values of Knudsen
number δ, computed with different types of NNs. It can be seen how the macroscopic velocity increases
for increasing the thickness of the channel, and then of Kn. Hence, Fig. 4 proves that all the NNs tested
are able to capture the qualitative behavior of the macroscopic velocity.
We coded the problem in the MATLAB R2019a platform and ran all our simulations with an Intel
Core i7-9700 CPU PC with 64 GB of RAM. Our numerical results are presented below. In Table 1, the
results for a channel of half width a = 1, for different values of the accommodation number, are reported.
All our digits match with the benchmark reported in Ref. [36]. These results are obtained by setting
N = 22, M = 200, and L = 50 ± 2. The computational time was 0.24 seconds.
Furthermore, we computed the flow rate:
a
1
Q=− 2 q(τ )dτ (6.6)
2a
−a

and, again, by making use of Eq. (4.5), we find:

a  
1 1 2
Q= 2 Y0 (τ )dτ − 1 − a2 (6.7)
2a 2a 3
−a

The numerical results for several values of channel width and accommodation number are listed
in Table 2. All our digits match with the benchmark reported in Ref. [36]. We used different spatial
discretizations based on the Knudsen number, from M = 40 and L = 24 for δ = 2a = 0.05, to M = 300
and L = 90 for δ = 2a = 9. All the results are obtained with a velocity discretization of N = 30. The
computational time ranges from 0.04 to 1.6 seconds.

107
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 13 of 18 126

-0.5

-1

-1.5

q -2

-2.5
2a = 1.0
2a = 1.5
-3 2a = 2.0
2a = 2.5
2a = 3.0
-3.5
-1 -0.5 0 0.5 1
/a

Fig. 4. Macroscopic velocity profiles for different values of a

Table 1. The macroscopic velocity profile q(τ ) for a plan channel of half width a = 1 , with L = 50 ± 2, M = 200, and
N = 22. All the digits match the benchmark published in Refs. [36, 37]

τ α = 0.50 α = 0.80 α = 0.88 α = 0.96 α = 1.00

0.0 − 3.652222 − 2.319616 − 2.117410 − 1.948801 − 1.874577

0.1 − 3.644836 − 2.312148 − 2.109921 − 1.941293 − 1.867059
0.2 − 3.622577 − 2.289638 − 2.087351 − 1.918663 − 1.844401
0.3 − 3.585117 − 2.251759 − 2.049368 − 1.880582 − 1.806271
0.4 − 3.531852 − 2.197901 − 1.995366 − 1.826440 − 1.752062
0.5 − 3.461789 − 2.127072 − 1.924350 − 1.755244 − 1.680778
0.6 − 3.373321 − 2.037666 − 1.834718 − 1.665394 − 1.590822
0.7 − 3.263728 − 1.926991 − 1.723784 − 1.554211 − 1.479519
0.8 − 3.127917 − 1.790039 − 1.586566 − 1.416741 − 1.341927
0.9 − 2.954020 − 1.615281 − 1.411628 − 1.241643 − 1.166756
1.0 − 2.676407 − 1.340372 − 1.137528 − 9.683813e-1 − 8.939247e-1

Finally, we pushed the code increasing the discretization of space, velocity, and number of neurons.
We were able to match all the seven digits reported in the most recent benchmark for the flow rate for
the Poiseuille flow problem between two parallel plates [40], achieved with adding and doubling method.
The results are reported in Tables 3. The parameters chosen and the computational time for each channel
width are reported in Table 4. The parameters reported in Table 4 are also plotted in Fig. 5 to better
appreciate their variations with respect to the channel thickness 2a.

108
126 Page 14 of 18 M. Florio et al. ZAMP

Table 2. The flow rate Q(a) for [36] digits

2a α = 0.50 α = 0.80 α = 0.88 α = 0.96 α = 1.00

0.05 5.22330 3.08971 2.73834 2.43735 2.30226

0.10 4.55641 2.70774 2.40605 2.14824 2.03271
0.30 3.77847 2.24477 2.00107 1.79451 1.70247
0.50 3.54437 2.10227 1.87662 1.68634 1.60187
0.70 3.43767 2.03877 1.82201 1.63985 1.55919
0.90 3.38389 2.00924 1.79764 1.62022 1.54180
1.00 3.36822 2.00187 1.79206 1.61631 1.53868
2.00 3.37657 2.04139 1.83856 1.66937 1.59486
5.00 3.77440 2.43823 2.23506 2.06548 1.99077
7.00 4.08811 2.74611 2.54144 2.37038 2.29493
9.00 4.41019 3.06346 2.85756 2.68530 2.60925

Table 3. The flow rate Q(a) for Ganapol digits [40]

2a α = 0.50 α = 0.80 α = 0.88 α = 0.96 α = 1.00

0.05 5.2232964 3.0897113 2.7383403 2.4373544 2.3022564

0.10 4.5564062 2.7077408 2.4060457 2.1482414 2.0327143
0.30 3.7784723 2.2447708 2.0010675 1.7945088 1.7024740
0.50 3.5443709 2.1022657 1.8766202 1.6863424 1.6018742
0.70 3.4376693 2.0387670 1.8220109 1.6398495 1.5591860
0.90 3.3838869 2.0092408 1.7976360 1.6202230 1.5417996
1.00 3.3682182 2.0018669 1.7920590 1.6163124 1.5386785
2.00 3.3765738 2.0413852 1.8385632 1.6693655 1.5948569
5.00 3.7744018 2.4382339 2.2350591 2.0654781 1.9907674
7.00 4.0881078 2.7461124 2.5414362 2.3703751 2.2949322
9.00 4.4101902 3.0634644 2.8575645 2.6852950 2.6092536

Table 4. Parameters used to compute the flow rate Q(a) for Table 3 for each channel width

2a N M L Comp. time (s)

0.05 69 100 34 1.35

0.10 59 120 54 2.39
0.30 35 140 60 0.78
0.50 30 140 60 0.51
0.70 30 140 60 0.52
0.90 30 200 67 0.81
1.00 30 200 74 0.99
2.00 24 400 90 1.30
5.00 24 700 130 4.31
7.00 24 900 150 6.92
9.00 24 1400 150 10.12

7. Conclusions

We have developed and successfully applied the PINN-TFC method, Extreme Theory of Functional
Connections, to solving the Poiseuille flow in a plane channel problem. The proposed method is proved
to be accurate and efficient. To check the accuracy of our solutions, we have compared them with the
benchmarks published in Refs. [6,37,40], matching their results up to the fifth and seventh digits. The

109
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 15 of 18 126

80

60

N
40

20
0 1 2 3 4 5 6 7 8 9
2a
1500

1000
M

500

0
0 1 2 3 4 5 6 7 8 9
2a
150

100
L

50

0 1 2 3 4 5 6 7 8 9
2a
computational time [s]

10

0
0 1 2 3 4 5 6 7 8 9
2a

Fig. 5. Variations of the chosen parameters and computational times in function of the channel width reported in Table 4

results of this paper show the feasibility of the proposed methodology and demonstrate the accuracy of
PINNs for a large variety of problems arising from the Boltzmann linear integrodifferential equation. We
are currently working on applying the proposed X-TFC method to solving more RGD problems such
as Couette flow in a plane channel [69] as formulated by Williams [7], and the thermal-jump problem.
Moreover, we are currently working on solving high-dimensional problems with the X-TFC algorithm,
which has been proved to be a significant improvement for the PINN frameworks for estimating solutions
of differential equations [42], in particular for PDEs. Future works will then focus on applying this
new method to solve 3D time-dependent problems arising from the Boltzmann linear integrodifferential
equation in rarefied gas dynamics, radiative transfer, and neutron transport problems.

Declarations

Conflict of interest The authors declare that they have no known competing financial interests or personal
relationships that could have appeared to influence the work reported in this paper.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps
and institutional affiliations.

110
 126 Page 16 of 18 M. Florio et al. ZAMP

References

[1] Reynolds, O.: Xviii. on certain dimensional properties of matter in the gaseous state.-Part I. Experimental researches
on thermal transpiration of gases through porous plates and on the laws of transpiration and impulsion, including an
experimental proof that gas is not a continuous plenum.-part ii. on an extension of the dynamical theory of gas, which
includes the stresses, tangential and normal, caused by a varying condition of gas, and affords an explanation of the
phenomena of transpiration and impulsion. Philos. Trans. R. Soc. Lond. 170, 727–845 (1879)
[2] Maxwell, J.C.: Iii. on stresses in rarefied gases arising from inequalities of temperature. Proc. R. Soc. Lond. 27(185–189),
304–308 (1878)
[3] Knudsen, M.: Die gesetze der molekularströmung und der inneren reibungsströmung der gase durch röhren. Ann. Phys.
333(1), 75–130 (1909)
[4] Knudsen, M.: Thermischer molekulardruck der gase in röhren. Ann. Phys. 338(16), 1435–1448 (1910)
[5] Loyalka, S.: Comments on “poiseuille flow and thermal creep of a rarefied gas between parallel plates”.” Phys. Fluids
17(5), 1053–1055 (1974)
[6] Loyalka, S., Petrellis, N., Storvick, T.: Some exact numerical results for the BGK model: Couette, Poiseuille and thermal
creep flow between parallel plates. Z. Angew. Math. Phys. ZAMP 30(3), 514–521 (1979)
[7] Williams, M.: A review of the rarefied gas dynamics theory associated with some classical problems in flow and heat
transfer. Z. Angew. Math. Phys. ZAMP 52(3), 500–516 (2001)
[8] Boffi, V., De Socio, L., Gaffuri, G., Pescatore, C.: Rigorous constructive solution to monodimensional poiseuille and
thermal creep flows. Meccanica 11(4), 183–190 (1976)
[9] Cercignani, C.: The boltzmann equation. In: The Boltzmann Equation and its Applications, pp. 40–103. Springer, New
york (1988)
[10] Sharipov, F.: Rarefied Gas Dynamics: Fundamentals for Research and Practice. Wiley, New Jersey (2015)
[11] Ferziger, J., Kaper, H.: Mathematical theory of transport processes in gases. Am. J. Phys. 41(4), 601–603 (1973)
[12] Cercignani, C.: Theory and Application of the Boltzmann Equation. Scottish Academic Press, UK (1975)
[13] Bhatnagar, P.L., Gross, E.P., Krook, M.: A model for collision processes in gases. I. Small amplitude processes in
charged and neutral one-component systems. Phys. Rev. 94(3), 511 (1954)
[14] Shakhov, E.: Generalization of the krook kinetic relaxation equation. Fluid Dyn. 3(5), 95–96 (1968)
[15] Loyalka, S.: Thermal transpiration in a cylindrical tube. Phys. Fluids 12(11), 2301–2305 (1969)
[16] Loyalka, S., Storvick, T.: Kinetic theory of thermal transpiration and mechanocaloric effect. III. Flow of a polyatomic
gas between parallel plates. J. Chem. Phys. 71(1), 339–350 (1979)
[17] Chernyak, V., Porodnov, B., Suetin, P.: Application of the variational method to the problem of thermomolecular
pressure difference in a cylindrical capillary. Inzh.-Fiz. Zh. 26, 446–450 (1974)
[18] Valougeorgis, D., Thomas, J., Jr.: Exact numerical results for poiseuille and thermal creep flow in a cylindrical tube.
Phys. Fluids 29(2), 423–429 (1986)
[19] Loyalka, S., Hickey, K.: Kinetic theory of thermal transpiration and the mechanocaloric effect: Planar flow of a rigid
sphere gas with arbitrary accommodation at the surface. J. Vac. Sci. Technol. A Vac. Surf. Films 9(1), 158–163 (1991)
[20] Sharipov, F.: Rarefied gas flow through a long tube at any temperature ratio. J. Vac. Sci. Technol. A Vac. Surf. Films
14(4), 2627–2635 (1996)
[21] Ritos, K., Lihnaropoulos, Y., Naris, S., Valougeorgis, D.: Pressure-and temperature-driven flow through triangular and
trapezoidal microchannels. Heat Transf. Eng. 32(13–14), 1101–1107 (2011)
[22] Ohwada, T., Sone, Y., Aoki, K.: Numerical analysis of the shear and thermal creep flows of a rarefied gas over a plane
wall on the basis of the linearized boltzmann equation for hard-sphere molecules. Phys. Fluids A 1(9), 1588–1599 (1989)
[23] Kanki, T., Iuchi, S.: Poiseuille flow and thermal creep of a rarefied gas between parallel plates. Phys. Fluids 16(5),
594–599 (1973)
[24] Chandrasekhar, S.: Radiative Transfer. Courier Corporation, USA (2013)
[25] Barichello, L., Siewert, C.E.: A discrete-ordinates solution for a non-grey model with complete frequency redistribution.
J. Quant. Spectrosc. Radiat. Transf. 62(6), 665–675 (1999)
[26] Vilhena, M., Segatto, C., Barichello, L.: A particular solution for the sn radiative transfer problems. J. Quant. Spectrosc.
Radiat. Transf. 53(4), 467–469 (1995)
[27] Chien, K.-Y.: Application of the s, method to spherically symmetric radiative-transfer problems. AIAA J. 10(1), 55–59
(1972)
[28] Simch, M., Segatto, C., Vilhena, M.: An analytical solution for the sn radiative transfer equation with polarization in
a slab by the ltsn method. J. Quant. Spectrosc. Radiat. Transf. 97(3), 424–435 (2006)
[29] Benoist, P., Kavenoky, A.: A new method of approximation of the boltzmann equation. Nucl. Sci. Eng. 32(2), 225–232
(1968)
[30] Siewert, C., Benoist, P.: The fn method in neutron-transport theory. Part I: Theory and applications. Nucl. Sci. Eng.
69(2), 156–160 (1979)

111
ZAMP Physics-Informed Neural Networks for rarefied-gas dynamics Page 17 of 18 126

[31] Devaux, C., Siewert, C.: Thef n method for radiative transfer problems without azimuthal symmetry. Z. Angew. Math.
Phys. ZAMP 31(5), 592–604 (1980)
[32] Garcia, R., Siewert, C.: The fn method for radiative transfer models that include polarization effects. J. Quant. Spectrosc.
Radiat. Transf. 41(2), 117–145 (1989)
[33] Ganapol, B.D., Myneni, R.: The fn method for the one-angle radiative transfer equation applied to plant canopies.
Remote Sens. Environ. 39(3), 213–231 (1992)
[34] Benassi, M., Garcia, R., Karp, A., Siewert, C.: A high-order spherical harmonics solution to the standard problem in
radiative transfer. Astrophys. J. 280, 853–864 (1984)
[35] Siewert, C., Thomas, J., Jr.: A particular solution for the pn method in radiative transfer. J. Quant. Spectrosc. Radiat.
Transf. 43(6), 433–436 (1990)
[36] Barichello, L., Siewert, C.: A discrete-ordinates solution for poiseuille flow in a plane channel. Z. Angew. Math. Phys.
ZAMP 50(6), 972–981 (1999)
[37] Barichello, L., Camargo, M., Rodrigues, P., Siewert, C.: Unified solutions to classical flow problems based on the BGK
model. Z. Angew. Math. Phys. ZAMP 52(3), 517–534 (2001)
[38] Siewert, C.: A concise and accurate solution to chandrasekhar’s basic problem in radiative transfer. J. Quant. Spectrosc.
Radiat. Transf. 64(2), 109–130 (2000)
[39] Ganapol, B.D.: The response matrix discrete ordinates solution to the 1d radiative transfer equation. J. Quant. Spec-
trosc. Radiat. Transf. 154, 72–90 (2015)
[40] Ganapol, B.D.: Poiseuille channel flow by adding and doubling. In: AIP Conference Proceedings, vol. 1786, p. 070009.
AIP Publishing LLC (2016)
[41] Ganapol, B.D.: 1d thermal creep channel flow in the bgk approximation by adding and doubling. Ann. Nucl. Energy
134, 441–451 (2019)
[42] Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving
forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
[43] Schiassi, E., De Florio, M., D’ambrosio, A., Mortari, D., Furfaro, R.: Physics-informed neural networks and functional
interpolation for data-driven parameters discovery of epidemiological compartmental models. Mathematics 9(17), 2069
(2021)
[44] Cercignani, C., Daneri, A.: Flow of a rarefied gas between two parallel plates. J. Appl. Phys. 34(12), 3509–3513 (1963)
[45] Cercignani, C.: Plane Poiseuille flow according to the method of elementary solutions. J. Math. Anal. Appl. 12(2),
254–262 (1965)
[46] Schiassi, E., Furfaro, R., Leake, C., De Florio, M., Johnston, H., Mortari, D.: Extreme theory of functional connections:
a fast physics-informed neural network method for solving ordinary and partial differential equations. Neurocomputing
457, 334–356 (2021)
[47] Mortari, D.: The theory of connections: connecting points. Mathematics 5(4), 57 (2017)
[48] Leake, C., Mortari, D.: Deep theory of functional connections: a new method for estimating the solutions of partial
differential equations. Mach. Learn. Knowl. Extr. 2(1), 37–55 (2020)
[49] Mortari, D.: Least-squares solution of linear differential equations. Mathematics 5(4), 48 (2017)
[50] Mortari, D., Johnston, H., Smith, L.: High accuracy least-squares solutions of nonlinear differential equations. J. Com-
put. Appl. Math. 352, 293–307 (2019)
[51] De Florio, M., Schiassi, E., D’Ambrosio, A., Mortari, D., Furfaro, R.: Theory of functional connections applied to linear
odes subject to integral constraints and linear ordinary integro-differential equations. Math. Comput. Appl. 26(3), 65
(2021)
[52] De Florio, M., Schiassi, E., Furfaro, R., Ganapol, B.D., Mostacci, D.: Solutions of chandrasekhar’ s basic problem in
radiative transfer via theory of functional connections. J. Quant. Spectrosc. Radiat. Transf. 259, 107384 (2020)
[53] De Florio, M., Schiassi, E., Ganapol, B.D., Furfaro, R.: Physics-informed neural networks for rarefied-gas dynamics:
thermal creep flow in the bhatnagar-gross-krook approximation. Phys. Fluids 33(4), 047110 (2021)
[54] Furfaro, R., Mortari, D.: Least-squares solution of a class of optimal space guidance problems via theory of connections.
Acta Astronaut. 168, 92–103 (2019)
[55] Johnston, H., Schiassi, E., Furfaro, R., Mortari, D.: Fuel-Efficient Powered Descent Guidance on Large Planetary Bodies
Via Theory of Functional Connections. arXiv preprint arXiv:2001.03572. (2020)
[56] Schiassi, E., D’Ambrosio, A., Johnston, H., Furfaro, R., Curti, F., Mortari, D.: Complete energy optimal landing on
planetary bodies via theory of functional connections. Acta Astronaut. Prep. (2020)
[57] Drozd, K., Furfaro, R., Schiassi, E., Johnston, H., Mortari, D.: Energy-optimal trajectory problems in relative motion
solved via theory of functional connections. Acta Astronaut. 182, 361–382 (2021)
[58] D’Ambrosio, A., Schiassi, E., Curti, F., Furfaro, R.: Pontryagin neural networks with functional interpolation for optimal
intercept problems. Mathematics 9(9), 996 (2021)
[59] Namatame, A.: Connectionist learning with chebychev networks and analyses of its internal representation. In: Appli-
cations of Learning and Planning Methods, pp. 35–48. World Scientific, Singapore (1991)

112
 126 Page 18 of 18 M. Florio et al. ZAMP

[60] Mall, S., Chakraverty, S.: Single layer chebyshev neural network model for solving elliptic partial differential equations.
Neural Process. Lett. 45(3), 825–840 (2017)
[61] Russell, R., Shampine, L.F.: A collocation method for boundary value problems. Numer. Math. 19(1), 1–28 (1972)
[62] Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70, 489–501
(2006)
[63] Liu, M., Hou, M., Wang, J., Cheng, Y.: Solving two-dimensional linear partial differential equations based on chebyshev
neural network with extreme learning machine algorithm. Eng. Comput. (2020)
[64] Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations.
IEEE Trans. Neural Netw. 9, 987–1000 (1998)
[65] Wang, S., Teng, Y., Perdikaris, P.: Understanding and mitigating gradient flow pathologies in physics-informed neural
networks. SIAM J. Sci. Comput. 43(5), A3055–A3081 (2021)
[66] Mertikopoulos, P., Papadimitriou, C., Piliouras, G.: Cycles in adversarial regularized learning. In: Proceedings of the
Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 2703–2717. SIAM (2018)
[67] Balduzzi, D., Racaniere, S., Martens, J., Foerster, J., Tuyls, K., Graepel, T.: The mechanics of n-player differentiable
games. In: International Conference on Machine Learning, pp. 354–363. PMLR (2018)
[68] Siewert, C., Garcia, R., Grandjean, P.: A concise and accurate solution for Poiseuille flow in a plane channel. J. Math.
Phys. 21(12), 2760–2763 (1980)
[69] Sharipov, F.: Application of the cercignani-lampis scattering kernel to calculations of rarefied gas flows. I. Plane flow
between two parallel plates. Eur. J. Mech.-B/Fluids 21(1), 113–123 (2002)

Mario De Florio, Enrico Schiassi and Roberto Furfaro

Department of Systems and Industrial Engineering
The University of Arizona
Tucson AZ85721-0020
USA
e-mail: mariodf@email.arizona.edu

Enrico Schiassi
e-mail: eschiassi@email.arizona.edu

Barry D. Ganapol and Roberto Furfaro

Department of Aerospace and Mechanical Engineering
The University of Arizona
Tucson AZ85721-0020
USA
e-mail: robertof@email.arizona.edu

Barry D. Ganapol
e-mail: ganapol@cowboy.ame.arizona.edu

(Received: November 18, 2021; revised: April 20, 2022; accepted: April 21, 2022)

113
Appendix E

De Florio et al. (under review) Couette

Flow

Reproduced from De Florio, M., Schiassi, E., Barichello, L. B., & Furfaro, R. (under-review).
Numerical analysis of the plane Couette flow of a rarefied gas with Physics-Informed Neural
Networks and Functional Interpolation. Physics of Fluids [11].

114
Couette flow of a rarefied gas with Physics-Informed Neural Networks

Numerical analysis of the plane Couette flow of a rarefied gas with

Physics-Informed Neural Networks and Functional Interpolation
Mario De Florio,1 Enrico Schiassi,1 Liliane B. Barichello,2 and Roberto Furfaro1, 3
1) Department of Systems & Industrial Engineering, The University of Arizona, 1127 James E. Rogers Way, Tucson, AZ, 85719,
USA
2) Instituto de Matemática e Estatística, Universidade Federal do Rio Grande do Sul, Porto Alegre, RS 91509,
Brazil
3) Department of Aerospace & Mechanical Engineering, The University of Arizona, 1130 N Mountain Ave, Tucson, AZ 85721,
USA
(*Electronic mail: robertof@arizona.edu)
(Dated: 8 November 2022)
The Boltzmann equation with the Bhatnagar-Gross-Krook collision model has been widely employed to describe the
evolution of a gas through a kinetic theory. We propose a new Machine Learning approach to accurately and efficiently
learn the solution of a class of problems in the transport theory of rarefied–gas dynamics, employing a particular
Physics-Informed Neural Networks method called Extreme Theory of Functional Connections (X-TFC). According to
X-TFC, the unknown function is approximated by the Constrained Expressions defined int the Theory of Functional
Connections. Constrained Expressions are functionals (e.g., function of functions), which are the sum of a free function
and a functional in which the constraints are analytically embedded. The constraints will therefore always be satisfied
analytically, no matter what the free-chosen function is. In this work, a single-layer Neural Network trained with
Extreme Learning Machine algorithm is employed. The proposed Machine Learning approach learns the solution of the
Linear Boundary Value Problem arising from the Couette Flow problem in the Bhatnagar-Gross-Krook approximation
between two parallel plates in relative motion, for a wide range of Knudsen numbers. The accuracy of X-TFC method
is tested and compared with published benchmarks in literature.

I. INTRODUCTION the evolution of a physical quantity, by minimizing the mean

squared error (MSE) between the data and the NN model, as
The Bhatnagar–Gross–Krook (BGK) model1,2 has been a loss function. The novelty of PINN lies in including the
widely studied in the last decades as a simplified model knowledge of physics (represented by the residuals of the
of the Boltzmann equation3,4 . In particular, in two basic DE and its constraints) as a penalizer in the loss function.
papers in the rarefied-gas dynamics (RGD), Cercignani and From the computer/data science perspective, PINNs define a
Daneri5,6 explore the behavior of a gas in a rarefied-state data-physics-driven approach to performing data regression
flowing between two parallel walls from both a theoretical using NNs. From the physics perspective, PINNs define a
and numerical point of view. From these works, many others data-physics-driven approach to solving DEs. Solving DEs
were born by different authors, where numerical methods with this data-physics-driven approach is a unique feature of
have been applied to solve several classic RGD problems7–16 , PINNs (and in general, Physics-Informed Machine Learning
such as Poiseuille, Thermal creep, and Couette flows, to name PIML) as real data intrinsically hold information about
a few. perturbations and/or unmodeled terms of the real physics that
In this paper, we consider the linearized Couette flow were not accounted for in the modeled one via DE. In the
simplified by the BGK approximation, which makes use of absence of observed or synthetic data – which is the case with
a statistical model for the molecular collisions1 . Around the present work – PINNs are built to learn the solutions of
1960, in many works, the Couette problem was solved for a DEs solely in a physics-driven fashion.
wide range of Knudsen numbers17,18 , creating comparable The main shortcoming of the standard PINN framework, as
benchmarks for future works. Among these, we can mention presented by Raissi et al.25 , is that the DE constraints are not
Refs.14,19–21 using the Analytical Discrete Ordinates (ADO) analytically satisfied, and then they need to be simultaneously
method, introduced by Barichello and Siewert in the field learned with the DE solution inside the domain. This presence
of radiative transfer22 . The ADO method can be seen as a of competing objectives in the loss function can lead to a
modern version of the discrete ordinates method proposed by gradient flow pathologies in PINNs, causing the failure in
Wick23 , and Chandrasekhar24 . ADO is expected to be more learning the DEs constraints. Different approaches have been
efficient than the numerical one since the spatial variable is proposed to try to overcome this limitation. For example, in26
not discretized, and its approach does not require an iterative a learning rate annealing algorithm is employed to adaptively
procedure. assign proper weights to each term in the PINNs loss function
during the network’s training.
A new numerical method called Physics-Informed Neural
Networks (PINNs) is arousing a lot of interest in the machine The authors of this manuscript developed a different PINN
learning community25 . Neural Networks (NNs) are generally framework, which we will use to solve the Couette flow
used to perform a regression on observed data representing problem, to overcome this limitation of the standard PINN

115
Couette flow of a rarefied gas with Physics-Informed Neural Networks 2

methods. This framework brings together NN, specifically II. PHYSICS-INFORMED NEURAL NETWORKS
shallow NNs trained via the Extreme Learning Machine
(ELM) algorithm and the Theory of Functional Connections PINNs have been presented by Raissi et al.25 to solve
(TFC). ELM is a training algorithm for shallow NN proposed physics problems with three main approaches: physics-driven,
by Huang et al.27 , in which the input weights and bias are data-physics-driven, and data-physics-driven PDEs’ parame-
randomly selected before the NN’s training process, thus ters discovery approaches. In a regular PINN framework, a
becoming NN’s hyperparameters. Thus the only trainable deep neural networks (DNNs) is employed to learn the DE
parameters remaining are the output weights, learned via solution in strong form51 . When PINNs are used solely in a
least-squares. TFC28 is a mathematical framework for linear physics-driven fashion, the training set consists of points to
functional interpolation. It derives functionals including a enforce the equation constraints on the boundary and the DE
free function and always satisfies a set of specified linear into the solution domain. The training points can be either
constraints regardless of the free function’s choice. These randomly sampled or sampled following a specific quadrature
functionals, called Constrained Expressions (CEs), reduce the scheme52 . When PINNs are used in a data-physics-driven
solutions search space of constrained problems to solely the fashion, the training set is made for the physics-driven sce-
subspace of functions that analytically satisfy the specified nario and must include the data points from the experimen-
linear constraints. Although TFC was recently developed tal data-set25 . The DE solution, approximated via DNN, is
(2017), it has found many applications, especially for solving plugged in the DE’s residual and initial/boundary conditions’
Differential Equations (DEs), both Ordinary DE (ODE)29–31 residuals. The DNN is next trained via gradient-based meth-
and Partial DEs (PDEs)32,33 . When applied to DEs, the ana- ods (usually Adam optimizer) in learning the network hy-
lytically satisfied constraints are the initial and/or boundary perparameters (e.g., input weights and bias) that minimize
conditions, and the free function is usually an expansion of the physics-based loss function made by the DE and its con-
Chebyshev or Legendre polynomials. Orthogonal polynomi- straints. Hence, the characteristic feature of PINNs is to em-
als are generally a good choice for their approximating and bed, in the loss function, the problem’s knowledge of the
convergence properties. For instance, Chebyshev polynomials physics acquired from the DE and its constraints. The inter-
generate a function that minimizes the maximum error in its ested reader, to better understand how classic PINNs work,
application, and thus, they are well suited for approximating can find the detailed step-by-step procedure to solve a first-
other functions34,35 . TFC applied to DEs with Chebyshev order linear ODE in Ref.48 .
or Legendre polynomials as free function is also known As previously mentioned, the main drawback of the classic
as Vanilla-TFC (V-TFC)36,37 . The main issue of V-TFC is PINN frameworks is that the DE’s constraints are not analyt-
that when solving PDEs with more than two independent ically satisfied. Hence, these constraints must be learned si-
variables is affected by the "curse of dimensionality," e.g., multaneously with the DE solution within the domain during
the problem formulation and the computational cost become the NN’s training. This leads to having competing objectives
prohibitive. To this end, in38 and39 , the authors propose the during the network’s training, which, as explained above, can
use of NN as the free function, as NN is not affected by the cause the PINN’s failure in learning the DE’s correct solution.
curse of dimensionality. Therefore, these TFC frameworks, The method we employ in this work completely removes this
as they employ NNs to learn DEs solutions, can be labeled issue by combining NN with the TFC. TFC allows satisfying
as PINNs frameworks, and they are known as PINN-TFC the equation constraints analytically. The PINN-TFC method
based methods39 . In38 deep NNs are used as free functions, employed in this work is X-TFC. The following subsection
and the framework is called Deep-TFC. In39 shallow NN gives a detailed explanation of how X-TFC works for solving
trained via ELM algorithm are used as the free function, a general linear ODE.
and the framework is called Extreme-TFC (X-TFC). X-TFC
is already been successfully applied to learn the solutions
of optimal control problems for space applications40–46 , A. X-TFC Approach to Solving First Order Linear ODEs
numerical integration31 , problems in transport theory such
as radiative transfer47 and rarefied gas dynamics48,49 , and
inverse problems for epidemiological models50 . Consider the following ODE as example:
After this introduction, we present an overview on PINNs and
X-TFC framework, and how the latter is used to solve linear D(t, u, u̇) = 0 subject to: u(t0 ) = u0 (1)
ODEs, in section II. Section III presents the formulation of
The CE defined in the TFC framework is built accordingly
the Couette flow in a plane channel, and the X-TFC formu-
to approximate the solution of the unknown function u(t) as
lation of the problem is shown in section III A. Section IV
follows:
presents the numerical results of several physical quantities
on interest with discussions, and finally conclusions are stated n
in section V. u(t) = g(t) + ∑ ηk pk (t) = g(t) + η T p(t) (2)
k=1

where, n is the number of constraints, pk (t) are assigned lin-

early independent functions and the g(t) is the free-chosen
function. The values of the ηk coefficients are retrieved by

116
Couette flow of a rarefied gas with Physics-Informed Neural Networks 3

forcing a set of n linear constraints in u(t), to analitically sat-

isfy the DE’s constraints, regardless of the choice of the free
function g(t). When facing a linear ODE with initial condi-
tion, the CE is built as follows:

u(t) = g(t) + η p(t) (3)

where p(t) = 1 (for the details about this selection, see

ref.29,30 ). Hence the CE is:

u(t) = g(t) + η (4)

By forcing the initial condition, we have:

u(t0 ) = u0 = g0 + η =⇒ η = u0 − g0 (5)

which is substituted into Eq. (4) to get the CE with embedded FIG. 1: Single-layer Neural Network with L neurons.
constraints:

u(t) = g(t) + (u0 − g0 ) (6) Where Pj for j + 1, ..., L is the Chebyshev polynomial of order
representing all the possible functions satisfying the con- j. Figure (1) shows the schema of the single-layer NN used
straints. Then, the derivative is: in this manuscript as CE’s free-chosen function. When the
activation functions are defined on an inconsistent domain z ∈
u̇(t) = ġ(t) (7) [z0 , z f ], the independent variable t ∈ [t0 ,t f ] needs to be mapped
into the z domain, with a linear transformation, such as:
The CE in the form of Eq. (6) and its derivative are replaced
into the ODE to get: 1
z = z0 + c(t − t0 ) ←→ t = t0 + (z − z0 )
c
D (t, u, u̇) = 0 (8)
where the mapping coefficient is defined as:
that is now an unconstrained problem. By plugging Eq. (6)
z f − z0
into Eq. (8), we obtain a new DE as a function of t and g(t), c=
i.e.: t f − t0

D̃ (t, g, ġ) = 0 (9) According to the derivative chain rule, the nth derivative of
Eq. (10) is defined as,
In this work, we employ a single-layer NN as free function
g(t), such as: dn g(t) dn σ(z)
= cn β T , (12)
 T dt n dzn
L
σ (w1t + b1 )
 ..  After discretizing the t domain with M discretization points,
g(t) = ∑ β j σ (w j t + b j ) =  .
T
 β = σ β (10) the unconstrained optimization problem will have the follow-
j=1
σ (wLt + bL ) ing form

where L is the number of neurons, w j ∈ R is the jth input D̄(β) = 0 (13)

weight connecting the input node with the jth neuron, β j ∈ R
with j = 1, ..., L is the jth output weight connecting the output and since it is a linear problem, can be rearranged into a linear
node with the jth neuron, b j is the bias of the jth neuron, and system of algebraic equations Aβ = b, which can be solved
σ j (·) is the NN’s activation function, which is selected by the via least-squares methods to find the optimal NN’s output
user. As mentioned before, this is a single-layer NN trained weights vector β. To better understand the steps of the X-TFC
via ELM algorithm27 , thus the only unknown parameters of framework for linear ODEs, a simplified schema is reported in
Eq. (10) that need to be computed are the output weights Fig. 2.
β = [β1 , ..., βL ]T . When using a Chebyshev NN (ChNN), the
choice of input weights and biases that guarantees the best ac- III. COUETTE FLOW IN A PLANE CHANNEL: X-TFC
curacy is to set w j = 1 and b j = 0 , ∀ j = 1, ..., L53 . For this PROBLEM FORMULATION
case, Eq. (10) can be rewritten as follows:
 T In this work, we consider a steady Couette flow confined
P1 (t)
L between two parallel plates in relative motion between them,
g(t) = ∑ β j Pj (t) =  ...  β = P T β
 
(11) with opposite velocities uw (here considered equal to 1), as
j=1
PL (t) shown in Fig. 3. This relative motion imposes a shear stress

117
Couette flow of a rarefied gas with Physics-Informed Neural Networks 4

FIG. 2: Schematic of X-TFC framework to solve linear ODEs with one constraint in one point.

With some assumptions on n∞ , T∞ , and ū54 , and converting

the velocity v to the non-dimensional velocity variable
 
m 1/2
c=v
2kT0
we can substitute Eq. (15) into Eq. (14) to get
    
3 3
cx c2 − Kx + Rx + 2cz K0 + cz c2 − Kz +
FIG. 3: Couette microchannel flow. 2 2
∂ h(x, c)
cz Rz + cx +V (c)h(x, c) =
∂x Z
on the fluid which causes its flow. The plane parallel walls are 02
separated by a distance d, or two semi-distances a. The prob- e−c K(c, c0 )h(x, c0 )dc0 (17)
lem we aim to solve follows the formulation of the Couette
where
flow as presented by Williams54 . The full non-linear Boltz-
 1/2
mann equation for an atomic gas can be written as m
K0 = K
2kT0
v · ∇r f (r, v) = J( f , f 0 ) (14)
Here, Rx,z and Kx,z represent the relative density and temper-
ature gradients in the x and z directions, respectively, and K
where J is the collision operator defined in Refs.55,56 ,
and
is the gradient of cz in the x-direction. V (c) is the collision
f (r, v) is the space and velocity distribution function of the
frequency, which is assumed constant and independent on the
atoms
velocity for the BGK model:
f (r, v) = f0 (r, v)[1 + h(r, v)] (15) V (c) = 1
Finally, the previous equation becomes54
Here, h is a perturbation to the local Maxwellian f0 (r, v)
    
caused by the plates. The general form of the local 3 3
Maxwellian is cx c2 − Kx + Rx + 2cz K0 + cz c2 − Kz +
2 2
 3/2 ∂ h(x, c)
m cz Rz + cx + h(x, c) =
f0 (r, v) = n∞ (x, z)  ∂x   
2πkT∞ (x, z) Z
02 2 2 3 2 3
  = e−c h(x, c0 ) 1 + 2c · c0 + c − c0 − dc0
m  2 3 2 2
exp − vx + v2y + [vz − ū(x)]2 (16)
2πkT∞ (x, z) (18)

118
Couette flow of a rarefied gas with Physics-Informed Neural Networks 5

When a fluid dynamics problem involves the velocity of the for u ∈ (0, ∞). The weight function Ψ(u) is defined as follows
gas rather than the temperature, by definition the gas velocity 2
in the z-direction is given by Ψ(u) = π −1/2 e−u (27)
R
vz f (x, v)dv Therefore, this work aims to solve the problem given by Eqs.
q(x) = R (19) (25) and (26) using the X-TFC framework.
f (x, v)dv
Now, assuming that the temperature and density are constant,
and linearizing, we get A. X-TFC formulation of the problem
Z  
1 mvz Kx
q(x) = f00 (v)vz h(x, v) + dv (20) Considering the integro-differential equation (25) with con-
n0 kT0 straints (26), we rewrite them according to the mapping
where f00 (v) is the equilibrium Maxwellian distribution func- Z ∞
∂ Z(χ, u)
tion. Eq. (20) reduces to cu + Z(χ, u) = Ψ(u0 )Z(χ, u0 )du0 (28)
∂χ −∞
Z ∞Z ∞Z ∞
1 2
By following Ref.22 , it is convenient to break the equation in
q(x) = K0 x+ cz e−c h(x, cx , cy , cz )dcx dcy dcz
π 3/2 −∞ −∞ −∞ two parts and eliminate the dependence on the velocity u by
(21) we can discretizing it into N points:
that is in units of (2kT0 /m)1/2 . Let’s introduce the following
function d
Z Z cui Z(χ, ui ) + Z(χ, ui )
1 ∞ ∞ 2 2 dχ
Z(x, cx ) = cz e−cy −cz h(x, cx , cy , cz )dcy dcz (22) N
π 1
∑ ωk uk Ψ(uk ) [Z(χ, uk ) + Z(χ, −uk )]
−∞ −∞
= (29)
then, Eq. (21) can be rewritten as k=1
Z ∞
1 2
q(x) = K0 x + e−cx Z(x, cx )dcx (23) d
π 1/2 −∞ − cui Z(χ, −ui ) + Z(χ, −ui )
dχ
Now, we can multiply Eq. (18) by cx and integrate over cy and N
1
cx , we get = ∑ ωk uk Ψ(uk ) [Z(χ, uk ) + Z(χ, −uk )] (30)
  k=1
1 1 1 ∂ Z(x, cz )
K0 cx + Rz + Kz c2x + + cx + Z(x, cx ) = In this way, we are considering that the N weights ωk and
2 2 2 ∂x quadrature points uk are defined for use on the integration in-
Z ∞
1 02 terval [0, ∞). We can use a further mapping transformation to
√ e−cx Z(x, c0x )dc0x (24)
π −∞ map the interval [0, ∞) onto [0, 1] by using the new following
variable:
As we mentioned in the beginning of this section, the problem
for the Couette flow is to have the upper and lower plates trav- u = − ln(|µ|)
eling at velocity uw in the directions z and −z, respectively.
1
Thus, the relative density and temperature gradients in the ith- which leads to du = − dµ, and the characteristic function
direction, Ri and Ki , are null. Now, we can make use of the |µ|
extension54 of the formulations adopted in references9 and6 to Ψ(u) becomes
the case of the accommodation coefficient α ranging from 0 2

to 1, and so we look for a solution of following linear integro- Ψ(µ) = π −1/2 e−(− ln(|µ|))
differential equation: in order to use a Gauss–Legendre scheme mapped onto the
∂ Z(τ, u)
Z ∞ interval [0, 1].
u + Z(τ, u) = Ψ(u0 )Z(τ, u0 )du0 (25) We can reformulate our problem as
∂τ −∞
d
where τ ∈ [0, a] is the half-width of the channel (thanks to the − c ln(µi ) Z(χ, µi ) + Z(χ, µi )
spatial symmetry of the problem, which will ease the com- dχ
putational efforts), u ∈ (−∞, ∞] is the cosine direction of the N
1
gas molecules, and Z(τ, µ) is the unknown function of the dif- = ∑ ωk µk Ψ(µk ) [Z(χ, µk ) + Z(χ, −µk )] (31)
k=1
ferential equation, representing the velocity momentum. The
problem is then subject to the following boundary conditions,
whose derivations can be found by the interested reader in d
c ln(µi ) Z(χ, −µi ) + Z(χ, −µi )
Ref.54 : dχ
( N
Z(0, u) = −Z(0, −u) 1
(26) = ∑ ωk µk Ψ(µk ) [Z(χ, µk ) + Z(χ, −µk )] (32)
Z(a, −u) = (1 − α)Z(a, u) − α k=1

119
Couette flow of a rarefied gas with Physics-Informed Neural Networks 6

T +
subject to: f = σf β + η
Z+ +
Z0− = σ0T β − + η −
(
Z(0, µi ) = −Z(0, −µi ) that replaced in the Eqs. (36) we have
(33) (
Z(a, −µi ) = (1 − α)Z(a, µi ) − α
σ0T βi+ + ηi+ = σ0T βi− + ηi−
(40)
Here, we take advantage of the symmetry of the flow in the σfT βi− + ηi− = (1 − α) σfT βi+ + (1 − α) ηi+ − α
channel, given by the first expression in the system (33) above.
For the reader’s convenience, we suppress the variables’ de- that gives us
pendence of the unknown functions by making use of +,− su- (
perscripts, and i,k subscripts ηi+ = [θ σ0 + θ (1 − α)σf ]T βi+ + [θ σ0 − θ σf ]T βi− − θ α
N
ηi− = [−ϕσ0 + θ (1 − α)σf ]T βi+ + [−ϕσ0 + θ σf ]T βi− + θ α
d + 1

−c ln(µi ) Zi + Zi+ = ∑ ωk Ψ(µk ) Zk+ + Zk− (34) (41)
dχ k=1 µk where θ and ϕ are parameters introduced to simplify the no-
d − N
1
tation:
c ln(µi ) Zi + Zi− = ∑ ωk Ψ(µk ) Zk+ + Zk− (35)
dχ µk 1 α −1
k=1 θ= ϕ=
α −2 α −2
subject to:
( Now that ηi± are found, the CEs can be built in the following
Z0+ = −Z0− form:
(36)
Z−f = (1 − α)Z f − α
+

T
Zi+ = (σ + θ σ0 − ϕσf )T βi+ + θ σ0 − σ f βi− − θ α
Since we are dealing with a problem made up of two ODEs
T
T
Zi− = ϕ σ f − σ0 βi+ + σ − ϕσ0 + θ σ f βi− + θ α
with one boundary condition each, we can build the con- (42)
strained expressions according to Eq. (2): and replacing them in the DEs of the problem, we get:
y(τ) = g(τ) + η p(τ). (37) T
(−c ln(µi )σ 0 + σ + θ σ0 − ϕσf ) βi+
The vector p(τ) is chosen to be a vector of 1s, according to N
1
Mortari28 , and the free function g(τ) is the single-layer NN: + θ (σ0 − σf )T βi− − ∑ ωk Ψ(µk )
k=1 µk
n o
g(τ) = σ T (τ)β (38) [σ + (θ − ϕ)σ0 ]T βk+ + [σ + (θ − ϕ)σ0 ]T βk− = θ α
Since the activation functions range in particular domains, it (43)
is appropriate to map the independent variable τ to a new in-
dependent variable, χ, that ranges in the activation function
T
domain, e.g. [−1, 1]. Thus, χ is defined as (c ln(µi )σ 0 + σ − ϕσ0 + θ σf ) βi−
N
χ = c(τ + a) − 1 1
+ ϕ (σ0 − σf )T βi+ − ∑ ωk Ψ(µk )
k=1 µ k
where the constant c is called mapping constant, that is n o
[σ + (θ − ϕ)σ0 ]T βk+ + [σ + (θ − ϕ)σ0 ]T βk− = −θ α
χ f − χ0 1 (44)
c= =
τ f − τ0 a
Thus, the original boundary value constrained problem has
It only comes into action when we need to compute the deriva- been turned into an unconstrained optimization problem
tive of the unknown function. Then, our mapping transforma- Aβ = B, where the unknown vector β is computed via Least-
tion gives us Squares, and the solutions are computed according to CEs ex-
pressed in Eqs. (42). More specifically, β = [β + , β − ] .
∂ ∂
Z(τ, u) = Z(χ, u) and Z(τ, u) = c Z(χ, u)
∂τ ∂χ
IV. RESULTS AND DISCUSSIONS
According to Eq. (37), the constrained expressions are

Z ± (χ) = σ ± β ± + η ± (39) A qualitative plot of the velocity momentum Z(µ, τ) for

a = 0.5 and α = 1 is given in Figure 4, obtained with acti-
To find the form of the ηi± , we use the boundary conditions of vation function tanh and showing the spatial symmetry of the
the problem as follows Couette flow. In the Couette problem, two quantities are in-
teresting to compute to prove the efficiency of a new numeri-
Z0+ = σ0T β + + η + Z− T −
f = σf β + η
−
cal method. One involves the gas molecular velocity between

120
Couette flow of a rarefied gas with Physics-Informed Neural Networks 7

which we believe to be accurate up to the 7th digit given the

similarity with the thermal creep problem already solved with
the same method48,49 . The parameters used and the compu-
tational times are shown in Table III, and obtained with Mat-
lab R2021a with an Intel Core i7 - 9700 CPU PC with 64
GB of RAM. In the plots of Figure 5, the trend of the macro-
scopic velocity profile is plotted for two different case scenar-
ios: in Figure 5a the value of the accommodation coefficient
is fixed (α = 1) while the channel width 2a (inverse Knud-
sen number) varies in the range [0.1, 100]. As one can see,
the velocity profile reaches higher values for greater inverse
Knudsen numbers. In Figure 5b, the inverse Knudsen number
is fixed (2a = 2), while the accommodation coefficient varies
in a range [0.2, 1]. As one can expect, the velocity profile
reaches higher values for greater accommodation coefficients
since the latter quantifies the interaction between the fluid and
FIG. 4: Velocity momentum in a channel of semi-width the wall, or also the degree to which the gas molecules are ac-
a = 0.5 and accommodation coefficient α = 1. commodated to the solid surface57 . The second macroscopic
quantity of interest consists in the pressure tensor54 , or stress
tensor, caused in the gas by the motion of the plates55 . Its xz
the channel’s walls. The definition of its velocity in the z- component is given by
direction, as previously mentioned, is given by
Z ∞Z ∞Z ∞
Z ∞ n0 2kT0 2
1 2 Pxz = cx cz e−c h(x, c)dcx dcy dcz (51)
q(x) = K0 x + e−cx Z(x, cx )dcx (45) π 3/2 m −∞ −∞ −∞
π 1/2 −∞

For the Couette flow assumptions8 , it becomes and following Barichello et al.14 , we wish to compute the
(constant) normalized stress in the form
Z ∞
1 2 Z ∞
q(x) = e−u Z(x, u)du (46)
π 1/2 −∞ Pxz = π 1/2 Ψ(u)Z(τ, u)u du (52)
−∞
which we rewrite as
Z ∞ Again, we make use of the change of variable for the velocity
q(τ) = Ψ(u)Z(τ, u)du for τ ∈ [0, a] (47) as
−∞
Z 1
1
By making use of change of variable we get Pxz = π 1/2 − Ψ(µ)Z(τ, µ) ln(|µ|)dµ (53)
−1 µ
Z 1
1
q(τ) = Ψ(µ)Z(τ, µ)dµ (48) which can be computed with a Gauss-Legendre quadrature:
−1 µ
N  
which can be computed with a Gauss-Legendre quadrature: 1 + 1 −
Pxz = π 1/2 ∑ −ωi Ψ(µi ) ln(µi ) Z + Z (54)
i=1 µi i −µi i
N
1

q(τ) = ∑ ωi Ψ(µi ) Zi + + Zi − (49)
i=1 µ i Our results are reported in Table IV for a wide range of inverse
Knudsen numbers (from 10−7 to 20). By producing these re-
Once we have the expression of the velocity profile, the mass sults, we can confirm the accuracy of the results reported in
flow rate for half channel can be easily calculated as follows: Ref.14 up to 7th digit. Also, the comparison with other two
Z a benchmarks produced by Loyalka et al.9 and Willis18 , where
1
Q= q(τ)dτ (50) the number of digits of accuracy is limited, is reported.
2a2 0

In Tables Ia and Ib, we show the compatibility of our results

with the ones provided by Scherer et al.20 . For the velocity V. CONCLUSIONS
profile, the results are accordant up to the fifth digit almost
in all the channel points, while for the mass flow rate, all the This work highlighted the accuracy, efficiency, and ro-
digits of accuracy are reached for each channel width. Since bustness of the Extreme Theory of Functional Connections
there is not a vast availability of benchmarks in the literature framework39 to solve classic flow problems in rarefied-gas dy-
for the BGK model of Couette flow, we propose here a new namics, in particular the Couette flow. The accuracy has been
benchmark for the macroscopic velocity for a wide range of proved by comparison with benchmarks published by Scherer
Knudsen numbers and different values of accommodation co- et al.20 , Barichello et al.14 , Loyalka et al.9 , and Willis18 ,
efficient α. The results we print here are shown in Table II, matching their solutions up to the 7th digit. In this work, we

121
Couette flow of a rarefied gas with Physics-Informed Neural Networks 8

TABLE I: Comparison between X-TFC results and

benchmarks proposed in Ref.20

(a) The macroscopic velocity profile q(τ) for a plane channel of width
2a = 1 and α = 1.

τ/a X-TFC Ref.20

0.0 0.0 0.0
0.1 –4.44498(–2) –4.44498(–2)
0.2 –8.90639(–2) –8.90638(–2)
0.3 –1.34020(–1) –1.34019(–1)
0.4 –1.79526(–1) –1.79525(–1)
0.5 –2.25844(–1) –2.25844(–1)
0.6 –2.73338(–1) –2.73338(–1)
0.7 –3.22559(–1) –3.22558(–1)
0.8 –3.74467(–1) –3.74467(–1)
0.9 –4.31190(–1) –4.31189(–1)
1.0 –5.03723(–1) –5.03722(–1)
(a) Macroscopic velocity profiles for different
values of a, when α = 1. (b) The mass flow rate Q(τ) for α = 1.

2a X-TFC Ref.20
0.10 –6.85779(–1) –6.85779(–1)
1.00 –2.32188(–1) –2.32188(–1)
10.0 –4.22811(–2) –4.22811(–2)

X-TFC method has been proved to be efficient and accurate

in solving PDEs without being affected by the curse of di-
mensionality. Therefore, we are currently working on solv-
ing high-dimensional transport theory problems such as two-
angle Radiative Transfer, time-dependent Radiative Transfer,
(b) Macroscopic velocity profiles for different 3D Radiative Transfer, and various problems in rarefied-gas
values of α, when a = 1. dynamics and neutron transport.

FIG. 5: Qualitative behaviour of the macroscopic velocity

profile for a given accommodation coefficient α (left) and for FUNDING
a given channel width (right).
L.B. Barichello acknowledges partial financial support
from Conselho Nacional de Desenvolvimento Científico e
Tecnológico - Brasil (CNPq).
presented a new benchmark for the macroscopic velocity pro-
file on the wall for a wide range of inverse Knudsen numbers
and different values of the accommodation coefficient, obtain- AVAILABILITY OF DATA AND MATERIALS
ing accuracy that we believe to be up to the 7th digit. Also, the
pressure tensor has been evaluated, obtaining a precise bench-
mark according to Ref.14 . Data sharing is not applicable to this article as no new data
This work has proven the feasibility of this new PINN method were created or analyzed in this study.
(that uses TFC constrained expressions, thanks to which the
boundary values are analytically satisfied) to solve parti-
cle transport problems arising from the Boltzmann integro- COMPETING INTERESTS
differential equation. The BGK model has been used to ap-
proximate the Boltzmann equation, but more problems us- The authors declare that they have no competing interests.
ing other models are already under investigation. Also, the

122
Couette flow of a rarefied gas with Physics-Informed Neural Networks 9

1 P. L. Bhatnagar, E. P. Gross, and M. Krook, “A model for collision pro- 28 D. Mortari, “The theory of connections: Connecting points,” Mathematics
cesses in gases. i. small amplitude processes in charged and neutral one- 5, 57 (2017).
component systems,” Physical review 94, 511 (1954). 29 D. Mortari, “Least-squares solution of linear differential equations,” Math-
2 P. Welander, “On the temperature jump in a rarefied gas,” Arkiv fysik 7 ematics 5, 48 (2017).
(1954). 30 D. Mortari, H. Johnston, and L. Smith, “High accuracy least-squares so-
3 S. Harris, An introduction to the theory of the Boltzmann equation (Courier lutions of nonlinear differential equations,” Journal of Computational and
Corporation, 2004). Applied Mathematics 352, 293 – 307 (2019).
4 C. Cercignani, “The Boltzmann equation,” in The Boltzmann equation and 31 M. De Florio, E. Schiassi, A. D’Ambrosio, D. Mortari, and R. Furfaro,

its applications (Springer, 1988) pp. 40–103. “Theory of functional connections applied to linear odes subject to integral
5 C. Cercignani and A. Daneri, “Flow of a rarefied gas between two parallel constraints and linear ordinary integro-differential equations,” Mathemati-
plates,” Journal of Applied Physics 34, 3509–3513 (1963). cal and Computational Applications 26, 65 (2021).
6 C. Cercignani, “Plane Poiseuille flow according to the method of elemen- 32 D. Mortari and C. Leake, “The Multivariate Theory of Connections,”

tary solutions,” Journal of Mathematical Analysis and Applications 12, MDPI Mathematics 7, 296 (2019).
254–262 (1965). 33 C. Leake and D. Mortari, “An Explanation and Implementation of Mul-
7 V. Boffi, L. De Socio, G. Gaffuri, and C. Pescatore, “Rigorous constructive tivariate Theory of Connections via Examples,” in 2019 AAS/AIAA As-
solution to monodimensional poiseuille and thermal creep flows,” Mecca- trodynamics Specialist Conference, Portland, MN, August 11–15, 2019
nica 11, 183–190 (1976). (AAS/AIAA, 2019).
8 S. Loyalka, N. Petrellis, and T. Storvick, “Some numerical results for 34 A. Gil, J. Segura, and N. Temme, Numerical Methods for Special Func-

the BGK model: Thermal creep and viscous slip problems with arbitrary tions (Society for Industrial and Applied Mathematics, 2007) pp. 51–80.
accomodation at the surface,” The Physics of Fluids 18, 1094–1099 (1975). 35 C. Lanczos, Applied Analysis (Dover Publications, Inc., New York, 1957)
9 S. Loyalka, N. Petrellis, and T. Storvick, “Some exact numerical results p. 453.
for the BGK model: Couette, Poiseuille and thermal creep flow between 36 H. Johnston, “The theory of functional connections: A journey from theory

parallel plates,” Zeitschrift für angewandte Mathematik und Physik ZAMP to application,” arXiv preprint arXiv:2105.08034 (2021).
30, 514–521 (1979). 37 C. Leake, “The multivariate theory of functional connections: An n-
10 S. Loyalka, “Thermal transpiration in a cylindrical tube,” The Physics of dimensional constraint embedding technique applied to partial differential
Fluids 12, 2301–2305 (1969). equations,” arXiv preprint arXiv:2105.07070 (2021).
11 S. Loyalka, “Comments on Poiseuille flow and thermal creep of a rarefied 38 C. Leake and D. Mortari, “Deep Theory of Functional Connections: A

gas between parallel plates,” The Physics of Fluids 17, 1053–1055 (1974). New Method for Estimating the Solutions of Partial Differential Equa-
12 C. Siewert, R. Garcia, and P. Grandjean, “A concise and accurate solution tions,” Machine Learning and Knowledge Extraction 2, 37–55 (2020).
for poiseuille flow in a plane channel,” Journal of Mathematical Physics 39 E. Schiassi, R. Furfaro, C. Leake, M. De Florio, H. Johnston, and

21, 2760–2763 (1980). D. Mortari, “Extreme Theory of Functional Connections: A Fast Physics-
13 L. Barichello and C. Siewert, “A discrete-ordinates solution for poiseuille Informed Neural Network Method for Solving Ordinary and Partial Dif-
flow in a plane channel,” Zeitschrift für angewandte Mathematik und ferential Equations,” Neurocomputing (2021).
Physik ZAMP 50, 972–981 (1999). 40 R. Furfaro and D. Mortari, “Least-squares solution of a class of optimal
14 L. Barichello, M. Camargo, P. Rodrigues, and C. Siewert, “Unified solu- space guidance problems via Theory of Connections,” Acta Astronautica
tions to classical flow problems based on the BGK model,” Zeitschrift für (2019), https://doi.org/10.1016/j.actaastro.2019.05.050.
angewandte Mathematik und Physik ZAMP 52, 517–534 (2001). 41 H. Johnston, E. Schiassi, R. Furfaro, and D. Mortari, “Fuel-efficient pow-
15 Y. Sone and Y. Sone, Kinetic theory and fluid dynamics (Springer, 2002). ered descent guidance on large planetary bodies via theory of functional
16 F. Sharipov and V. Seleznev, “Data on internal rarefied gas flows,” Journal connections,” arXiv preprint arXiv:2001.03572 (2020).
of Physical and Chemical Reference Data 27, 657–706 (1998). 42 E. Schiassi, A. D’Ambrosio, H. Johnston, R. Furfaro, F. Curti, and
17 E. Gross, E. Jackson, and S. Ziering, “Boundary value problems in kinetic D. Mortari, “Complete Energy Optimal Landing on Planetary Bodies via
theory of gases,” Annals of Physics 1, 141–167 (1957). Theory of Functional Connections,” Acta Astronautica - in preparation
18 D. R. Willis, “Comparison of kinetic theory analyses of linearized couette (2020).
flow,” The Physics of Fluids 5, 127–135 (1962). 43 K. Drozd, R. Furfaro, E. Schiassi, H. Johnston, and D. Mortari, “Energy-
19 C. Siewert and D. Valougeorgis, “An analytical discrete-ordinates solution optimal trajectory problems in relative motion solved via theory of func-
of the S-model kinetic equations for flow in a cylindrical tube,” Journal of tional connections,” Acta Astronautica 182, 361–382 (2021).
Quantitative Spectroscopy and Radiative Transfer 72, 531–550 (2002). 44 A. D’Ambrosio, E. Schiassi, F. Curti, and R. Furfaro, “Pontryagin neu-
20 C. S. Scherer, J. F. Prolo Filho, and L. B. Barichello, “An analytical ap- ral networks with functional interpolation for optimal intercept problems,”
proach to the unified solution of kinetic equations in rarefied gas dynamics. Mathematics 9, 996 (2021).
I. Flow problems,” Zeitschrift für angewandte Mathematik und Physik 60, 45 E. Schiassi, A. D’Ambrosio, A. Scorsoglio, R. Furfaro, and F. Curti,

70–115 (2009). “Class of optimal space guidance problems solved via indirect methods
21 C. Siewert, “Poiseuille, thermal creep and Couette flow: results based on and physics-informed neural networks,” (2021).
the CES model of the linearized Boltzmann equation,” European Journal 46 A. D’Ambrosio, E. Schiassi, F. Curti, and R. Furfaro, “Physics-informed

of Mechanics-B/Fluids 21, 579–597 (2002). neural networks applied to a series of constrained space guidance prob-
22 L. Barichello and C. E. Siewert, “A discrete-ordinates solution for a non- lems,” (2021).
grey model with complete frequency redistribution,” Journal of Quantita- 47 M. De Florio, E. Schiassi, R. Furfaro, B. D. Ganapol, and D. Mostacci,

tive Spectroscopy and Radiative Transfer 62, 665–675 (1999). “Solutions of Chandrasekhar’ s Basic Problem in Radiative Transfer via
23 G. C. Wick, “Über ebene diffusionsprobleme,” Zeitschrift für Physik 121, Theory of Functional Connections,” Journal of Quantitative Spectroscopy
702–718 (1943). and Radiative Transfer , 107384 (2020).
24 S. Chandrasekhar, Radiative transfer (Courier Corporation, 2013). 48 M. De Florio, E. Schiassi, B. D. Ganapol, and R. Furfaro, “Physics-
25 M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neu- informed neural networks for rarefied-gas dynamics: Thermal creep flow
ral networks: A deep learning framework for solving forward and in- in the Bhatnagar–Gross–Krook approximation,” Physics of Fluids 33,
verse problems involving nonlinear partial differential equations,” Journal 047110 (2021).
of Computational Physics 378, 686–707 (2019). 49 M. De Florio, E. Schiassi, B. D. Ganapol, and R. Furfaro, “Physics-
26 S. Wang, Y. Teng, and P. Perdikaris, “Understanding and mitigating gra- informed neural networks for rarefied-gas dynamics: Poiseuille flow in the
dient pathologies in physics-informed neural networks,” arXiv preprint bgk approximation,” Zeitschrift für angewandte Mathematik und Physik
arXiv:2001.04536 (2020). 73, 1–18 (2022).
27 G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “ Extreme learning machine: 50 E. Schiassi, M. De Florio, A. D’ambrosio, D. Mortari, and R. Furfaro,

Theory and applications ,” Neurocomputing 70, 489–501 (2006). “Physics-informed neural networks and functional interpolation for data-
driven parameters discovery of epidemiological compartmental models,”

123
Couette flow of a rarefied gas with Physics-Informed Neural Networks 10

Mathematics 9, 2069 (2021). 78 C. Cercignani, Theory and application of the Boltzmann equation (Scottish
51 F. Jacob and B. Ted, A first course in finite elements (Wiley, 2007). Academic Press, 1975).
52 S. Mishra and R. Molinaro, “Estimates on the generalization error of 79 B. D. Ganapol, “The response matrix discrete ordinates solution to the
physics-informed neural networks for approximating a class of inverse 1d radiative transfer equation,” Journal of Quantitative Spectroscopy and
problems for PDEs,” IMA Journal of Numerical Analysis (2021). Radiative Transfer 154, 72–90 (2015).
53 M. Liu, M. Hou, J. Wang, and Y. Cheng, “Solving two-dimensional lin- 80 C. Siewert, “A concise and accurate solution to chandrasekhar’s basic

ear partial differential equations based on Chebyshev neural network with problem in radiative transfer,” Journal of Quantitative Spectroscopy and
extreme learning machine algorithm,” Engineering Computations (2020). Radiative Transfer 64, 109–130 (2000).
54 M. Williams, “A review of the rarefied gas dynamics theory associated 81 R. Furfaro and D. Mortari, “Least-squares solution of a class of optimal

with some classical problems in flow and heat transfer,” Zeitschrift für space guidance problems via theory of connections,” Acta Astronautica
angewandte Mathematik und Physik ZAMP 52, 500–516 (2001). 168, 92–103 (2020).
55 C. Cercignani et al., Mathematical methods in kinetic theory (Springer, 82 B. D. Ganapol, “1D thermal creep channel flow in the BGK approximation

1969). by adding and doubling,” Annals of Nuclear Energy 134, 441–451 (2019).
56 M. M. Williams, “Mathematical methods in particle transport theory.” 83 T. Kanki and S. Iuchi, “Poiseuille flow and thermal creep of a rarefied gas

(1971). between parallel plates,” The Physics of Fluids 16, 594–599 (1973).
57 S. K. Prabha and S. P. Sathian, “Molecular-dynamics study of poiseuille 84 I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for

flow in a nanochannel and calculation of energy and momentum accom- solving ordinary and partial differential equations,” IEEE Transactions on
modation coefficients,” Physical Review E 85, 041201 (2012). Neural Networks 9, 987–1000 (1998).
58 M. Abramowitz and I. Stegun, “Handbook of Mathematical Functions,” 85 R. Fletcher, Practical Methods of Optimization, 2nd ed. (Wiley, New York,

(Dover, 1970) p. 3.3.7, 9th ed. 1987).

59 T. Barker, D. G. Schaeffer, P. Bohórquez, and J. M. N. T. Gray, “Well- 86 S. A. Coons, “SURFACES FOR COMPUTER-AIDED DESIGN OF

posed and ill-posed behaviour of the µ(I)-rheology for granular flows,” J. SPACE FORMS,” Tech. Rep. (Massachusetts Institute of Technology,
Fluid. Mech. 779, 794–818 (2015). Cambridge, MA, USA, 1967).
60 G. Batchelor, “Small-scale variation of convected quantities like temper- 87 J. Sirignano and K. Spiliopoulos, “DGM: A deep learning algorithm for

ature in turbulent fluid Part 1. General discussion and the case of small solving partial differential equations,” (2018), arXiv:1708.07469v5.
conductivity.” J. Fluid Mech. 5, 113–133 (1971). 88 J. Argyris and S. Kelsey, “Energy Theorems and Structural Analysis: A
61 A. V. Briukhanov, S. S. Grigorian, S. M. Miagkov, M. Y. Plam, I. E. Generalized Discourse with Applications on Energy Principles of Struc-
Shurova, M. E. Eglit, and Y. L. Yakimov, “On some new approaches tural Analysis Including the Effects of Temperature and Non-Linear
to the dynamics of snow avalanches,” in Physics of Snow and Ice, Pro- Stress-Strain Relations,” Aircraft Engineering and Aerospace Technology
ceedings of the International Conference on Low Temperature Science, 26, 347–356 (1954), https://doi.org/10.1108/eb032482.
Vol. 1 (Institute of Low Temperature Science, Hokkaido University, Sap- 89 M. J. Turner, R. W. Clough, H. C. Martin, and L. J. Topp, “Stiffness and

poro, Hokkaido, Japan, 1967) pp. 1221–1241. Deflection Analysis of Complex Structures,” Journal of the Aeronautical
62 C. Brownell and L. Su, “Planar measurements of differential diffusion in Sciences 23, 805–823 (1956).
turbulent jets,” AIAA Paper 2004-2335 (2004). 90 R. W. Clough, The finite element method in plane stress analysis (Ameri-
63 C. Brownell and L. Su, “Scale relations and spatial spectra in a differen- can Society of Civil Engineers, 1960) pp. 345–378.
tially diffusing jet,” AIAA Paper 2007-1314 (2007). 91 J. N. Reddy, “An Introduction to the Finite Element Method,”
64 S. Dennis, “Compact explicit finite difference approximations to the Journal of Pressure Vessel Technology 111, 348–349 (1989),
Navier–Stokes equation,” in Ninth Intl Conf. on Numerical Methods in https://asmedigitalcollection.asme.org/pressurevesseltech/article-
Fluid Dynamics, Lecture Notes in Physics, Vol. 218, edited by Soubbara- pdf/111/3/348/5110690/348_1.pdf.
mayer and J. Boujot (Springer, 1985) pp. 23–51. 92 H. Johnston and D. Mortari, “Linear Differential Equations Subject to
65 A. N. Edwards, S. Viroulet, B. P. Kokelaar, and J. M. N. T. Gray, “For- Multivalued, Relative and/or Integral Constraints with Comparisons to
mation of levees, troughs and elevated channels by avalanches on erodible Chebfun,” SIAM Journal of Numerical Analysis (2018), submitted.
slopes,” J. Fluid Mech. 823, 278–315 (2017). 93 A. G. Baydin, B. A. Pearlmutter, and A. A. Radul, “Automatic differ-
66 L.-S. Hwang and E. Tuck, “On the oscillations of harbours of arbitrary entiation in machine learning: a survey,” CoRR abs/1502.05767 (2015),
shape,” J. Fluid Mech. 42, 447–464 (1970). arXiv:1502.05767.
67 D. D. Joseph and J. C. Saut, “Short-wave instabilities and ill-posed initial- 94 TensorFlow™, “Automatic differentiation and gradient tape,”
value problems,” Theoretical and Computational Fluid Dynamics 1, 191– https://www.tensorflow.org/tutorials/eager/automatic_
227 (1990), 10.1007/BF00418002. differentiation (2018), accessed: 11-04-2018.
68 M. Worster, “The dynamics of mushy layers,” in Interactive dynamics of 95 X. Glorot and Y. Bengio, “Understanding the difficulty of training deep

convection and solidification, edited by S. Davis, H. Huppert, W. Muller, feedforward neural networks,” in Proceedings of the Thirteenth Interna-
and M. Worster (Kluwer, 1992) pp. 113–138. tional Conference on Artificial Intelligence and Statistics, Proceedings of
69 W. Koch, “Resonant acoustic frequencies of flat plate cascades,” J. Sound Machine Learning Research, Vol. 9, edited by Y. W. Teh and M. Tittering-
Vib. 88, 233–242 (1983). ton (PMLR, Chia Laguna Resort, Sardinia, Italy, 2010) pp. 249–256.
70 J.-J. Lee, “Wave-induced oscillations in harbours of arbitrary geometry,” 96 D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,”

J. Fluid Mech. 45, 375–394 (1971). CoRR abs/1412.6980 (2014), arXiv:1412.6980.

71 C. Linton and D. Evans, “The radiation and scattering of surface waves by 97 J. Duchi, E. Hazan, and Y. Singer, “Adaptive Subgradient Methods for

a vertical circular cylinder in a channel,” Phil. Trans. R. Soc. Lond. 338, Online Learning and Stochastic Optimization,” J. Mach. Learn. Res. 12,
325–357 (1992). 2121–2159 (2011).
72 P. Martin, “On the null-field equations for the exterior problems of acous- 98 T. Tieleman and G. Hinton, “Lecture 6.5 - RMSProp, COURSERA: Neural

tics,” Q. J. Mech. Appl. Maths 33, 385–396 (1980). Networks for Machine Learning. Technical report, 2012.” (2012).
73 R. Rogallo, “Numerical experiments in homogeneous turbulence,” Tech. 99 C. Leake, H. Johnston, L. Smith, and D. Mortari, “Analytically Embed-

Rep. 81835 (NASA Tech. Mem., 1981). ding Differential Equation Constraints into Least Squares Support Vector
74 F. Ursell, “Surface waves on deep water in the presence of a submerged Machines Using the Theory of Functional Connections,” Machine Learn-
cylinder I,” Proc. Camb. Phil. Soc. 46, 141–152 (1950). ing and Knowledge Extraction 1, 1058–1083 (2019).
75 L. van Wijngaarden, “On the oscillations near and at resonance in open 100 S. Mall and S. Chakraverty, “Single Layer Chebyshev Neural Network

pipes,” J. Engng Maths 2, 225–240 (1968). Model for Solving Elliptic Partial Differential Equations,” Neural Process-
76 P. Miller, Mixing in high Schmidt number turbulent jets, Ph.D. thesis, Cal- ing Letters 45, 825–840 (2017).
ifornia Institute of Technology (1991). 101 H. Sun, M. Hou, Y. Yang, T. Zhang, F. Weng, and F. Han, “Solving Partial
77 J.-Y. Bouguet, “Camera calibration toolbox for matlab (2008),” URL Differential Equation Based on Bernstein Neural Network and Extreme
http://www. vision. caltech. edu/bouguetj/calib_doc 1080 (2008). Learning Machine Algorithm,” Neural Processing Letters 50, 1153–1172

124
Appendix F

De Florio et al. (2022) Stiff Chemical

Kinetics

Reproduced from De Florio, M., Schiassi, E., & Furfaro, R. (2022). Physics-informed neural
networks and functional interpolation for stiff chemical kinetics. Chaos: An Interdisciplinary
Journal of Nonlinear Science, 32(6), 063107. https://doi.org/10.1063/5.0086649 [12], with
the permission of AIP Publishing.

125
Physics-informed neural networks and
functional interpolation for stiff chemical
kinetics
Cite as: Chaos 32, 063107 (2022); https://doi.org/10.1063/5.0086649
Submitted: 27 January 2022 • Accepted: 06 May 2022 • Published Online: 01 June 2022

Mario De Florio, Enrico Schiassi and Roberto Furfaro

126
Chaos 32, 063107 (2022); https://doi.org/10.1063/5.0086649 32, 063107

© 2022 Author(s).
 Chaos ARTICLE scitation.org/journal/cha

Physics-informed neural networks and functional

interpolation for stiff chemical kinetics
Cite as: Chaos 32, 063107 (2022); doi: 10.1063/5.0086649
Submitted: 27 January 2022 · Accepted: 6 May 2022 ·
Published Online: 1 June 2022 View Online Export Citation CrossMark

Mario De Florio,1 Enrico Schiassi,1 and Roberto Furfaro1,2,a)

AFFILIATIONS
1
Department of Systems & Industrial Engineering, The University of Arizona, 1127 James E. Rogers Way, Tucson,
Arizona 85719, USA
2
Department of Aerospace & Mechanical Engineering, The University of Arizona, 1130 N Mountain Ave, Tucson, Arizona 85721, USA

a)
Author to whom correspondence should be addressed: robertof@email.arizona.edu

ABSTRACT
This work presents a recently developed approach based on physics-informed neural networks (PINNs) for the solution of initial value
problems (IVPs), focusing on stiff chemical kinetic problems with governing equations of stiff ordinary differential equations (ODEs). The
framework developed by the authors combines PINNs with the theory of functional connections and extreme learning machines in the
so-called extreme theory of functional connections (X-TFC). While regular PINN methodologies appear to fail in solving stiff systems of
ODEs easily, we show how our method, with a single-layer neural network (NN) is efficient and robust to solve such challenging problems
without using artifacts to reduce the stiffness of problems. The accuracy of X-TFC is tested against several state-of-the-art methods, showing
its performance both in terms of computational time and accuracy. A rigorous upper bound on the generalization error of X-TFC frameworks
in learning the solutions of IVPs for ODEs is provided here for the first time. A significant advantage of this framework is its flexibility to
adapt to various problems with minimal changes in coding. Also, once the NN is trained, it gives us an analytical representation of the solution
at any desired instant in time outside the initial discretization. Learning stiff ODEs opens up possibilities of using X-TFC in applications with
large time ranges, such as chemical dynamics in energy conversion, nuclear dynamics systems, life sciences, and environmental engineering.
Published under an exclusive license by AIP Publishing. https://doi.org/10.1063/5.0086649

The recently developed physics-informed neural networks equations (ODEs) starting from the 1990s.1–3 In 1998, Lagaris et al.4
(PINNs) is attracting much attention among the numerical meth- extended this idea to a new numerical method based on feedforward
ods for solving differential equations. Despite its many advan- neural networks (FNNs) for the solution of non-linear ODEs and
tages, its failure in solving stiff systems of ordinary differential partial differential equations (PDEs). The method aims to train the
equations has been demonstrated. This study shows that merg- FNNs to minimize the error between the FNN prediction and the
ing PINN with the theory of functional connections allows one right-hand side of the differential equations. Subsequently, Filici5
to obtain highly accurate solutions for stiff non-linear chemical proposed a single-layer multiple linear output perceptron, in which
kinetics problems. The initial conditions are always analytically the training of the NN was performed with a Levenberg–Marquardt
satisfied via particular expressions called constrained expres- algorithm (LMDER optimization solver) from MINPACK,6 which
sions, and the boundary-free solution is expanded with a single- was tested by solving simple non-stiff ODE problems. Gerst-
layer forward neural network, trained via the extreme learning berger and Rentrop7 developed one of the first frameworks to
machine algorithm. The proposed method is tested by solving solving stiff ODEs and differential-algebraic equations involving
classic, challenging chemical kinetics problems from the liter- NNs.
ature, and it can be applied to many chemical, biological, and Recently, a new class of methods to solve differential equa-
nuclear systems subject to stiffness. tions has been developed in the machine learning community, and
it is called physics-informed neural networks (PINNs). The meth-
ods underlying this category can be generalized as algorithms that
I. INTRODUCTION include the physics of the problem into a data-driven representation
Artificial neural networks (NNs) have been widely used in of functional relationships underlying collections of input–output
the recent past to solve linear and non-linear ordinary differential pairs.8–13

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 127 32, 063107-1

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

Large-scale stiff systems of ODEs are mathematical models As opposed to traditional methods for solving DEs such as finite ele-
governing a pretty wide range of real-world problems such as ment method31 and finite difference method, the PINN framework
chemistry, biology, epidemiology, engineering, neuroscience, finan- constructs a loss function (cost function) informed by the physics of
cial systems, and so on. For this reason, recently, there has been the problem. Then, the optimal set of weights and biases parameters
a resurgence in the interest in developing new numerical methods in the NN are sought to minimize the error. The NN parameters
capable of solving large time-scale stiff problems.14–17 Two recent weights and biases are learned by minimizing the mean squared
works aim to solve stiff chemical kinetics problems using PINN error (MSE) loss with gradient-based methods, such as stochastic
and neural ODEs.18,19 In particular, Ref. 18 highlights the failure of gradient descent (e.g., Adam optimizer).
regular PINNs in solving stiff problems and proposes a quasi-steady- One of the significant drawbacks of the standard PINN frame-
state assumption (QSSA) to reduce the stiffness of the systems works is that the initial and boundary conditions (i.e., the equation
of ODEs. This artifact converts the problems into non-/mild-stiff constraints) are not analytically satisfied. Hence, during the PINN
problems, where the PINN framework can be applied. The pro- training, there are competing objectives: learning the DE solu-
posed algorithm has been named Stiff-PINN. In Ref. 19, the authors tion within the domain and satisfying the equation constraints.
employ neural ordinary differential equations to solve stiff systems This leads to unbalanced gradients during the network training via
of ODEs. In both papers, two classical stiff chemical kinetics prob- gradient-based methods that causes PINNs to have often difficulties
lems are used for testing the performance of their methods: the in properly approximating the solution of DEs.32 Indeed, it is well
ROBER problem20 and the POLLU problem.21 The results presented known that gradient-based methods may get stuck in limit cycles
in both papers are not numerically compared with any other method or even diverge if multiple competing objectives are present.33,34 In
and any benchmark in the literature, and no computational times Ref. 32, to overcome this issue, the authors developed a learning rate
required for the training are given. Although these two methods annealing algorithm that employs gradient statistics to assign appro-
clean up the regular PINNs failure, they are far for being consid- priate weights to different terms adaptively (e.g., DE residuals within
ered accurate and fast methods for solving stiff problems since the the domain and DE residuals on the boundaries) in the PINNs loss
results reported in the manuscripts are limited to reproduction of function during the network training. The approach used here is dif-
the species profiles only. ferent and is based on the TFC35 which allows to analytically satisfy
Wang et al.13 have recently demonstrated that PINNs are not the equation constraints, hence removing the competing objective
accurate (or even not capable) to solve problems when the dynamics in the NN training.
are particularly complex (e.g., stiff). This is due to unbalanced back- In this work, we aim to learn the solutions of initial value prob-
propagated gradients during the model training. lems (IVPs) governed by the chemical kinetics equations using a
We propose a numerical scheme based on PINN and The- recently developed PINN framework named X-TFC. The solutions
ory of Functional Connections (TFC), named Extreme Theory of are approximated with the so-called constrained expressions (CEs),
Functional Connections (X-TFC),22 which has already proved its defined in the TFC.35 The structure of the X-TFC algorithm for solv-
successful performance in many applications.23–28 We show how ing first-order ODEs is presented in Sec. II A and the estimate of the
X-TFC, by using extreme learning machine (ELM) and the TFC con- generalization error is given in Sec. II B.
strained expressions (CEs), overcomes the difficulties of PINNs in
solving stiff systems of ODEs, and it is comparable with the state-of- A. X-TFC to solving first-order ODEs
the-art numerical methods in terms of accuracy and computational
A general non-linear first-order IVP can be written as follows:
times. In particular, we will focus on the solution of four classical 
stiff chemical kinetics problems: the ROBER problem,20 the Chem-  dy
= f(y, t), ∀t ∈ [0, T],
ical Akzo Nobel problem,29 the Belousov–Zhabotinsky reaction,30 dt (1)
and the POLLU problem.21 y(0) = y .
0
The manuscript is organized as follows. First, in Sec. II, the gen-
eral PINN methodology is presented, explaining how we efficiently According to the TFC method,35 the unknown function can be
merge it with TFC and extreme learning machine (ELM) to obtain expanded via the CE, as follows:
highly accurate solutions. Also, a rigorous upper bound on the gen- y(t) ' yCE (t) = g(t) + ηp(t). (2)
eralization error of X-TFC frameworks in learning the solutions
The first term g(t) is a freely chosen function, which must exist
of initial value problems (IVPs) for ODEs is provided. In Sec. III,
on the constraint. In the second term, we have p(t), that is an
the general form of the chemical kinetics equations and the X-TFC
user-defined function which must be independent from g(t) and an
algorithm with time decomposition are presented. In Sec. IV, the
unknown constant parameter η. With this form of the CE, the con-
accuracy of the proposed PINN-based method is tested against sev-
straint of the problem is always analytically satisfied, regardless of
eral methods and benchmarks from the literature for four classical
the choice of g(t). For IVPs, the best selection of independent func-
stiff chemical kinetics problems. Conclusions are drawn in Sec. V.
tion is p(t) = 1. The reasons for this choice are explained in detail in
Refs. 36 and 37. Thus, the CE becomes
II. X-TFC TO SOLVING NON-LINEAR INITIAL VALUE
yCE (t) = g(t) + η. (3)
PROBLEMS
By embedding the constraint into the CE, we have
PINNs is a recently developed algorithm by Raissi et al.8 for
solving forward and inverse problems governed by ODEs and PDEs. yCE (0) = y0 = g0 + η =⇒ η = y0 − g0 , (4)

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 128 32, 063107-2

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

which substituted into Eq. (3) gives us the x domain is required. In general, to achieve better training accu-
racy, x ∈ [0, 1] or x ∈ [−1, 1] domains are employed, depending on
yCE (t) = g(t) + (y0 − g0 ) (5)
the codomain of the chosen activation function. The mapping linear
and its derivative transformation used is
dyCE dg 1
= . (6) x = x0 + c(t − t0 ) ←→ t = t0 + (x − x0 ),
dt dt c
By plugging Eqs. (5) and (6) into the general problem (1), we obtain where the mapping coefficient c is
an unconstrained ODE
xf − x0
dyCE c= .
= f(yCE , t), ∀t ∈ [0, T]. (7) tf − t0
dt
The nth derivative of Eq. (9) is given by
Thus, the new ODE depends only on t and g(t),
dn g(t) dn σ (x)
dg = cn β T , (10)
= f(g(t) + y0 − g0 , t), ∀t ∈ [0, T]. (8) dt n dxn
dt
according to the derivative chain rule.
X-TFC employs a shallow NN as the free function g(t), which is
A discretization in N quadrature (training) points of the x
represented as
domain is performed, and then the following loss function is mini-
 1 T mized via unconstrained optimization techniques:
L
σ
X
 . 
g(t) = βj σ wj t + bj =  ..  , β = σ T β, (9) N
X dσi T β
2


j=1 L
L(β) = wi c − f σi T β + y0 − σ T0 β, xi . (11)
σ i=1
dx
where j = 1, . . . , L, and L is the number of neurons, wj , βj , bj ∈ R are Here, wi is the ith quadrature weight corresponding to the ith
the input weight, output weight, and bias, respectively, correspond- quadrature point, σi and σ0 are the activation functions computed
ing to the jth neuron. The activation function σ j (·) is selected by the in the ith training point and the first training point (initial time
user. instant), respectively. To note that, throughout this paper, even
The NN parameters in Eq. (9) are trained via the ELM though σ0 actually coincides with σ1 , the authors prefer to maintain
algorithm, whose convergence proofs can be found in Ref. 38. the notation σ0 , because it refers to the traditional subscript of the
The known parameters are the input weights and biases, randomly initial time instant (e.g., t0 ).
selected. Thus, the only unknown parameters learned by X-TFC to When the DEs under study are non-linear, an iterative least-
solve the IVP are the output weights β = [β1 , . . . , βL ]T . Figure (1) squares approach is employed37 to minimize Eq. (11). By following
shows a graphical representation of a general single-layer NN used the iterative procedure, at each kth iteration corresponds an update
in this work. of the output weights. That is,
The domain of some activation functions (x ∈ [x0 , xf ]) and the
domain of the problem under study (t ∈ [t0 , tf ]) may not be con- β k+1 = β k + 1β k , (12)
sistent. Hence, a mapping of the independent variable t ∈ [t0 , tf ] in k
where 1β is evaluated as
 −1
1β k = − JT (β k )J(β k ) JT (β k )L(β k ). (13)

J is the Jacobian matrix of the loss functions. By imposing a tol-

erance  (user-defined), the iterative procedure continues until the
following conditions is reached:
L2 [L(β k )] < . (14)

B. Estimate on the generalization error

In this subsection, by following the work by Mishra and
Molinaro,39 we derive an estimate of the generalization error when
using X-TFC for learning the solution of a general non-linear IVP.
We consider a single non-linear ODE, but the results can be
extended to systems of non-linear ODEs, without loss of generality,

 dy
= f(y, t), ∀t ∈ [0, T],
dt (15)
y(0) = y .
0
FIG. 1. Example of a shallow NN, trained via ELM, used as the free-function in
the X-TFC method.
Here, y0 ∈ R are the initial data, y ∈ [0, T] ⊂ R is the solution of
(15), and f : R is the non-linear forcing term. We assume that f is

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 129 32, 063107-3

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

globally Lipschitz i.e., there exists a constant Cf ≥ 0 (independent of terms of the training error defined as
t) such that N
X
|f(tv ) − f(tw )| ≤ Cf |tv − tw |, tv , tw ∈ [0, T]. (16) εT2 = wi |Rβ ∗ (ti )|2 . (22)
i=1

In order to apply the X-TFC algorithm to learn the solution of the Note that the training error can be readily computed a posteriori
IVP, we need to specify the training set S and define the appropriate from the loss function (20).
residual. According to Ref. 39, we also need the following assumptions
on the quadrature error: for any function u ∈ Cl ([0, T]), the quadra-
ture rule corresponding to quadrature weights wi at points ti ∈ S ,
1. Training set
with 1 ≤ i ≤ N, satisfies
Consider the time domain [0, T] ∈ R. We will choose the train- Z T N
ing set S ⊂ [0, T], based on suitable quadrature (training) points. X
u(t)dt − wi u(ti ) ≤ Cquad (||u||Cl )N-α . (23)
Since the IVP constraints are analytically satisfied thanks to the CE, 0 i=1
our training points are
Here, α > 0 and different order quadrature rules can be used.
S = {ti } for 1 ≤ i ≤ N, (17) The estimate of the generalization error for X-TFC is given in the
following theorem.
with each ti ∈ [0, T]. According to Ref. 39, these points can be
the quadrature points, corresponding to a suitable time grid-based Theorem 1. Let y ∈ Ck ([0, T]) be the unique classical solution of the
composite Gauss quadrature rule or randomly selected points. non-linear problem (15) with the dynamics f satisfying Eq. (16). Let
y∗ = yβ ∗ be the solution approximated via X-TFC, corresponding to
loss function (20). Then, the generalization error (21) can be estimated
2. Residuals as

1
In order to apply the X-TFC algorithm, we need to define εG ≤ C1 εT2 + Cquad N-α 2 , (24)
appropriate residuals. We approximate the IVP solution with the
following constrained expression: where the constant C1 is given by

1   21
yβ (t) = gβ (t) + y0 − gβ (0), (18) C1 = e(1+2Cf )T − 1
1 + 2Cf
where gβ (t) is a single-layer NN with random features. Then, we
define the following residuals: and
Cquad = Cquad (||R2 ||Ck−1 ),
d d

Rβ (t) = yβ (t) − f(yβ (t), t) = gβ (t) − f gβ (t) + y0 − gβ (0), t . which is a positive constant depending on the number of training
dt dt
(19) points, the quadrature scheme used, and the residuals evaluated on
the training points.39

3. Loss function Proof. By the definitions of the residuals (19) and the under-
lying ODE (15), we can verify that the error ŷ = y∗ − y satisfies the
To learn the unknown solution, we need to minimize the following equation:
following loss function:
dŷ
N = f(y∗ , t) − f(y, t) + R, ∀t ∈ [0, T],
X dt (25)
L(β) = wi |Rβ (ti )|2 . (20)
i=1 ŷ(0) = 0.

The X-TFC algorithm is now completely specified and it can be run Here, we have denoted R = Rβ ∗ for convenience of notation.
for learning the solution of the non-linear IVP of Eq. (15), which Multiplying both sides of the ODE (25) by ŷ yields
we denote as y∗ = yβ ∗ , where β ∗ are the computed optimal output dŷ  
weights. ŷ = ŷ f(y∗ , t) − f(y, t) + ŷR, (26)
dt
We are interested in estimating the generalization error for X-
TFC, which is defined as that is,
1 d 2
Z T 1/2 |ŷ| = ŷ[f(y∗ , t) − f(y, t)] + ŷR. (27)
2 dt
εG = |y(t) − y∗ (t)|2 dt . (21)
2
0 By leveraging on the following inequality ŷ − R ≥ 0, Eq. (27) is
εG , according to Ref. 39 and references within, is the error emerging bounded by
from the approximation of the true solution, y(t), via X-TFC, within 1 d 2 1 1
in the domain 0 ≤ t ≤ T. We estimate the generalization error in |ŷ| ≤ |ŷ| f(y∗ , t) − f(y, t) + |R|2 + |ŷ|2 . (28)
2 dt 2 2

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 130 32, 063107-4

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

By (16), we have the following: III. CHEMICAL KINETICS EQUATIONS AND ADAPTIVE
SCHEME
1 d 2 1 1
|ŷ| ≤ |ŷ|Cf |y∗ − y| + |R|2 + |ŷ|2 . (29) The stiff chemical kinetics systems of equations40 we aim to
2 dt | {z } 2 2
|ŷ| solve are IVPs of ODEs, having the following form:

Rearranging, we get  dyi (t)
= fi (t, y1 , y2 , . . . , ym ),
dt (38)
d 2
y (0) = y ,
|ŷ| ≤ 1 + 2Cf |ŷ|2 + |R|2 . (30) i i0
dt
with i = 1, 2, . . . , m, where the unknowns are the functions yi (t),
Integrating the above inequality over [0, T̄] for any T̄ ≤ T, we obtain representing the time-dependent chemical species concentrations.
Z

T̄ 2
Z T The non-linear functions fi and the initial conditions yi 0 are known.
|ŷ(T̄)|2 ≤ 1 + 2Cf |ŷ| dt + |R|2 dt. (31) The computational challenge for numerical methods with this kind
0 0 of problem lies in the complex behavior and existence of sharp gra-
Applying the integral form of Grönwall’s inequality to the above, we dients in the unknown solution. This particular behavior is the so-
get called stiffness of a problem whose complete and precise definition
 Z T has not yet been formulated. A problem can be considered stiff when
|ŷ(T̄)|2 ≤ e(1+2Cf )T̄ |R|2 dt. (32) a solution varies very slowly, but other nearby solutions can present
0 steep changes. According to Lambert,41 if a numerical method used
Integrating over [0, T], to solve IVPs of ODEs is forced to decompose the time domain into
 multiple excessively small subintervals in relation to the smoothness
Z T
1   Z T
εG2 = |ŷ(T̄)|2 dT̄ ≤ e(1+2Cf )T − 1 |R|2 dt. of the exact solution in that domain, then that system of ODEs is
0 1 + 2Cf 0 stiff in that domain. Or even, according to Ernst Hairer,42 a prob-
(33) lem is considered being stiff if explicit methods are not capable of
solving it.
That is, For the convenience of the reader, the schematic of Fig. 2
 
1  summarizes the all X-TFC process used in this work.
εG2 ≤ e(1+2Cf )T − 1 Since we are facing long-time domain problems, a decompo-
1 + 2Cf
! sition of the temporal domain into subintervals was performed to
N
X
2 2 -α
minimize the carryover of the error during the chemical reactions
wi |R(ti )| + Cquad (||R ||Ck−1 )N . (34) between the species. The temporal domain (whose decomposition
i=1 is pictured in Fig. 3) is divided into n time steps of equal size
Thus, h = tk − tk−1 for k = 1, . . . , n, each of which is discretized into nx
  points.
1 

Thus, this decomposition into time steps leads to the solution
εG2 ≤ e(1+2Cf )T − 1 εT2 + Cquad (||R2 ||Ck−1 )N-α .
1 + 2Cf of several consecutive IVPs
(35) 
(k)
 dyi (t)

Finally, = fi (t, y(k) (k) (k)
1 , y2 , . . . , ym ),
dt (39)

y(k) (0) = y (k) ,

1   12
1 i i 0
εG ≤ e(1+2Cf )T − 1 εT2 + Cquad N-α 2 , (36)
1 + 2Cf with i = 1, 2, . . . , m. Initially, we solve the system of ODEs in the
time step 1 by using the initial conditions yi (t) = yi 0 given by the
where problem, thus finding the unknown solutions at time t1 , which

1   21 become the new initial conditions for the next step, and so on,
e(1+2Cf )T − 1 = C1 . (37)
1 + 2Cf yi (k) (k−1)
0 = yi f (tk ). (40)
This completes the proof.  Thus, according to Eq. (5), the CE for the kth time step is
The estimate (24) bounds the generalization error in terms of  
training and quadrature errors. The training error is computed once y(k) (k−1)
i (tk ) = g(tk ) + yi f (tk ) − g(k)
0 . (41)
the training is completed. As long as X-TFC is well trained, i.e., the
training error is small, the bound (24) implies that the generalization
error will be small for a large enough number of training points. The IV. NUMERICAL RESULTS AND DISCUSSIONS
estimate is not sharp as triangle inequalities, and Grönwall’s inequal- In this section, four chemical kinetics benchmark problems are
ity is used. If random training points are used, the estimate needs to analyzed, and numerically solved, and the solutions are compared
be slightly modified for the generalization error (24). Since random with state-of-art methods’ results. All the problems tackled in this
training points are not used in this work, we suggest the interested manuscript have been coded in Matlab R2021a and run with an Intel
reader to see these details in Ref. 39. Core i7-9700 CPU PC with 64 GB of RAM.

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 131 32, 063107-5

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

FIG. 2. Schematic of the X-TFC algorithm to solve the chemical kinetics equations.

Four benchmark problems in chemical kinetics are used for probability distribution used to sample the weights and bias of the
our simulations: Robertson’s chemical reaction model, as known inputs. However, it is to the number of neurons and training points.
as the ROBER problem,20 the Chemical Akzo Nobel problem,43
the Belousov–Zhabotinsky reaction,44 an air pollution problem, as
A. ROBER problem
known as POLLU problem.21
In the X-TFC framework, several hyperparameters can be The first test case is given by Robertson’s system of equations,20
adjusted to obtain accurate solutions, such as the number of train- denoted as ROBER problem by Wanner and Hairer.42 This model
ing points, nx , the number of neurons, L, the type of activation has been developed to describe the chemical kinetics of the following
function, and the probability distribution where input weights and autocatalytic reaction:
bias are sampled. For all the problems, we have used a hyperbolic k1
tangent (tanh) as a activation function, and the weights and biases A−
→ B,
are randomly sampled from U [−1, 1]. An analysis to study the X- k2
TFC’s sensitivity to these hyperparameters has been performed in B+B−
→ C + B,
our previous work,22 which showed that the accuracy of the solution k3
is not as sensitive to the type of activation function and/or to the B+C−
→ A + C,

FIG. 3. Decomposition of the time domain into subintervals of length h.

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 132 32, 063107-6

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

where k1 , k2 , and k3 are the reaction rate constants, and A, B,

and C are the chemical species, whose evolution of concentrations
(y1 , y2 , y3 ) is governed by the following system of three non-linear
ODEs:
 
0
y1 = −k1 y1 + k3 y2 y3 ,
 y1 (0) = 1,

y02 = k1 y1 − k2 y22 − k3 y2 y3 , s.t. y2 (0) = 0, (42)

y0 = k y2 , 
y (0) = 0.
3 2 2 3

The values of the reaction rate constants are k1 = 0.04, k2 = 3 × 107 ,

and k3 = 104 , thus varying in a range of 9 orders. This large variation
of the parameters is the cause of the strong stiffness of the problem.
The constrained expressions and their derivatives are
y1 (t) = (σ − σ 0 )β 1 + y1 (0), y01 (t) = cσ 0 β 1 ,
y2 (t) = (σ − σ 0 )β 2 + y2 (0), y02 (t) = cσ 0 β 2 ,
y3 (t) = (σ − σ 0 )β 3 + y3 (0), y03 (t) = cσ 0 β 3 .
As we are facing a non-linear problem, we need to compute the
losses and their derivatives
L1 = y01 + k1 y1 − k3 y2 y3 ,
L2 = y02 − k1 y1 + k2 y22 + k3 y2 y3 ,
L3 = y03 − k2 y22 ,
∂ L1
= cσ 0 + k1 (σ − σ 0 ),
∂β 1 FIG. 4. Species concentration of the ROBER problem computed with MATLAB
ode15s and X-TFC (top) and absolute values of the errors between the two
∂ L1
= −k3 y3 (σ − σ 0 ), methods (bottom).
∂β 2
∂ L1
= −k3 y2 (σ − σ 0 ),
∂β 3
In Table I, the solutions over t = [0, 4000] are reported. The
∂ L2 results for t = 0.4, 40, 4000 are obtained with L = 11, 9, 10, and
= −k1 (σ − σ 0 ),
∂β 1 nx = 11, 9, 10, respectively, fixed step size h = 0.001, and tolerance
∂ L2 of 10−12 . The training points are uniformly spaced in a linear time
= cσ 0 + (2k2 y2 + k3 y3 )(σ − σ 0 ), scale. The computational times for t = 0.4, 40, 4000 are ≈ 0.06 , 6.7,
∂β 2
and 580 s, respectively. The average training errors for the three time
∂ L2 instants are 7.3 × 10−16 , 7.1 × 10−17 , and 3.2 × 10−13 , respectively.
= k3 y2 (σ − σ 0 ,)
∂β 3 In this table, we can appreciate a good compatibility with the
∂ L3 hybrid method presented in Ref. 45, the MEBDF code,46 and LSODE
= 0, method presented by Shampine,47 up to the 11th significant digit.
∂β 1 In Fig. 4, we reported X-TFC solutions compared with those
∂ L2 obtained by the MATLAB stiff ode solver ode15s, which we
= −2k2 y2 (σ − σ 0 ),
∂β 2 consider here the true solution for our qualitative and quantitative
comparison. The time span for producing this plot is t ∈ [10−5 , 105 ],
∂ L3
= cσ 0 . and the training points are uniformly spaced in a logarithmic time
∂β 3 scale. The step size used is h = 2000, which allows us to obtain these
The linear system that we want to iteratively solve is J1β = L, that results in a computational time of about 0.015 s, which is particularly
is, low, contrary to what is reported in Ref. 19. The average train-
 ∂L ∂ L1 ∂ L1   ing error is 8.5 × 10−12 . In the top three subplots, we can see how
1   
our solutions overlap with those computed by ode15s, with abso-
 ∂β 1 ∂β 2 ∂β 3  1β 1 L1
     lute error in the order of 10−15 /10−05 (bottom subplots). This proves
 ∂ L2 ∂ L2 ∂ L2     
  1β  = L2  . that the X-TFC method is much more reliable in solving the ROBER
 ∂β   2  
 1 ∂β 2 ∂β 3    problem than the regular PINN, which fails in this task, even if
 ∂ L3 ∂ L3  L3 artifacts to reduce the stiffness of the ODE system are employed, as
0 1β 3
∂β 2 ∂β 3 as for Stiff-PINNs.18

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 133 32, 063107-7

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

B. Chemical Akzo Nobel problem and the values of the parameters are
The second taste case is named chemical Akzo Nobel k1 = 18.7, k2 = 0.58, k3 = 0.09,
problem, and it describes a chemical process which originates from
the Akzo Nobel Centeral Research in Arnhem, The Netherlands. In k4 = 0.42, K = 34.4.
the reactions, the two species MBT and CHA are initially mixed, Fin denotes the inflow of oxygen per volume unit and satisfies the
while oxygen is continuously added. The resulting species of impor- following:
tance is CBS, and secondary products of reactions are BT, H2 O, and  
sulfate. According to Ref. 29, the chemical reaction scheme is as p(O2 )
Fin = klA − y2 , (44)
follows: H
1 where klA = 3.3 is the mass transfer coefficient, H = 737 is the
2 MBT + O2 MBTS + H2 O,
2 Henry constant, and p(O2 ) = 0.9 is the partial oxygen pressure.
The chemical process begins by mixing 0.437 mol/l of MBT and
0.367 mol/l of MBT.CHA, with a concentration of oxygen of 0.001 23
CBS + MBT MBTS + CHA mol/l. The other species are created afterwards, and the process sim-
ulation is performed in the range of [0, 180] minutes. Again, we need
MBT + 2 CHA + O2 BT + sulfate, to compute the losses and their derivatives with the same procedure
reported in the previous problem. For this and the following prob-
1
MBT.CHA + O2 CBS + H2 O, lems, we report only the linear system J1β = L that we want to
2 solve iteratively,
MBT + CHA MBT.CHA. 
∂ L1 ∂ L1 ∂ L1 ∂ L1 ∂ L1

 ∂β 0
 1 ∂β 2 ∂β 3 ∂β 4 ∂β 5
   
The last equation describes an equilibrium, where the value Ks  ∂L
 1β 1 L1
 2 ∂ L2 ∂ L2 ∂ L2  

[MBT · CHA]   
= is used in parameter estimation. The other  0 0    
[MBT] · [CHA]  ∂β 1 ∂β 2 ∂β 4 ∂β 6  1β 2  L2 
    
balance equations describe chemical reactions. The mathematical  ∂ L3 ∂ L3 ∂ L3 ∂ L3 ∂ L3    
model describing this set of reactions is a stiff of 6 non-linear ODEs,

 ∂β 0 
 1β 3
  
 L3 
 1 ∂β 2 ∂β 3 ∂β 4 ∂β 5    
   =  .

0
  ∂ L4 ∂ L4 ∂ L4 ∂ L4  1β  L 
y1 = −2r1 + r2 − r3 − r4 ,
 y1 (0) = 0.437,
 
 ∂β 1 0 0   4  4

 0 1 1

 ∂β 3 ∂β 4 ∂β 5    

 y 2 = − r
2 1
− r 4 − r
2 5
+ Fin , 
 y2 (0) = 0.001 23, 
 ∂L
   
 5 ∂ L5 ∂ L5 ∂ L5 ∂ L5 ∂ L5  1β 5  L5 

 
     
y0 = r − r + r , y (0) = 0,
3 1 2 3 3     
0
s.t. (43)  ∂β 1 ∂β 2 ∂β 3 ∂β 4 ∂β 5 ∂β 6 
y4 = −r2 + r3 − 2r4 ,
 y4 (0) = 0,
   1β L6

 
  ∂ L6 ∂ L6  6


 y05 = r2 − r3 + r5 , 

 y5 (0) = 0, 0 0 0 0

y0 = −r 
y (0) = 0.367, ∂β 2 ∂β 6
6 5 6
In Table II, the solutions of the concentration at the final time
for t ∈ [0, 180], where the reaction rates are given by t = 180 are reported, obtained with L = 20, nx = 50, fixed step
size h = 0.01, and tolerance of 10−12 . The training points are uni-
1 k2 formly spaced in a linear time scale. The computational times for
r1 = k1 y41 y22 , r2 = k2 y3 y4 , r3 = y1 y5 ,
K t = [0, 180] is ≈ 64 s with an average training error of 2.1 × 10−16 .
1 In this Table, we compare our results with the Test set for IVP solvers
r4 = k3 y1 y24 , r5 = k4 y26 y22 , benchmarks presented in Ref. 48, and the results obtained using the

TABLE I. Species concentrations at times t = 0.4, 40, 4000 of X-TFC, Ref. 45, MEBDF, and SDMM, for the ROBER problem.

t yi X-TFC Hybrid method MEBDF LSODE

0.4 y1 9.851 721 138 609 84E−01 9.851 721 138 632 31E−01 9.851 721 138 61E−01 9.851 7E−01
y2 3.386 395 378 974 81E−05 3.386 395 378 909 47E−05 3.386 397 378 97E−05 3.386 4E−05
y3 1.479 402 218 522 60E−02 1.479 402 218 548 82E−02 1.479 402 218 54E−02 1.479 4E−02
40 y1 7.158 270 687 081 93E−01 7.158 270 687 189 94E−01 7.158 270 687 82E−01 7.158 2E−01
y2 9.185 534 764 122 04E−06 9.185 534 764 567 52E−06 9.185 534 765 64E−06 9.185 1E−06
y3 2.84163745757042E−01 2.841 637 457 463 61E−01 2.841 637 457 33E−01 2.841 7E−01
4000 y1 1.832 022 577 274 15E−01 1.832 020 418 731 52E−01 1.832 022 818 48E−01 1.832 0E−01
y2 8.942 371 250 213 86E−07 8.942 358 400 025 91E−07 9.942 372 681 15E−07 8.942 3E−07
y3 8.167 968 480 012 25E−01 8.167 970 638 913 67E−01 8.167 941 451 52E−01 8.168 0E−01

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 134 32, 063107-8

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

TABLE II. Species concentrations at time t = 180 of X-TFC, test set IVP solvers,48 and exponential propagation method,49 for the chemical Akzo Nobel problem.

t yi X-TFC Test set IVP solvers Exp propagation

180 y1 1.161 602 274 780 192E−01 1.161 602 274 780 192E−01 1.161 583 704 781 95E−01
y2 1.119 418 166 040 844E−03 1.119 418 166 040 848E−03 0.011 194 091 556 33E−01
y3 1.621 261 719 785 830E−01 1.621 261 719 785 814E−01 1.621 265 677 283 22E−01
y4 3.396 981 299 297505E−03 3.396 981 299 29 7459E−03 0.033 959 155 455 71E−01
y5 1.646 185 108 335 093E−01 1.646 185 108 335 055E−01 1.646 185 137 343 56E−01
y6 1.989 533 275 954337E−01 1.989 533 275 954 281E−01 1.989 543 899 415 37E−01

exponential propagation method from Ref. 49. As one can see, our order of 10−15 /10−05 (error subplots). This shows the accuracy of
results match from 13 to 16 digits of accuracy with the benchmarks X-TFC for larger time steps (lower computational times).
of Ref. 48, while with the exponential propagation method there is
a compatibility up to the 8th digit (different figures are marked in
bold).
In Fig. 5, we reported X-TFC solutions compared with those
obtained by the MATLAB stiff ode solver ode15s, which we consider C. Belousov–Zhabotinsky reaction
here the true solution for our qualitative and quantitative compar- The third test case we consider is the Belousov–Zhabotinsky
ison. In the concentration subplots, we can see how our solutions reaction mechanism,30 which can be represented by the follow-
overlap with those computed by ode15s, with absolute error in the ing scheme of homogeneous chemical reactions in a symbolic

FIG. 5. Species concentration of the chemical Akzo Nobel problem computed with MATLAB ode15s and X-TFC, and absolute values of the errors between the two methods.

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 135 32, 063107-9

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

FIG. 6. Species concentrations (Y, P, Z, Q) of the

Belousov–Zhabotinsky reaction computed with X-TFC for
integration time of 40 s (a) and 120 s (b). A simulation for integra-
tion time of 5000 s shows the robustness of the X-TFC method for
very large time domains (c).

form: following linear system:

 ∂L ∂ L1 
k1 1
A+Y −
→ X, 0 0 0 0 0
 ∂β 1 ∂β 2  1β  L 
  1 1
k2  ∂ L2 ∂ L2 ∂ L2
X+Y −
→ P, ∂ L2    

 ∂β 0 0 0  1β  L 
  
k3  1 ∂β 2 ∂β 3 ∂β 6   2  2 
B+X −
→ 2X + Z, (45)  3 ∂ L3 ∂ L3
 ∂L ∂ L3 
   
 0 0 0    
1β 3  L3 
k4  ∂β 1 ∂β 2 ∂β 3 ∂β 5 
2X −
→ Q,     
 ∂ L4 ∂ L4 ∂ L4 
  
  
k5
 0 0 0 0  1β 4  = L4 .

Z −
→ Y,

 ∂β 2 ∂β 3 ∂β 4    
   
 ∂ L5 ∂ L5 
 L5 
1β 5 
 0   
 0 0 0 0 
9  ∂β 3 ∂β 5    
where the reaction rate constants are k1 =4.72, k2 = 3 × 10 ,     
k3 = 1.5 × 104 , k4 = 4 × 107 , and k5 = 1 moll s . The letters denote  ∂ L6 ∂ L6 ∂ L6 1β 6  L6 
  
 0 0 0 0 
the species taking part of the reactions, and the initial concentrations  ∂β 3 ∂β 5 ∂β 6    
 
at time t = 0 (expressed in mol ) are  ∂ L7 ∂ L7  1β 7 L7
l 0 0 0 0 0
∂β 3 ∂β 7
Y = X = P = Q = 0, A = B = 0.066, Z = 0.002. (47)

The system of ODE modeling the Belousov–Zhabotinsky reaction is Figure 6 shows the simulation results obtained using X-TFC
with L = 20, nx = 20, fixed step size h = 0.01, and tolerance of
 10−9 , for three different integration times. The training points are
 y01 = −k1 y1 y2 ,

 uniformly spaced in a linear time scale, and the computational
y02

 = −k1 y1 y2 − k2 y3 y2 + k5 y6 ,

 times for t = [0, 40] is ≈ 3 s. One can clearly see the rapid “jumps”
0
y3

 = −k2 y3 y2 + k3 y3 y5 − 2k4 y23 + k1 y1 y2 , in the solutions with a period of approximately 16 s. The quali-
y04 = k2 y3 y2 , (46) tative analogy with the converged solutions presented by Gear’s



y05 = −k3 y5 y3 , method50 and the Almost Runge–Kutta and Aluffi–Pentini methods

 (obtained with CPU times of ≈ 9 s)51 is evident. As one can see from

y06 = k3 y5 y3 − k5 y6 ,


 0 Fig. 6(c), we can extend the integration time to a very large range
y7 = k4 y23 , (e.g., 5000 s) while continuing to obtain accurate solutions without
negative values.
subject to yi (0) = (0.066, 0, 0, 0, 0.066, 0.002, 0)T , for t ∈ [0, 40]. Two MATLAB functions, ode15s and ode15i, have been imple-
After computing the losses and their derivatives, we obtain the mented to compare their results with the X-TFC framework. The

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 136 32, 063107-10

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

FIG. 7. Species concentrations of the

Belousov–Zhabotinsky reaction computed
with X-TFC, ode15i, and ode15s, for inte-
gration time of 40 s.

solutions of the three methods for all the seven species reacting in D. POLLU problem
the chemical process are reported in Figs. 7 and 8, for the integra-
tion times of 40 and 120 s, respectively. Both solutions of ode15s and The fourth and last test case we use in this work to
ode15i have been obtained with time step h = 0.0001 and tolerance test the robustness and efficiency of our method is a real-
of 10−9 . We can see that ode15s is able to find the solution only for istic air pollution model52,53 from atmospheric chemistry with
the first jump, after which it fails, keeping the species concentrations 20 components and 25 reactions. This air pollution model,
constant in time. From Fig. 7, we can note that ode15i accurately also known as POLLU problem, has been developed at The
captures the solutions for the first jump, but with a delay of about 1 s Dutch National Institute of Public Health and Environmen-
for the second jump and about 3 s for the third jump. By extending tal Protection (RIVM) and described by Verwer.21 The reac-
the time interval (see Fig. 8), we see that after the third jump, it stop tion scheme, reaction rates, and rate constants are given in
working, keeping the species concentrations constant in time. Table III.

FIG. 8. Species concentrations of

the Belousov–Zhabotinsky reaction
computed with X-TFC, ode15i, and
ode15s, for integration time of 120 s.

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 137 32, 063107-11

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

The mathematical modeling of the POLLU problem is given by the following system of 20 ODEs with initial conditions:
 0 
 y = −r1 − r10 − r14 − r23 − r24 + r2 + r3 + r9 + r11 + r12 + r22 + r25 ,  y1 (0) = 0,
 01
 




 y2 = −r 2 − r 3 − r 9 − r 12 + r 1 + r 21 , 


 y2 (0) = 0.2,
 0 

y = −r 15 + r 1 + r 17 + r 19 + r 22 , 
 y3 (0) = 0,
 03







y4 = −r2 − r16 − r17 − r23 + r15 , 

y4 (0) = 0.04,
 0




y 5 = −r 3 + 2r4 + r6 + r7 + r13 + r20 ,
 5 (0) = 0,


y
 
y06 = −r6 − r8 − r14 − r20 + r3 + 2r18 ,

 y6 (0) = 0,



 0




y 7 = −r 4 − r5 − r 6 + r 13 , y7 (0) = 0.1,


 0 
y8 = r4 + r5 + r6 + r7 ,

 y8 (0) = 0.3,



 

0


y 9 = −r 7 − r8 , y9 (0) = 0.01,



y0 = −r + r + r , 
y (0) = 0,
10 12 7 9 10
0
s.t. (48)
y
 11 = −r 9 − r 10 + r 8 + r 11 ,  11 (0) = 0,
y

 

0

y
 12 = r 9 , y12 (0) = 0,



 0


y13 = −r11 + r10 ,
 y13 (0) = 0,


 

0

y
 14 = −r 13 + r 12 , y14 (0) = 0,



 0




y 15 = r 14 , 

y15 (0) = 0,

 0




y 16 = −r 18 − r 19 + r 16 , 

y16 (0) = 0,
 0 


y 17 = −r 20 , 

y17 (0) = 0.007,

 0


y18 = r20 ,

 y18 (0) = 0,


 0




y 19 = −r 21 − r 22 − r 24 + r 23 + r 25 , y19 (0) = 0,



 0 
y20 = −r25 + r24 , y20 (0) = 0,
After computing the losses and their derivatives, we obtain the following linear system, where the symbol F represents the non-zero terms:

F F F F F F F F F F
    
0 0 0 0 0 0 0 0 0 0 1β 1 L1
F F 0 F F 0 0 0 0 F F 0 0 0 0 0 0 0 F 0   1β 2   L2 
F F F F F
    
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0   1β 3   L3 
F F F F   L4 
0   1β 4 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    

F F 0 0 F F F 0 F 0 0 0 0 F 0 0 F 0 0 0  1β   L 
  5  5
F F F F F F F F
0 0 0 0 0 0 0 0 0 0 0 0  1β 6   L6 
   

0
 0 0 0 0 F F 0 0 0 0 0 0 F 0 0 0 0 0 0  1β 7   L7 
   
0
 0 0 0 0 F F F F 0 0 0 0 0 0 0 0 0 0 0
 1β   L8 
8   
0
 0 0 0 0 F 0 0 F 0 0 0 0 0 0 0 0 0 0 0  1β 9   L9 
   
0
 F 0 0 0 0 0 0 F F F 0 0 0 0 0 0 0 0 0
1β 10  L10 
 =  .
F F F F F F
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1β 11  L11 
   

0
 F 0 0 0 0 0 0 0 0 F F 0 0 0 0 0 0 0 0
1β 12  L12 
  
F F F
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1β 13  L13 
   
F F F

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1β 14  L14 
F F F
    
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  1β 15  L15 
F F
    
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  1β 16  L16 
F F
    
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  1β 17  L17 
F F F
    
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  1β 18  L18 
F 0 0 F 0 0 0 0 0 0 0 0 0 0 0 0 0 0 F F  1β 19  L19 
F 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 F F 1β 20 L20

In Figs. 9 and 10, we reported X-TFC solutions and the abso- results are obtained with L = 10, nx = 10, fixed step size h = 0.001,
lute errors, respectively, compared with the solutions obtained by and tolerance of 10−9 . The training points are uniformly spaced
the MATLAB stiff ode solver ode15s, which we consider here the in a linear time scale. The computational times for t = [0 60] is
true solution for our qualitative and quantitative comparison. The ≈ 380 s (against 2 h needed in Ref. 19) and the average training error

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 138 32, 063107-12

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

TABLE III. Reaction scheme of the air pollution problem.

No. Reaction Reaction rate Rate constant

1 NO2 → NO + O3 P r 1 = k1 y 1 k1 = 0.35
2 NO + O3 → NO2 r 2 = k2 y 2 y 4 k2 = 26.6
3 HO2 + NO → NO2 + OH r 3 = k3 y 5 y 2 k3 = 0.123 × 105
4 HCHO → 2HO2 + CO r 4 = k4 y 7 k4 = 0.86 × 10−3
5 HCHO + OH → HO2 + CO r 5 = k5 y 7 k5 = 0.82 × 10−3
6 HCHO → CO r6 = k6 y7 y6 k6 = 0.15 × 105
7 ALD → MEO2 + HO2 + CO r7 = k7 y9 k7 = 0.13 × 10−3
8 ALD + OH → C2 O3 r 8 = k8 y 9 y 6 k8 = 0.24 × 105
9 C2 O3 + NO → NO2 + MEO2 + CO2 r9 = k9 y11 y2 k9 = 0.165 × 105
10 C2 O3 + NO2 → PAN r10 = k10 y11 y1 k10 = 0.9 × 104
11 PAN → C2 O3 + NO2 r11 = k11 y13 k11 = 0.22 × 10−1
12 MEO2 + NO → CH3 O + NO2 r12 = k12 y10 y2 k12 = 0.12 × 105
13 CH3 O → HCHO + HO2 r13 = k13 y14 k13 = 1.88
14 NO2 + OH → HNO3 r14 = k14 y1 y6 k14 = 0.163 × 105
15 O 3 P → O3 r15 = k15 y3 k15 = 0.48 × 107
16 O 3 → O1 D r16 = k16 y4 k16 = 0.35 × 10−3
17 O 3 → O3 P r17 = k17 y4 k17 = 0.0175
18 O1 D → 2OH r18 = k18 y16 k18 = 0.1 × 109
19 O 1 D → O3 P r19 = k19 y16 k19 = 0.444 × 1012
20 SO2 + OH → SO4 + HO2 r20 = k20 y17 y6 k20 = 0.124 × 104
21 NO3 → NO r21 = k21 y19 k21 = 2.1
22 NO3 → NO2 + O3 P r22 = k22 y19 k22 = 5.78
23 NO2 + O3 → NO3 r23 = k23 y1 y4 k23 = 0.0474
24 NO3 + NO2 → N2 O5 r24 = k24 y19 y1 k24 = 0.178 × 104
25 N2 O5 → NO3 + NO2 r25 = k25 y20 k25 = 3.12

FIG. 9. Species concentrations of the air pollution

(POLLU) problem computed with MATLAB ode15s and
X-TFC.

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 139 32, 063107-13

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

FIG. 10. Absolute values of the errors

between X-TFC and MATLAB ode15s for
each species concentration of the air pol-
lution (POLLU) problem.

is 3.5 × 10−08 . We can see how our solutions overlap with those of IVPs for ODEs. Source code is available from the authors upon
computed by ode15s, and the absolute errors are in the order of reasonable request.
10−25 /10−05 . Future works will focus on stiff problems related to different
fields such as biological systems. Also, the authors are currently
working on estimating the generalization error of X-TFC frame-
V. CONCLUSIONS works for boundary value problems.
A new, efficient, and precise algorithm, named X-TFC, is
employed in this work for solving stiff chemical kinetics prob- ACKNOWLEDGMENTS
lems. In X-TFC, the physics-informed neural network idea has been
merged with the theory of functional connections to analytically The authors would like to acknowledge Dr. Thomas Rigotti
satisfy the initial conditions, removing the issue of having compet- (Department of Chemistry, The Technical University of Munich)
ing objectives in the loss function. To reach high computational and Dr. Andrea D’Ambrosio (Department of Aeronautics and
efficiency, the shallow NN is trained via the ELM algorithm. An Astronautics, Massachusetts Institute of Technology) for fruitful
important highlight of X-TFC is that we obtain an analytical rep- discussions, and Jason Curtis for his support during the drafting of
resentation (via NNs) of chemical kinetics equations’ solutions once the manuscript.
the NN output weights are evaluated on the training points. Thus,
the solutions can be calculated at any required time instant in the
domain without further computational efforts (e.g., the NN does not AUTHOR DECLARATIONS
need to be re-trained). Conflict of Interest
In this work, the X-TFC algorithm with time decomposition
has been applied to solve stiff systems of ODEs, showing its accu- The authors have no conflicts to disclose.
racy and efficiency by comparing it with state-of-the-art methods
and benchmarks in the literature. No artifacts were needed to reduce
the stiffness of the problems, directly solving them in their original DATA AVAILABILITY
form. For the first time, we provided a rigorous upper bound on the Data sharing is not applicable to this article as no new data were
generalization error of X-TFC frameworks in learning the solutions created or analyzed in this study.

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 140 32, 063107-14

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

REFERENCES 23
M. De Florio, E. Schiassi, B. D. Ganapol, and R. Furfaro, “Physics-informed
1 neural networks for rarefied-gas dynamics: Thermal creep flow in the Bhatna-
H. Lee and I. S. Kang, “Neural algorithm for solving differential equations,”
gar–Gross–Krook approximation,” Phys. Fluids 33(4), 047110 (2021).
J. Comput. Phys. 91(1), 110–131 (1990). 24
2 M. De Florio, E. Schiassi, R. Furfaro, B. D. Ganapol, and D. Mostacci, “Solutions
A. J. Meade, Jr. and A. A. Fernandez, “The numerical solution of linear ordinary
of Chandrasekhar’s basic problem in radiative transfer via theory of functional
differential equations by feedforward neural networks,” Math. Comput. Model.
connections,” J. Quant. Spectrosc. Radiat. Trans. 259, 107384 (2020).
19(12), 1–25 (1994). 25
3 M. De Florio, E. Schiassi, A. D’Ambrosio, D. Mortari, and R. Furfaro, “Theory
A. J. Meade, Jr. and A. A. Fernandez, “Solution of nonlinear ordinary differential
of functional connections applied to linear ODEs subject to integral constraints
equations by feedforward neural networks,” Math. Comput. Model. 20(9), 19–44
and linear ordinary integro-differential equations,” Math. Comput. Appl. 26(3),
(1994).
4 65 (2021).
I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solv- 26
E. Schiassi, M. De Florio, A. D’Ambrosio, D. Mortari, and R. Furfaro, “Physics-
ing ordinary and partial differential equations,” IEEE Trans. Neural Netw. 9(5),
informed neural networks and functional interpolation for data-driven param-
987–1000 (1998).
5 eters discovery of epidemiological compartmental models,” Mathematics 9(17),
C. Filici, “On a neural approximator to ODEs,” IEEE Trans. Neural Netw. 19(3),
2069 (2021).
539–543 (2008). 27
6 E. Schiassi, M. De Florio, B. D. Ganapol, P. Picca, and R. Furfaro, “Physics-
J. J. Moré, B. S. Garbow, and K. E. Hillstrom, “User guide for minpack-1,”
informed neural networks for the point kinetics equations for nuclear reactor
Technical Report, CM-P00068642 (1980).
7 dynamics,” Ann. Nucl. Energy 167, 108833 (2021).
R. Gerstberger and P. Rentrop, “Feedforward neural nets as discretization 28
E. Schiassi, A. D’Ambrosio, H. Johnston, M. De Florio, K. Drozd, R. Fur-
schemes for ODEs and DAEs,” J. Comput. Appl. Math. 82(1-2), 117–128
faro, F. Curti, and D. Mortari, “Physics-informed extreme theory of functional
(1997).
8 connections applied to optimal orbit transfer,” in Proceedings of the AAS/AIAA
M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural net-
Astrodynamics Specialist Conference, Lake Tahoe, CA (AAS, 2020), pp. 9–13.
works: A deep learning framework for solving forward and inverse problems 29
K. Eriksson, C. Johnson, and A. Logg, “Explicit time-stepping for stiff ODEs,”
involving nonlinear partial differential equations,” J. Comput. Phys. 378, 686–707
SIAM J. Sci. Comput. 25(4), 1142–1157 (2004).
(2019). 30
9
A. M. Zhabotinsky, “Periodical oxidation of malonic acid in solution (a study of
S. Dong and Z. Li, “Local extreme learning machines and domain decomposition the Belousov reaction kinetics),” Biofizika 9, 306–311 (1964).
for solving linear and nonlinear partial differential equations,” Comput. Methods 31
J. N. Reddy, “An introduction to the finite element method,” J. Pressure Vessel
Appl. Mech. Eng. 387, 114129 (2021). Technol. 111, 348–349 (1989).
10 32
A. D. Jagtap, E. Kharazmi, and G. E. Karniadakis, “Conservative physics- S. Wang, Y. Teng, and P. Perdikaris, “Understanding and mitigating gradient
informed neural networks on discrete domains for conservation laws: Applica- pathologies in physics-informed neural networks,” arXiv:2001.04536 (2020).
tions to forward and inverse problems,” Comput. Methods Appl. Mech. Eng. 365, 33
P. Mertikopoulos, C. Papadimitriou, and G. Piliouras, “Cycles in adversarial
113028 (2020). regularized learning,” in Proceedings of the Twenty-Ninth Annual ACM-SIAM
11
G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Symposium on Discrete Algorithms (SIAM, 2018), pp. 2703–2717.
Yang, “Physics-informed machine learning,” Nat. Rev. Phys. 3(6), 422–440 34
D. Balduzzi, S. Racaniere, J. Martens, J. Foerster, K. Tuyls, and T. Graepel,
(2021). “The mechanics of n-player differentiable games,” in International Conference on
12
X. Meng, Z. Li, D. Zhang, and G. E. Karniadakis, “Ppinn: Parareal physics- Machine Learning (PMLR, 2018), pp. 354–363.
informed neural network for time-dependent PDEs,” Comput. Methods Appl. 35
D. Mortari, “The theory of connections: Connecting points,” Mathematics 5(4),
Mech. Eng. 370, 113250 (2020). 57 (2017).
13 36
S. Wang, Y. Teng, and P. Perdikaris, “Understanding and mitigating gradient D. Mortari, “Least-squares solution of linear differential equations,” Mathematics
flow pathologies in physics-informed neural networks,” SIAM J. Sci. Comput. 5(4), 48 (2017).
43(5), A3055–A3081 (2021). 37
D. Mortari, H. Johnston, and L. Smith, “High accuracy least-squares solu-
14
D. A. Goussis and M. Valorani, “An efficient iterative algorithm for the approx- tions of nonlinear differential equations,” J. Comput. Appl. Math. 352, 293–307
imation of the fast and slow dynamics of stiff systems,” J. Comput. Phys. 214(1), (2019).
316–346 (2006). 38
G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: Theory
15
M. Hadjinicolaou and D. A. Goussis, “Asymptotic solution of stiff PDEs with and applications,” Neurocomputing 70, 489–501 (2006).
the CSP method: The reaction diffusion equation,” SIAM J. Sci. Comput. 20(3), 39
S. Mishra and R. Molinaro, “Estimates on the generalization error of physics
781–810 (1998). informed neural networks (pinns) for approximating PDEs,” arXiv:2006.16144
16
M. Valorani and D. A. Goussis, “Explicit time-scale splitting algorithm for stiff (2020).
problems: Auto-ignition of gaseous mixtures behind a steady shock,” J. Comput. 40
M. R. Roussel, “The use of delay differential equations in chemical kinetics,”
Phys. 169(1), 44–79 (2001). J. Phys. Chem. 100(20), 8323–8330 (1996).
17 41
E. Galaris, F. Calabrò, D. di Serafino, and C. Siettos, “Numerical solution of J. D. Lambert et al., Numerical Methods for Ordinary Differential Systems
stiff ordinary differential equations with random projection neural networks,” (Wiley, New York, 1991), Vol. 146.
arXiv:2108.01584 (2021). 42
G. Wanner and E. Hairer, Solving Ordinary Differential Equations II (Springer,
18
W. Ji, W. Qiu, Z. Shi, S. Pan, and S. Deng, “Stiff-PINN: Physics-informed neu- 1996), Vol. 375.
ral network for stiff chemical kinetics,” J. Phys. Chem. A 125(36), 8098–8106 43
W. M. Lioen and J. J. de Swart, Test Set for Initial Value Problem Solvers
(2021). (Centrum voor Wiskunde en Informatica, 1998).
19 44
S. Kim, W. Ji, S. Deng, Y. Ma, and C. Rackauckas, “Stiff neural ordinary A. M. Zhabotinskii, “Periodic course of the oxidation of malonic acid in a
differential equations,” Chaos 31(9), 093122 (2021). solution (studies on the kinetics of Beolusov’s reaction),” Biofizika 9, 306–311
20
H. Robertson, “The solution of a set of reaction rate equations,” Numerical (1964).
Analysis: An Introduction (Academic Press, 1966), Vol. 178182. 45
M. Mehdizadeh Khalsaraei, A. Shokri, and M. Molayi, “The new high approx-
21
J. G. Verwer, “Gauss–Seidel iteration for stiff ODEs from chemical kinetics,” imation of stiff systems of first order IVPs arising from chemical reactions
SIAM J. Sci. Comput. 15(5), 1243–1250 (1994). by k-step l-stable hybrid methods,” Iranian J. Math. Chem. 10(2), 181–193
22
E. Schiassi, R. Furfaro, C. Leake, M. De Florio, H. Johnston, and D. Mor- (2019).
tari, “Extreme theory of functional connections: A fast physics-informed neu- 46
T. Abdulla, J. Cash, and M. Diamantakis, “An MEBDF package for the numeri-
ral network method for solving ordinary and partial differential equations,” cal solution of large sparse systems of stiff initial value problems,” Comput. Math.
Neurocomputing 457, 334–356 (2021). Appl. 42(1-2), 121–129 (2001).

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 141 32, 063107-15

Published under an exclusive license by AIP Publishing
 Chaos ARTICLE scitation.org/journal/cha

47 51
L. F. Shampine, Numerical Solution of Ordinary Differential Equations (Rout- V. Shulyk, O. Klymenko, and I. Svir, “Numerical solution of stiff ODEs describ-
ledge, 2018). ing complex homogeneous chemical processes,” J. Math. Chem. 43(1), 252–264
48
W. M. Lioen, J. J. deSwart, and W. van deVeen, “Test set for IVP solvers,” (2008).
52
Department of Numerical Mathematics [NM], no. R 9615 (1996). A. C. Hindmarsh, “LSODE and LSODI: Two new initial value ordi-
49
M. Falati and G. Hojjati, “Integration of chemical stiff ODEs using exponential nary differential equation solvers,” ACM Signum Newslett. 15(4), 10–11
propagation method,” J. Math. Chem. 49(10), 2210–2230 (2011). (1980).
50 53
O. Klymenko and I. Svir, “Modelling complex chemical processes in homo- F. De Leeuw, “Numerical solution of ordinary differential equations
geneous solutions: Automatic numerical simulation,” Nonlinear Anal. Model. arising from chemical kinetics,” RIVM Rapport 228603005
Control 11(3), 247–261 (2006). (1988).

Chaos 32, 063107 (2022); doi: 10.1063/5.0086649 142 32, 063107-16

Published under an exclusive license by AIP Publishing