Mario R. Eden, Marianthi Ierapetritou and Gavin P.

Towler (Editors) Proceedings of the 13th

International Symposium on Process Systems Engineering – PSE 2018
July 1-5, 2018, San Diego, California, USA © 2018 Elsevier B.V. All rights reserved.
https://doi.org/10.1016/B978-0-444-64241-7.50363-3

Next Generation Multi-Scale Process Systems

Engineering Framework
David C. Millera*, John D. Siirolab, Deb Agarwald, Anthony P. Burgarda,
Andrew Leea, John C. Eslicka, Bethany Nicholsonb, Carl Lairdb, Lorenz T.
Bieglerc, Debangsu Bhattacharyyae, Nikolaos V. Sahinidisc, Ignacio E.
Grossmannc, Chrysanthos E. Gounarisc, Dan Gunterd
a
National Energy Technology Laboratory, 626 Cochrans Mill Road, Pittsburgh, PA
15236, USA
b
Sandia National Laboratories, P.O. Box 5800, Albuquerque, NM 87185-1326 USA
c
Department of Chemical Engineering, Carnegie Mellon University, 5000 Forbes
Avenue, Pittsburgh, PA 15213-3890 USA
d
Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720 USA
e
Department of Chemical and Biomedical Engineering, West Virginia University, 401
Evansdale Drive, Morgantown, West Virginia 26506-6070 USA
david.miller@netl.doe.gov

Abstract
The IDAES PSE framework represents a new approach for the design and optimization
of innovative steady state and dynamic processes by integrating an extensible, equation-
oriented process model library with the Pyomo algebraic modeling language. Built spe-
cifically to enable rigorous large-scale mathematical optimization, the framework
includes capabilities for conceptual design, steady state and dynamic optimization, multi-
scale modeling, uncertainty quantification, and the automated development of thermody-
namic, physical property, and kinetic submodels from experimental data.
Keywords: Multiscale modeling, conceptual design, process optimization, dynamic op-
timization, parameter estimation

1. Introduction
The U.S. Department of Energy’s Institute for the Design of Advanced Energy Systems
(IDAES) was formed in 2016 to develop new advanced process systems engineering
(PSE) capabilities to support the design and optimization of innovative new processes
that go beyond current equipment/process constraints. Engineers currently rely on state-
of-the-art simulation packages to synthesize, model, and optimize process flowsheets
with sufficiently detailed models. However, commercial simulation tools struggle with
flowsheet optimization and typically limit consideration of nonlinear optimization prob-
lems to less than 100 degrees of freedom. As a result, process optimization with large-
scale solvers is rarely performed by industry. On the other hand, many of the capabilities
for large-scale optimization are available in general algebraic modeling languages
(AMLs), which generally require fully open, equation-oriented (EO) models with exact
first and second derivatives. Large-scale NLP algorithms routinely solve optimization
problems with thousands of variables and degrees of freedom. Moreover, the extension
of these optimization models for MINLP, global optimization, and optimization under
uncertainty (OUU) requires a flexible and open optimization modeling environment.
2210 D. Miller et al.

The open-source IDAES process systems engineering framework addresses the capability
gap between state-of-the-art simulation packages and AMLs by integrating an extensible,
equation-oriented process model library with the Pyomo AML (Hart et al, 2011, 2017),
which enables formulating, manipulating, and solving large, complex, structured optimi-
zation problems. The IDAES framework includes tools for: (1) process synthesis and
conceptual design, including process intensification, (2) process design and optimization,
including process integration, (3) process control and dynamic optimization, (4) using
advanced solvers and computer architectures, (5) automated development of thermody-
namic, physical property, and kinetic submodels from experimental data, (6) integration
of multi-scale models, (7) comprehensive, end-to-end uncertainty quantification (UQ)
and model validation, (8) maintaining complete provenance information, and (9) the abil-
ity to support multiple time and length scales, from materials to process to market.

2. IDAES PSE Framework

The IDAES framework consists of multiple components as shown in Figure 1. The core
of the IDAES framework is the customizable process model library, built on the Pyomo
AML. The core models can be combined to capture the hierarchical structure of sub-pro-
cesses and complex unit models. In addition, the process model library can be used to
support conceptual design via a superstructure formulation, process design, optimization,
and integration, and dynamic optimization and control. A suite of machine learning tech-
niques based on ALAMO (Cozad et al., 2014) enables the development of submodels for
physical properties, thermodynamics and kinetics directly from experimental data, ena-
bling more effective tracking of uncertainty. To enable the solution of large-scale
parameter estimation problems, the machine learning tools can utilize a variety of high
performance computing (HPC) capabilities, include GPUs and the National Energy Re-
search Scientific Computing Center (NERSC). Finally, the framework utilizes PySP
(Watson et al., 2012) to incorporate and assess uncertainty across models and scales as
well as to enable simultaneous, multiscale parameter estimation. All components are de-
veloped using open-source best practices on the Github collaborative coding platform.
2.1. Pyomo Algebraic Modeling Language
The IDAES PSE Framework leverages the open source, DOE-funded platform, Pyomo
(Hart et al., 2011, 2017), which addresses challenges in formulating, manipulating, and
solving large complex structured optimization problems. Pyomo is an extensible algebraic

Figure 1. The major components and connection of the IDAES PSE Framework to support concep-
tual design, process design & optimization, process integration, dynamic optimization, and control
while incorporating multi-scale surrogate models.
Next Generation Multi-Scale Process Systems Engineering Framework 2211

modeling environment written in the Python® programming language. It allows users to

formulate, initialize, solve, and manipulate large-scale optimization problems with con-
cise notation all within the Python ecosystem. Pyomo provides interfaces to dozens of
optimization solvers, both open source and commercial, and supports executing solvers
both locally and in distributed environments. Most importantly, Pyomo is extensible be-
yond “standard” algebraic optimization models. In particular, Pyomo provides extensions
for block-oriented modeling and Generalized Disjunctive Programming (Friedman, et al.,
2013), stochastic programming (Watson, et al., 2012), and dynamic systems containing
(partial) differential-algebraic equations (Nicholson, et al., 2017). This enables modeling
flexibility and the consideration of constraints and objective functions in non-standard
forms not easily handled by other modeling frameworks. Further, Pyomo enables com-
plex analyses by combining these extensions and capabilities with the ability to script
custom algorithms and solution processes.
2.2. Hierarchical Equation Oriented Process Models
A major strength of commercial simulation packages is their library of unit models and
thermo-physical properties. Such simulation packages, however, often have limited sup-
port for incorporating non-standard, dynamic unit models, and they do not directly
support UQ. AMLs, on the other hand, are eminently flexible and readily support large-
scale optimization, but considerable work is required to construct process models, which
are often not easily reused. To overcome this issue, the IDAES framework extends
Pyomo’s block-based hierarchical modeling constructs to create a library of models for
common process unit operations and a framework for rapid development of process flow-
sheets. Rather than developing highly detailed, but rigidly structured models, the IDAES
model library consists of simpler, but extensible, models for each unit operation (Lee et
al., 2018). The model library contains an internally consistent suite of interoperable mod-
els at different levels of complexity and accuracy for both steady-state and dynamic
analysis. Models within the library can be extended with additional detail or rigor as
needed, allowing full control of the complexity and accuracy of the models. This focus
on adaptability and extensibility enables them to be used for novel processes and config-
urations.
2.3. Machine Learning and Parameter Estimation
Although commercial simulation tools have extensive databases and pre-regressed mod-
els for thermo-physical properties, when investigating innovative, new processes with
new materials, such databases are of limited use. In addition, it is not always straightfor-
ward to assess the uncertainty inherent in these models. Thus, tools for modeling physical
property and reaction kinetics have been developed based on ALAMO, which builds al-
gebraic models that work well in an optimization framework. Building these models
directly from data (either from literature or experiments) ensures they address the right
range of conditions and enable tight coupling of uncertainty estimates that allow end to
end UQ. In addition, through the data management framework, provenance can be more
clearly established enabling tracking through the development of a new technology.
The Reaction Identification and Parameter Estimation (RIPE) tool can estimate kinetic
reaction parameters from data describing the time evolution of concentrations of chemi-
cals in a reaction system that may include reversible and irreversible reactions as well as
homogeneous and heterogeneous catalytic mechanisms. ALAMO is first used to obtain
algebraic models of concentration profiles in terms of polynomial, exponential, and log-
arithmic basis functions, then the constrained regression features of ALAMO are used for
2212 D. Miller et al.

data reconciliation by requiring the time profiles to satisfy physical bounds on concentra-
tions – even over time domains for which concentration measurements do not exist
(Wilson and Sahinidis, 2017). Finally, RIPE utilizes integer programming techniques to
obtain explicit mathematical representations of the dynamics of the system.
More broadly, parameter estimation for multi-scale systems is a significant challenge
since only a subset of parameters can be estimated from data at a given scale. Complex
interactions among the parameters can lead to biased estimates unless they are estimated
simultaneously. This can lead to a large-scale optimization problem with data that span
multiple scales and contain inherent noise. Traditional modeling approaches are often not
amenable to such a simultaneous approach since process models and physical prop-
erty/thermodynamic models are often developed separately. In the IDAES framework, all
models and sub-models are implemented in a common platform allowing access to all
model parameters, enabling formulation of the complete, simultaneous multiscale prob-
lem. The problem can then be readily solved using the capabilities of the PySP module
(Watson et al., 2011). The IDAES framework can also be used to develop optimal design
of experiments for minimizing the variance of the parameter estimates.
2.4. Process synthesis and conceptual design
The major goal in process synthesis and conceptual design is to optimize, reliably and
efficiently, superstructures of process systems to help identify novel process configura-
tions and enable process intensification. These remain major research challenges (Chen
and Grossmann, 2017). The approach relies on Generalized Disjunctive Programming
(GDP) (Grossmann and Trespalacios, 2013) in which a model consisting of equations,
disjunctions, and logic statements is used instead of formulating and solving a single
monolithic MINLP model. Thus, only NLP subproblems corresponding to fixed flow-
sheet structures need to be solved, which greatly increases the robustness of the nonlinear
optimization. Furthermore, the conceptual design algorithms in IDAES directly leverage
the native support of Generalized Disjunctive Programming in Pyomo. Several test prob-
lems have been solved successfully with this approach, including a simplified version of
the chemical looping process.
2.5. Process design and optimization
The IDAES framework enables large-scale system-wide optimization using rigorous
models of process equipment and thermophysical properties. Complex interactions be-
tween multiple process systems can be explored simultaneously without the need for
extensive simplifying assumptions that render the problems tractable at the risk of com-
promising the results. For example, both traditional and advanced combustion
technologies such as oxycombustion or chemical looping combustion can be optimized
together with their power islands and pollution control systems without relying on fixed
steam cycle designs, the use of overly simplistic “Gibbs reactors” that cannot capture key
nonlinear responses to operating conditions, or fixed design points from CFD simulations
that overlook critical decision variables. Methods for enabling optimization over broad
ranges of operation have also been developed that can handle non-smooth features such
as vapor/liquid or sub/supercritical phase transitions as well as vanishing phases. The
process design and optimization capabilities developed within IDAES are applicable not
only to the energy sector but also to several other industries (e.g., chemicals, petrochem-
icals, pharmaceuticals) where tight integration is desired.
Next Generation Multi-Scale Process Systems Engineering Framework 2213

2.6. Dynamics and control

The dynamic process simulation and optimization capability builds on Pyomo’s support
for expressing ordinary and partial differential equations as large-scale systems of non-
linear equations. The IDAES process model library includes both steady-state and
dynamic versions of common unit models allowing users to switch between steady-state
and dynamic analysis with minimal changes to their flowsheet implementation. In addi-
tion, the IDAES framework includes capabilities for state estimation and control which
enable the rapid deployment of approaches such as moving horizon estimation (MHE)
and nonlinear model predictive control (NMPC) to arbitrary process flowsheets.
2.7. Multi-scale surrogate modeling and optimization
Even within a powerful equation-oriented platform for process optimization, complex,
black box models that resist a direct equation-based description must often be included.
These include CFD models for advanced combustors, reactors and separators as well as
complex physical property models. When incorporating simplified, surrogate models of
such complex models, optimizers often exploit the weaknesses of surrogate models and
fail to find the true optimum. To guarantee convergence to the optimal solution, we have
developed a trust region filter algorithm, which monitors and controls the errors in the
reduced model and guarantees convergence to first order critical points of the complex
model. The trust region filter algorithm, based on the SQP filter method along with con-
cepts of Derivative Free Optimization, is developed, analyzed and demonstrated on
challenging power plant models in Eason and Biegler (2016). Now implemented in
Pyomo, it enhances a powerful optimization capability to include complex black box
models.
2.8. Data management
The IDAES Data Management Framework (DMF) is responsible for managing the
knowledge and provenance required for the synthesis, design, optimization, and scale up
of innovative new processes. The DMF stores property data, inputs, model parameters,
model codes, and results in an easily searchable form. Standard representations of prov-
enance (e.g., journal publications and websites) support validation of results. The DMF’s
Python API can be used within scripts or for interactive analyses in Jupyter Notebooks.
The Jupyter Notebooks are themselves stored in the DMF, providing a self-contained re-
producible unit of analysis. The API also enables efficient access to large scale data sets
for parameter estimation to other components of the IDAES PSE framework.
2.9. Other applications
The core approach of using data to develop submodels, which can then be utilized to
formulate larger-scoped optimization problems, has also be applied to materials design
(Hanselman and Gounaris, 2017) and multi-scale infrastructure planning (Lara et al.,
2017). In the case of designing crystalline materials, for example, computational chemis-
try data is used to develop simplified structure-function relationships linking atomic
neighborhood descriptors to the reactivity of atomic sites. These relationships can then be
incorporated into an optimization model that seeks to identify the overall material struc-
ture that performs best in the context of the process of interest. In the case of infrastructure
planning, the IDAES framework has been used to determine representative dispatch days
and growth forecast for an electricity grid, which forms the basis for determining what
types of power plants should be built to most economically meet all the expected demand.
2214 D. Miller et al.

3. Conclusions
In conclusion, this paper has described a comprehensive, open source framework to ena-
ble large scale process optimization of complex, innovative processes spanning
conceptual design and process synthesis to dynamic optimization. The capabilities of this
framework are applicable at the process scale as well as for materials design and infra-
structure planning.
This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United
States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes
any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or
process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific com-
mercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or
imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and
opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency
thereof. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engi-
neering Solutions of Sandia LLC, a wholly owned subsidiary of Honeywell International Inc. for the U.S. Department of
Energy’s National Nuclear Security Administration under contract DE-NA0003525. KeyLogic Systems, Inc.’s contributions to
this work were funded by the National Energy Technology Laboratory under the Mission Execution and Strategic Analysis
contract (DE-FE0025912) for support services. This material is based upon work supported by the U.S. Department of Energy,
Office of Science, under Contract No. DEAC02-05CH11231. This research used resources of the National Energy Research
Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department
of Energy under Contract No. DE-AC02-05CH11231.

References
Q. Chen, I.E. Grossmann, 2017, Recent developments and challenges in optimization-based
process synthesis, Annual Review of Chemical and Biomolecular Engineering, 8, 249-283.
A. Cozad, N.V. Sahinidis and D.C. Miller, 2014, Learning surrogate models for simulation-based
optimization, AIChE Journal, 60, 2211-2227.
J.P. Eason, L.T. Biegler, 2016, A trust region filter method for glass box/black box optimization,
AIChE Journal, 62, 3124-3136.
Z. Friedman, J. Ingalls, J.D. Siirola, and J.-P. Watson, 2013, Block-oriented modeling of
superstructure optimization problems, Computers & Chemical Engineering 57, 10-23.
I.E. Grossmann, F. Trespalacios, 2013, Systematic Modeling of Discrete-Continuous
Optimization Models through Generalized Disjunctive Programming, AIChE J. 59, 3276-
3295.
C. L. Hanselman, C.E. Gounaris, 2016, A mathematical optimization framework for the design of
nanopatterned surfaces, AIChE Journal 62, 9, 3250-3263.
W.E. Hart, J.-P. Watson, and D.L. Woodruff, 2011, Pyomo: modeling and solving mathematical
programs in Python, Mathematical Programming Computation 3, 3, 219-260.
W.E. Hart, C.D. Laird, J.-P. Watson, D.L. Woodruff, G.A. Hackebeil, B.L. Nicholson, and J.D.
Siirola, 2017, Pyomo – Optimization Modeling in Python. Second Edition. Vol. 67. Springer.
C.L. Lara, D. Mallapragada, D. Papageorgiou, A. Venkatesh, I.E. Grossmann, 2017, Electric
Power Infrastructure Planning: Mixed-Integer Programming Model and Nested
Decomposition Algorithm, Submitted for publication, 1-36.
A. Lee, J.H. Ghouse, Q. Chen, J.C. Eslick, J.D. Siirola, I.E. Grossman, D.C. Miller, 2018, A
Flexible Framework and Model Library for Process Simulation, Optimization and Control,
Proceedings of the 13th International Symposium on Process Systems Engineering – PSE
2018, July 1-5, 2018, San Diego, California, USA.
B. L. Nicholson, J. D. Siirola, J. P. Watson, V. M. Zavala, L. T. Biegler (2017). Pyomo.DAE: A
modeling and automatic discretization framework for optimization with differential and
algebraic equations. Mathematical Programming Computation, 10.1007/s12532-017-0127-0
J.P. Watson, D. L. Woodruff, and WHart, 2012, PySP: modeling and solving stochastic programs
in Python. Mathematical Programming Computation 4, 109-149.
Z.T. Wilson, N.V. Sahinidis, 2017, The ALAMO approach to machine learning, Computers and
Chemical Engineering, 106, 785-795.