ElmerSolver

Setup, execution Thomas Zwinger

thomas.zwinger[at]csc.fi

Computational Environment & Application CSCScientic Computing Ltd. The Finnish IT center for science Espoo, Finland

CSC

Zwinger p.1/33

Contents
On Bodies, Elements and Boundaries Finite Elements Linear Solver Preconditioning Solution Levels The Solver Input File (SIF) Header Simulation Solver Body Equation Bodyforce Material Initial Conditions Boundary Conditions Bodies on Boundaries Executing Elmer Tables MATC User Dened Functions User Dened Subroutine Multiple Meshes Element Types Specialities Elmer parallel version
CSC

Zwinger p.2/33

On Bodies and Boundaries

boundary element

body 2

body 1 node

bulk element

CSC

Zwinger p.3/33

Finite Elements
3 linear 2 5 1 1 4 2 3 Real geometry mesh 2 Elmer unit size elements quadratic 6

+ Coordinatesystem metric 3 3

CSC

Zwinger p.4/33

Finite Elements contd.

General advection-diffusion:

/t + u = () +

CSC

Zwinger p.5/33

Finite Elements contd.

General advection-diffusion:

/t + u = () +
Test function, and integration over domain :

/t d+

u d =

() d+

CSC

Zwinger p.5/33

Finite Elements contd.

General advection-diffusion:

/t + u = () +
Test function, and integration over domain :

/t d+

u d =

() d+

Partial integration of diffusion term:

/t d +

u d +

d = d

CSC

( ) nd +

Zwinger p.5/33

Finite Elements contd.

Discretization of variable::

CSC

Zwinger p.6/33

Finite Elements contd.

Discretization of variable:: and time derivative: /t a(t) b(t)tt

CSC

Zwinger p.6/33

Finite Elements contd.

Discretization of variable:: and time derivative: /t a(t) b(t)tt

a(t)

d +
M

u + d =
S

( ) nd + b(t)tt

nat. BC

d +

d
f

timeforce

CSC

Zwinger p.6/33

Finite Elements contd.

Discretization of variable:: and time derivative: /t a(t) b(t)tt

a(t)

d +
M

u + d =
S

( ) nd + b(t)tt

nat. BC

d +

d
f

timeforce

(M + S) = f
M. . . Mass matrix, S. . . Stiffness matrix, f . . . force vector
CSC

Zwinger p.6/33

Linear Solver
Three solution methods for A = f
Linear System Solver = Keyword

CSC

Zwinger p.7/33

Linear Solver
Three solution methods for A = f
Linear System Solver = Keyword

Direct methods (Keyword:

Direct)

Linear System Direct Method =

standard LAPACK ( banded ) alternatively UMFPACK - Unsymmetrical Multi Frontal ( UMFPACK )

CSC

Zwinger p.7/33

Linear Solver
Three solution methods for A = f
Linear System Solver = Keyword

Direct methods (Keyword:

Direct)

Linear System Direct Method =

standard LAPACK ( banded ) alternatively UMFPACK - Unsymmetrical Multi Frontal ( UMFPACK ) Krylov subspace iterative methods (Keyword:
Linear System Iterative Method =
Conjugate Gradient ( ( (

Iterative)

CG ), Conjugate Gradient Squared ( CGS ), BiConjugate Gradient Stabilized

BiCGStab ), Transpose-Free Quasi-Minimal Residual ( TFQMR ), Generalized Minimal Residual GMRES )

CSC

Zwinger p.7/33

Linear Solver
Three solution methods for A = f
Linear System Solver = Keyword

Direct methods (Keyword:

Direct)

Linear System Direct Method =

standard LAPACK ( banded ) alternatively UMFPACK - Unsymmetrical Multi Frontal ( UMFPACK ) Krylov subspace iterative methods (Keyword:
Linear System Iterative Method =
Conjugate Gradient ( ( (

Iterative)

CG ), Conjugate Gradient Squared ( CGS ), BiConjugate Gradient Stabilized

BiCGStab ), Transpose-Free Quasi-Minimal Residual ( TFQMR ), Generalized Minimal Residual GMRES )

Multilevel (Keyword:

Multigrid) Geometric (GMG) and Algebraic (AMG) Multigrid

CSC

Zwinger p.7/33

Precondition strategies
A=f

CSC

Zwinger p.8/33

Precondition strategies
A=f
solving instead:

AM1 = f , with = M

CSC

Zwinger p.8/33

Precondition strategies
A=f
solving instead:

AM1 = f , with = M
Why?: AM1 shall have a towards convergence of iterative methods improved spectrum

CSC

Zwinger p.8/33

Precondition strategies
A=f
solving instead:

AM1 = f , with = M
Why?: AM1 shall have a towards convergence of iterative methods improved spectrum Linear System Preconditioning =
None Diagonal ILUn n = 0,1,2,. . . ILUT Multigrid

CSC

Zwinger p.8/33

Solution Levels
time integration steady state iteration Solver 1 non-linear iteration linear iteration . . . end linear iteration end non-linear iteration Solver 2 direct solver . . . end steady state iteration
CSC

Time Steps

end time integration

Zwinger p.9/33

Solution Levels
time integration steady state iteration Solver 1 non-linear iteration linear iteration . . . end linear iteration end non-linear iteration Solver 2 direct solver . . . end steady state iteration end time integration
CSC

Time Steps Steady State Max Iterations

Steady State Convergence Tolerance

Zwinger p.9/33

Solution Levels
time integration steady state iteration Solver 1 non-linear iteration linear iteration . . . end linear iteration end non-linear iteration Solver 2 direct solver . . . end steady state iteration end time integration

Time Steps Steady State Max Iterations

Nonlinear Max Iterations

Nonlinear System Convergence Tolerance

Steady State Convergence Tolerance

CSC

Zwinger p.9/33

Solution Levels
time integration steady state iteration Solver 1 non-linear iteration linear iteration . . . end linear iteration end non-linear iteration Solver 2 direct solver . . . end steady state iteration end time integration

Time Steps Steady State Max Iterations

Nonlinear Max Iterations Linear System Max Iterations

Linear System Convergence Tolerance Nonlinear System Convergence Tolerance

Steady State Convergence Tolerance

CSC

Zwinger p.9/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . .

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor The Rules:

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor The Rules: comments start with

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor The Rules: comments start with

Important: do not use tabulators for indents!

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor The Rules: comments start with

! End

Important: do not use tabulators for indents! a section always ends with the keyword

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor The Rules: comments start with

! End

Important: do not use tabulators for indents! a section always ends with the keyword

parameters (except from Elmer keyword database) need to be casted by their types: Integer Real Logical String File

CSC

Zwinger p.10/33

The Solver Input File (SIF)

contains all the information for the solution step,

ElmerSolver mpi
can be exported by ElmerFront . . . . . . but simply also composed using a text editor The Rules: comments start with

! End

Important: do not use tabulators for indents! a section always ends with the keyword

parameters (except from Elmer keyword database) need to be casted by their types: Integer Real Logical String File
Parametername(n,m)

indicates a nm array
CSC

Zwinger p.10/33

Header
The header declares where to search for the mesh database

Header Mesh DB "." "dirname" End

preceding path + directory name of mesh database

CSC

Zwinger p.11/33

Constants
Declaration of constant values that can be obtained from within every solver and boundary condition subroutine or function, can be declared.

Constants Gas Constant = Real 8.314E00 Gravity (4) = 0 -1 0 9.81 End

a scalar constant Gravity vector, an array with a registered name

CSC

Zwinger p.12/33

Simulation
Principle declarations for simulation
Simulation Coordinate System = "Cartesian 2D"
choices:

Cartesian {1D,2D,3D}, Polar {2D,3D}, Cylindric, Cylindric Symmetric, Axi Symmetric

Coordinate Mapping(3) = Integer 1 2 3 Simulation Type ="Steady" Output Intervals = 1 Steady State Max Iterations = 10 Steady State Min Iterations = 2 Output File = "name.result" Post File = "name.ep" max output level = n End

permute, if you want to interchange directions either Steady or Transient how often you want to have results maximum rounds on one time level minimum rounds on one Timestep contains data to restart run

ElmerPost-le n=1 talkative like a Finnish lumberjack, n=42 all and everything

CSC

Zwinger p.13/33

Solver
Example: (Navier) Stokes solver
Solver 1 Equation = "Navier-Stokes" Linear System Solver = "Direct" Linear System Direct Method = "UMFPack" Linear System Convergence Tolerance = 1.0E-06 Linear System Abort Not Converged = True Linear System Preconditioning = "ILU2" Steady State Convergence Tolerance = 1.0E-03 Stabilization Method = Stabilized Nonlinear System Convergence Tolerance = 1.0E-05 Nonlinear System Max Iterations = 1 Nonlinear System Min Iterations = 1 Nonlinear System Newton After Iterations = 30 Nonlinear System Newton After Tolerance = 1.0E-05 End
CSC

name of the solver alt.

Iterative

a linear problem

Zwinger p.14/33

Body
Here the different bodies (there can be more than one) get their Equation, Material, Body Force and Initial Condition assigned
Body 1 Name = "identifier" Equation = 1 Material = 1 Body Force = 1 Initial Condition = 1 End
give the body a name

CSC

Zwinger p.15/33

Equation
set active solvers give keywords for the behaviour of different solvers
Equation 1 Active Solvers(2) = 1 2 Convection = Computed Flow Solution Name = String "Flow Solution" NS Convect = False End

CSC

Zwinger p.16/33

Bodyforce
declares the solver-specic f from A = f for the body body force can also be a dependent function (see later). Here for the (Navier) Stokes solver
Body Force 1 Flow BodyForce 1 = 0.0 Flow BodyForce 2 = -9.81 ! End good old gravity

CSC

Zwinger p.17/33

Material
sets material properties for the body. material properties can be scalars or tensors and also can be given as dependent functions
Material 1 Viscosity = 1.0E13 Density = 918.0 My Heat Capacity = Real 1002.0 End
not in keyword DB!

CSC

Zwinger p.18/33

Initial Conditions
initializes variable values sets initial guess for steady state simulation sets initial value for transient simulation variable values can be functions
Initial Condition 1 Velocity 1 = 0.0 Velocity 2 = 1.0 Pressure = 0.0 My Variable = Real 0.0 End
not in keyword DB

CSC

Zwinger p.19/33

Boundary Conditions
Dirichlet:

variablename = value Heat Flux

Neumann: often enabled with keyword (e.g., HTEqu. BC = True) followed by the ux value No BC Natural BC! values can be given as functions

Example: (Navier) Stokes with no penetration (normal) and free slip (tangential)
Boundary Condition 1 Name = "slip" Target Boundaries = 4 Normal-Tangential Velocity = Logical True Velocity 1 = Real 0.0 End
CSC

name refers to boundary no. 4 in mesh

components with respect to surface nor normal component

Zwinger p.20/33

Bodies on Boundaries
need to solve (dimension-1) PDEs (e.g., kinematic BC on free surface)

CSC

Zwinger p.21/33

Bodies on Boundaries
need to solve (dimension-1) PDEs (e.g., kinematic BC on free surface) need to dene the (dimension-1) entity as a separate body

CSC

Zwinger p.21/33

Bodies on Boundaries
need to solve (dimension-1) PDEs (e.g., kinematic BC on free surface) need to dene the (dimension-1) entity as a separate body in the corresponding Boundary-section: Body ID = n with n > highest occurring body in the mesh

CSC

Zwinger p.21/33

Bodies on Boundaries
need to solve (dimension-1) PDEs (e.g., kinematic BC on free surface) need to dene the (dimension-1) entity as a separate body in the corresponding Boundary-section: Body ID = n with n > highest occurring body in the mesh dene Body that body
Force, Material, Equation

and Initial

Condition

for

CSC

Zwinger p.21/33

Bodies on Boundaries
need to solve (dimension-1) PDEs (e.g., kinematic BC on free surface) need to dene the (dimension-1) entity as a separate body in the corresponding Boundary-section: Body ID = n with n > highest occurring body in the mesh dene Body that body
Force, Material, Equation

and Initial

Condition

for

full dimensional metric is still valid on the BC body has to be taken into account in user supplied subroutines

CSC

Zwinger p.21/33

Executing Elmer
make sure that the directory $ELMER of your OS

HOME/bin is in the path

CSC

Zwinger p.22/33

Executing Elmer
make sure that the directory $ELMER of your OS serial either launch directly, with

HOME/bin is in the path

ElmerSolver mysolverinputfile.sif

CSC

Zwinger p.22/33

Executing Elmer
make sure that the directory $ELMER of your OS serial either launch directly, with

HOME/bin is in the path

ElmerSolver mysolverinputfile.sif

or write create a le ELMERSOLVER where you launch from

STARTINFO in the directory

CSC

Zwinger p.22/33

Executing Elmer
make sure that the directory $ELMER of your OS serial either launch directly, with

HOME/bin is in the path

ElmerSolver mysolverinputfile.sif

or write create a le ELMERSOLVER where you launch from e.g.

STARTINFO in the directory

echo mysolverinputfile.sif > ELMERSOLVER STARTINFO

CSC

Zwinger p.22/33

Executing Elmer
make sure that the directory $ELMER of your OS serial either launch directly, with

HOME/bin is in the path

ElmerSolver mysolverinputfile.sif

or write create a le ELMERSOLVER where you launch from e.g.

STARTINFO in the directory

echo mysolverinputfile.sif > ELMERSOLVER STARTINFO ElmerSolver

launch simply with

CSC

Zwinger p.22/33

Executing Elmer
make sure that the directory $ELMER of your OS serial either launch directly, with

HOME/bin is in the path

ElmerSolver mysolverinputfile.sif

or write create a le ELMERSOLVER where you launch from e.g.

STARTINFO in the directory

echo mysolverinputfile.sif > ELMERSOLVER STARTINFO ElmerSolver

launch simply with

parallel needs ELMERSOLVER distributed execution)

STARTINFO plus evtl. a host-le (if

CSC

Zwinger p.22/33

Executing Elmer
make sure that the directory $ELMER of your OS serial either launch directly, with

HOME/bin is in the path

ElmerSolver mysolverinputfile.sif

or write create a le ELMERSOLVER where you launch from e.g.

STARTINFO in the directory

echo mysolverinputfile.sif > ELMERSOLVER STARTINFO ElmerSolver

launch simply with

parallel needs ELMERSOLVER distributed execution) launch with

STARTINFO plus evtl. a host-le (if

mpirun -np 4 --hostfile hostfilename ElmerSolver

CSC

Zwinger p.22/33

Tables and Arrays

Tables may be used for piecewise linear dependency of a variable

CSC

Zwinger p.23/33

Tables and Arrays

Tables may be used for piecewise linear dependency of a variable
Density Real 0 273 300 400 End 900 1000 1020 1000 = Variable Temperature

CSC

Zwinger p.23/33

Tables and Arrays

Tables may be used for piecewise linear dependency of a variable
Density Real 0 273 300 400 End 900 1000 1020 1000 = Variable Temperature

Arrays may be used to declare vector/tensor parameters

CSC

Zwinger p.23/33

Tables and Arrays

Tables may be used for piecewise linear dependency of a variable
Density Real 0 273 300 400 End 900 1000 1020 1000 = Variable Temperature

Arrays may be used to declare vector/tensor parameters

Target Boundaries(3) = 2 4 5 My Parameter Array(3,3) = Real 1 2 3 \ 4 5 6 \ 7 8 9

CSC

Zwinger p.23/33

MATC
library for the numerical evaluation of mathematical expressions

CSC

Zwinger p.24/33

MATC
library for the numerical evaluation of mathematical expressions dened in SIF for use in ElmerSolver

CSC

Zwinger p.24/33

MATC
library for the numerical evaluation of mathematical expressions dened in SIF for use in ElmerSolver or by ElmerPost as post-processing feature e.g. K C: math Celsius = Temperature +

273.16

CSC

Zwinger p.24/33

MATC
library for the numerical evaluation of mathematical expressions dened in SIF for use in ElmerSolver or by ElmerPost as post-processing feature e.g. K C: math Celsius = Temperature + very close to C-syntax

273.16

CSC

Zwinger p.24/33

MATC
library for the numerical evaluation of mathematical expressions dened in SIF for use in ElmerSolver or by ElmerPost as post-processing feature e.g. K C: math Celsius = Temperature + very close to C-syntax also logical evaluations (if) and loops (for)

273.16

CSC

Zwinger p.24/33

MATC
library for the numerical evaluation of mathematical expressions dened in SIF for use in ElmerSolver or by ElmerPost as post-processing feature e.g. K C: math Celsius = Temperature + very close to C-syntax also logical evaluations (if) and loops (for) documentation on Funet (MATC Manual)

273.16

CSC

Zwinger p.24/33

simple numerical evaluation:

MATC contd.
or

Viscosity Exponent = Real MATC "1.0/3.0" Viscosity Exponent = Real $1.0/3.0

CSC

Zwinger p.25/33

simple numerical evaluation:

MATC contd.
or

Viscosity Exponent = Real MATC "1.0/3.0" Viscosity Exponent = Real $1.0/3.0

as a function dependent on a variable:

Heat Capacity = Variable Temperature Real MATC "2.1275D03 + 7.253D00*(tx - 273.16)"

CSC

Zwinger p.25/33

simple numerical evaluation:

MATC contd.
or

Viscosity Exponent = Real MATC "1.0/3.0" Viscosity Exponent = Real $1.0/3.0

as a function dependent on a variable:

Heat Capacity = Variable Temperature Real MATC "2.1275D03 + 7.253D00*(tx - 273.16)"

as a function of multiple variables:

Temp = Variable Latitude, Coordinate 3 Real MATC "49.13 + 273.16 - 0.7576 * tx(0) - 7.992E-03 * tx(1)"

CSC

Zwinger p.25/33

simple numerical evaluation:

MATC contd.
or

Viscosity Exponent = Real MATC "1.0/3.0" Viscosity Exponent = Real $1.0/3.0

as a function dependent on a variable:

Heat Capacity = Variable Temperature Real MATC "2.1275D03 + 7.253D00*(tx - 273.16)"

as a function of multiple variables:

Temp = Variable Latitude, Coordinate 3 Real MATC "49.13 + 273.16 - 0.7576 * tx(0) - 7.992E-03 * tx(1)"

as function dened before header:

$ function stemp(X) { stemp = 49.13 + 273.16 - 0.7576*X(0) - 7.992E-03*X(1) } Temp = Variable Latitude, Coordinate 3 Real MATC "stemp(tx)"

CSC

Zwinger p.25/33

simple numerical evaluation:

MATC contd.
or

Viscosity Exponent = Real MATC "1.0/3.0" Viscosity Exponent = Real $1.0/3.0

as a function dependent on a variable:

Heat Capacity = Variable Temperature Real MATC "2.1275D03 + 7.253D00*(tx - 273.16)"

as a function of multiple variables:

Temp = Variable Latitude, Coordinate 3 Real MATC "49.13 + 273.16 - 0.7576 * tx(0) - 7.992E-03 * tx(1)"

as function dened before header:

$ function stemp(X) { stemp = 49.13 + 273.16 - 0.7576*X(0) - 7.992E-03*X(1) } Temp = Variable Latitude, Coordinate 3 Real MATC "stemp(tx)"

CSC

Zwinger p.25/33

User Dened Functions

Example:
(T ( C)) = 1000 [1 104 (T 273.0)]

CSC

Zwinger p.26/33

User Dened Functions

Example:
(T ( C)) = 1000 [1 104 (T 273.0)]
FUNCTION getdensity( Model, n, T ) RESULT(dens) USE DefUtils IMPLICIT None TYPE(Model t) :: INTEGER :: n T, dens Model

REAL(KIND=dp) ::

dens = 1000*(1-1.0d-4(T-273.0d0)) END FUNCTION getdensity

CSC

Zwinger p.26/33

User Dened Functions

Example:
(T ( C)) = 1000 [1 104 (T 273.0)]
FUNCTION getdensity( Model, n, T ) RESULT(dens) USE DefUtils IMPLICIT None TYPE(Model t) :: INTEGER :: n T, dens Model

REAL(KIND=dp) ::

dens = 1000*(1-1.0d-4(T-273.0d0)) END FUNCTION getdensity

compile:

elmerf90 mydensity.f90 -o mydensity

CSC

Zwinger p.26/33

User Dened Functions

Example:
(T ( C)) = 1000 [1 104 (T 273.0)]
FUNCTION getdensity( Model, n, T ) RESULT(dens) USE DefUtils IMPLICIT None TYPE(Model t) :: INTEGER :: n T, dens Model

REAL(KIND=dp) ::

dens = 1000*(1-1.0d-4(T-273.0d0)) END FUNCTION getdensity

compile: SIF:

elmerf90 mydensity.f90 -o mydensity

Density = Variable Temperature

Procedure "mydensity" "getdensity"

CSC

Zwinger p.26/33

User Dened Subroutines

RECURSIVE SUBROUTINE & mysolver( Model,Solver,dt,TransientSimulation ) TYPE(Model t) :: TYPE(Solver t) :: REAL(KIND=dp) :: LOGICAL :: ... assembly, solution ... END SUBROUTINE mysolver Model Solver dt

TransientSimulation

CSC

Zwinger p.27/33

User Dened Subroutines

RECURSIVE SUBROUTINE & mysolver( Model,Solver,dt,TransientSimulation ) TYPE(Model t) :: TYPE(Solver t) :: REAL(KIND=dp) :: LOGICAL :: ... assembly, solution ... END SUBROUTINE mysolver Model Solver dt TransientSimulation
pointer to the whole Model (solvers, variables) pointer to the particular solver current time step size

Model Solver dt

TransientSimulation

.TRUE. if transient simulation

CSC

Zwinger p.27/33

User Dened Subroutines

RECURSIVE SUBROUTINE & mysolver( Model,Solver,dt,TransientSimulation ) TYPE(Model t) :: TYPE(Solver t) :: REAL(KIND=dp) :: LOGICAL :: ... assembly, solution ... END SUBROUTINE mysolver Model Solver dt TransientSimulation
compile: pointer to the whole Model (solvers, variables) pointer to the particular solver current time step size

Model Solver dt

TransientSimulation

.TRUE. if transient simulation

elmerf90 mysolverfile.f90 -o mysolverexe

CSC

Zwinger p.27/33

User Dened Subroutines

RECURSIVE SUBROUTINE & mysolver( Model,Solver,dt,TransientSimulation ) TYPE(Model t) :: TYPE(Solver t) :: REAL(KIND=dp) :: LOGICAL :: ... assembly, solution ... END SUBROUTINE mysolver Model Solver dt TransientSimulation
compile: pointer to the whole Model (solvers, variables) pointer to the particular solver current time step size

Model Solver dt

TransientSimulation

.TRUE. if transient simulation

SIF:

elmerf90 mysolverfile.f90 -o mysolverexe

Procedure = "/path/to/mysolverexe" "mysolver"

CSC

Zwinger p.27/33

User Dened Subroutines contd.

ElmerSolver Main
Timestepping loop Steady state iteration (coupled system)

User Subroutine
Initialization Nonlinear iteration loop Domain element loop Matrix assembly for domain element until last bulk element Boundary element loop Matrix assembly for von Neumann and Newton conditions at boundary element until last boundary element set Dirichlet boundary conditions solve the system relative change of norms < Nonlinear Tolerance or nonlinear max. iterations exceeded relative change of norms < Steady State Tolerance until last timestep often provided as subroutine inside the solver routine often provided as subroutine inside the solver routine

CSC

Zwinger p.28/33

User Dened Subroutines contd.

ElmerSolver Main

Pre-dened routines
CALL DefaultInitialize() CALL DefaultUpdateEquations( STIFF, FORCE ) CALL DefaultFinishAssembly()

Timestepping loop Steady state iteration (coupled system)

User Subroutine
Initialization Nonlinear iteration loop Domain element loop Matrix assembly for domain element until last bulk element Boundary element loop Matrix assembly for von Neumann and Newton conditions at boundary element until last boundary element often provided as subroutine inside the solver routine often provided as subroutine inside the solver routine

CALL DefaultDirichletBCs() Norm = DefaultSolve()

set Dirichlet boundary conditions solve the system relative change of norms < Nonlinear Tolerance or nonlinear max. iterations exceeded relative change of norms < Steady State Tolerance until last timestep

CSC

Zwinger p.28/33

Multiple Meshes
In the Header, declare the global mesh database
Mesh DB "." "dirname"

CSC

Zwinger p.29/33

Multiple Meshes
In the Header, declare the global mesh database
Mesh DB "." "dirname"

In the Solver, declare the local mesh the solver is run on:
Mesh = File "/path/to/" "mesh"

CSC

Zwinger p.29/33

Multiple Meshes
In the Header, declare the global mesh database
Mesh DB "." "dirname"

In the Solver, declare the local mesh the solver is run on:
Mesh = File "/path/to/" "mesh"

variable values will be interpolated

CSC

Zwinger p.29/33

Multiple Meshes
In the Header, declare the global mesh database
Mesh DB "." "dirname"

In the Solver, declare the local mesh the solver is run on:
Mesh = File "/path/to/" "mesh"

variable values will be interpolated they will boldly be extrapolated, should your meshes not be congruent!

CSC

Zwinger p.29/33

Element Types
In section Equation:

CSC

Zwinger p.30/33

Element Types
In section Equation:
Element = [n:#dofs d:#dofs p:#dofs b:#dofs e:#dofs f:#dofs] n . . . nodal, d . . . DG element, p p-element, b . . . bubble, e . . . edge, f . . . face DOFs

CSC

Zwinger p.30/33

Element Types
In section Equation:
Element = [n:#dofs d:#dofs p:#dofs b:#dofs e:#dofs f:#dofs] n . . . nodal, d . . . DG element, p p-element, b . . . bubble, e . . . edge, f . . . face DOFs Element = [d:0]

. . . DG DOFs mesh element nodes

CSC

Zwinger p.30/33

Element Types
In section Equation:
Element = [n:#dofs d:#dofs p:#dofs b:#dofs e:#dofs f:#dofs] n . . . nodal, d . . . DG element, p p-element, b . . . bubble, e . . . edge, f . . . face DOFs Element = [d:0]

. . . DG DOFs mesh element nodes

Element = ...

If Equation applies to more than one solver, applies for all solver.

CSC

Zwinger p.30/33

Element Types
In section Equation:
Element = [n:#dofs d:#dofs p:#dofs b:#dofs e:#dofs f:#dofs] n . . . nodal, d . . . DG element, p p-element, b . . . bubble, e . . . edge, f . . . face DOFs Element = [d:0]

. . . DG DOFs mesh element nodes

Element = ...

If Equation applies to more than one solver, applies for all solver. selectively for each solver:
Element[1] = ... Element[2] = ... . . . Element[n] = ...

CSC

Zwinger p.30/33

Specialities
given names for components of vector elds:
Variable = var name[cname 1:#dofs cname 2:#dofs ... ]

CSC

Zwinger p.31/33

Specialities
given names for components of vector elds:
Variable = var name[cname 1:#dofs cname 2:#dofs ... ]

internal Solver can be run as external Procedures (enabling denition of variable names)
Procedure = "FlowSolve" "FlowSolver" Variable = Flow[Veloc:3 Pres:1]

CSC

Zwinger p.31/33

Specialities
given names for components of vector elds:
Variable = var name[cname 1:#dofs cname 2:#dofs ... ]

internal Solver can be run as external Procedures (enabling denition of variable names)
Procedure = "FlowSolve" "FlowSolver" Variable = Flow[Veloc:3 Pres:1]

Residuals of solver variables (e.g., Navier Stokes):

Procedure = "FlowSolve" "FlowSolver" Variable = Flow[Veloc:3 Pres:1] Exported Variable 1 = Flow Loads[Stress Vector:3 CEQ Residual:1]

CSC

Zwinger p.31/33

Specialities
given names for components of vector elds:
Variable = var name[cname 1:#dofs cname 2:#dofs ... ]

internal Solver can be run as external Procedures (enabling denition of variable names)
Procedure = "FlowSolve" "FlowSolver" Variable = Flow[Veloc:3 Pres:1]

Residuals of solver variables (e.g., Navier Stokes):

Procedure = "FlowSolve" "FlowSolver" Variable = Flow[Veloc:3 Pres:1] Exported Variable 1 = Flow Loads[Stress Vector:3 CEQ Residual:1]

Solver execution:
Exec Solver = {Before Simulation, After Simulation, Never, Always}
CSC

Zwinger p.31/33

Elmer Parallel Version

Pre-processing:
Partition by direction:

ElmerGrid with options:

First partition elements (default) First partition nodes

-partition 2 2 1 0 -partition 2 2 1 1
22=4 Partition using METIS:

-metis n 0 -metis n 1 -metis n 2 -metis n 3

PartMeshNodal (default) PartGraphRecursive PartGraphKway PartGraphVKway

CSC

Zwinger p.32/33

Elmer Parallel Version

Pre-processing:
Partition by direction:

ElmerGrid with options:

First partition elements (default) First partition nodes

-partition 2 2 1 0 -partition 2 2 1 1
22=4 Partition using METIS:

-metis n 0 -metis n 1 -metis n 2 -metis n 3

PartMeshNodal (default) PartGraphRecursive PartGraphKway PartGraphVKway

Execution:

mpirun -np n ElmerSolver mpi

CSC

Zwinger p.32/33

Elmer Parallel Version

Pre-processing:
Partition by direction:

ElmerGrid with options:

First partition elements (default) First partition nodes

-partition 2 2 1 0 -partition 2 2 1 1
22=4 Partition using METIS:

-metis n 0 -metis n 1 -metis n 2 -metis n 3

PartMeshNodal (default) PartGraphRecursive PartGraphKway PartGraphVKway

Execution:

mpirun -np n ElmerSolver mpi

Combining parallel results: in mesh-database directory

ElmerGrid 15 3 name
CSC

Zwinger p.32/33

Elmer Parallel Version contd.

need iterative method for linear solver!

CSC

Zwinger p.33/33

Elmer Parallel Version contd.

need iterative method for linear solver! standard Krylov subspace in domain decomposition shows different behaviour!
Domain 1

Domain 2

Domain 3

CSC

Zwinger p.33/33

Elmer Parallel Version contd.

need iterative method for linear solver! standard Krylov subspace in domain decomposition shows different behaviour!
Domain 1

Domain 2

Domain 3

Linear System Use Hypre = Logical True

CSC

Zwinger p.33/33

Elmer Parallel Version contd.

need iterative method for linear solver! standard Krylov subspace in domain decomposition shows different behaviour!
Domain 1

Domain 2

Domain 3

Linear System Use Hypre = Logical True Linear System Preconditioning = ParaSails

CSC

Zwinger p.33/33

Elmer Parallel Version contd.

need iterative method for linear solver! standard Krylov subspace in domain decomposition shows different behaviour!
Domain 1

Domain 2

Domain 3

Linear System Use Hypre = Logical True Linear System Preconditioning = ParaSails ParaSails Threshold, ParaSails Filter, ParaSails Maxlevel, ParaSails Symmetry
CSC

Zwinger p.33/33