Institute for Computational Mathematics

JANUS BLOCK ILU Guide

http://bilu.tu-bs.de
Matthias Bollhöfer, May 28, 2021
Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 2

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 3

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 4

Preconditioning Systems
Objective
Given a large sparse nonsingular matrix A and a linear system

Ax = b,

1. construct approximate factorization A ≈ Ã = LDU

2. solve Ax = b using a preconditioned Krylov subspace iteration method
How large, how sparse, and why using an approximate
factorization?
system size n = 105 → 109 , number of nonzeros
typically less 100n
memory requirements, computation time

M. Bollhöfer JANUS Guide Page 5

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 6

Getting started with C

downlowad JANUS at https://github.com/mbollhoefer/JANUS

before you can use JANUS BLOCK ILU, you are asked to download and
install external packages
Install the approximate mininum degree (AMD) [Amestoy, Davis and Duff 1996]
ordering from the SuiteSparse and create a library libamd.a
Install the MT-METIS package [LaSalle and Karypis 2013]
Install MC64 from the HSL Mathematical Software Library, e.g. adding the
object files to libjanus.a or somewhere else
make sure to have efficient versions of BLAS and LAPACK installed, e.g., the
MKL library would be helpful but it is not mandatory
After that you are ready to use the JANUS library. You will find in the
subdirectory janus/samples a makefile along with a sample C-code
janusdriver.c which explains the usage of the JANUS BLOCK ILU
preconditioning package.

M. Bollhöfer JANUS Guide Page 7

Getting started with C — a sample code
# include < j a n u s . h>
...
/ / major JANUS v a r i a b l e s
S p ar s e Ma t r i x A ;
JanusOptions o p t i o n s ;
JanusPrec PREC;
...
/ / t a g your m a t r i x , e . g . , as r e a l , nonsymmetric
A . i s r e a l =1;A . i s d e f i n i t e =0;A . i s s y m m e t r i c =0;A . i s h e r m i t i a n =0;
J a n u s D e f a u l t O p t i o n s (& o p t i o n s ) ; / / use d e f a u l t o p t i o n s
/ / compute BLOCK ILU
i e r r =JanusFactor (&A , &PREC, o p t i o n s ) ;
/ / i t e r a t i v e solver
i e r r =JanusSolver (&A,&PREC, rhs , s o l , 3 0 , 1 e − 6 ,1000 ,& i t e r ) ;
JanusDelete (&PREC ) ; / / r e l e a s e p r e c o n d i t i o n e r

M. Bollhöfer JANUS Guide Page 8

Parameter setting

JANUS offers several parameters

Some Default Parameters
options
.matching: 1 | 0
.ordering: PERM_MTMETIS | PERM_NONE|PERM_AMD|PERM_RCM
.droptol: 1.0e-03 | ...
.cosine: 0 | 1
.blocking_strategy: BLOCK_ILUPT | BLOCK_NONE|BLOCK_SUPERNODES
.progressive_aggregation: 1 | 0
.symmetric_structure: 0 | 1
.invert_blocks: 1 | 0

M. Bollhöfer JANUS Guide Page 9

Parameter Setting
Most of the parameters you will find familiar
matching improve diagonal dominance using maximum
weighted matchings
ordering preprocess the system by a symbolic reordering
(e.g. PERM_AMD for ’Approximate Minimum
Degree’)
droptol threshold to drop small entries during the facto-
rization
cosine cosine-based strategy to build blocks in the ini-
tial matrix
blocking_strategy heuristic approach to improve the given scalar
or block structure preparing the BLOCK ILU
progressive_aggregation aggregate blocks during the factorization to ob-
tain even larger blocks
symmetric_structure only use symmetric permutations
invert_blocks compute LD −1 U or LDU
M. Bollhöfer JANUS Guide Page 10
How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol

M. Bollhöfer JANUS Guide Page 11

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol

You prefer to use JANUS inside your own Krylov subspace method
⇒ use JanusSol(&PREC, rhs,sol, buff,m), JanusSolT(&PREC,
rhs,sol, buff,m), JanusSolH(&PREC, rhs,sol, buff,m) instead for
solving m right hand sides with JANUS, its transpose or its conjugate
transpose. buff could either be set to NULL or you provide n*m spaces.

M. Bollhöfer JANUS Guide Page 11

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol

You prefer to use JANUS inside your own Krylov subspace method
⇒ use JanusSol(&PREC, rhs,sol, buff,m), JanusSolT(&PREC,
rhs,sol, buff,m), JanusSolH(&PREC, rhs,sol, buff,m) instead for
solving m right hand sides with JANUS, its transpose or its conjugate
transpose. buff could either be set to NULL or you provide n*m spaces.

You are interested in some statistics concerning JANUS

⇒ JanusNnz(&PREC, &nnz, &mxblock, &avblock, &stddev) tells you
about the nonzeros (nnz), the size of the largest diagonal block (mxblock),
the average block size (avblock) and their standard deviation (stddev)

M. Bollhöfer JANUS Guide Page 11

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol

You prefer to use JANUS inside your own Krylov subspace method
⇒ use JanusSol(&PREC, rhs,sol, buff,m), JanusSolT(&PREC,
rhs,sol, buff,m), JanusSolH(&PREC, rhs,sol, buff,m) instead for
solving m right hand sides with JANUS, its transpose or its conjugate
transpose. buff could either be set to NULL or you provide n*m spaces.

You are interested in some statistics concerning JANUS

⇒ JanusNnz(&PREC, &nnz, &mxblock, &avblock, &stddev) tells you
about the nonzeros (nnz), the size of the largest diagonal block (mxblock),
the average block size (avblock) and their standard deviation (stddev)

You would like to handle complex–valued or symmetric matrices

⇒ pass your data and tag ther matrix suitably, (e.g. A.isreal=0 or
A.issymmetric=1)
M. Bollhöfer JANUS Guide Page 11
Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 12

Getting started with MATLAB

After adding JANUS system path (e.g.  addpath ’janus’) a large sparse
system Ax = b could be solved as follows:

Approximate Factorization
 [PREC,options]=janus(A);

M. Bollhöfer JANUS Guide Page 13

Getting started with MATLAB

After adding JANUS system path (e.g.  addpath ’janus’) a large sparse
system Ax = b could be solved as follows:

Approximate Factorization
 [PREC,options]=janus(A);
⇒ preconditioner is built using the default options.

M. Bollhöfer JANUS Guide Page 13

Getting started with MATLAB

After adding JANUS system path (e.g.  addpath ’janus’) a large sparse
system Ax = b could be solved as follows:

Approximate Factorization
 [PREC,options]=janus(A);
⇒ preconditioner is built using the default options.

Iterative Solution
 x=janussolver(A,b,30,1e-6,1000,PREC);

M. Bollhöfer JANUS Guide Page 13

Getting started with MATLAB

After adding JANUS system path (e.g.  addpath ’janus’) a large sparse
system Ax = b could be solved as follows:

Approximate Factorization
 [PREC,options]=janus(A);
⇒ preconditioner is built using the default options.

Iterative Solution
 x=janussolver(A,b,30,1e-6,1000,PREC);
⇒ system is solved.

M. Bollhöfer JANUS Guide Page 13

Parameter setting

JANUS offers several parameters

Some Default Parameters
 [PREC,options]=janus(A); options
options =
matching:1
ordering:’mtmetis’
droptol:1.0000e-03
perturbation:1
symmetric_structure:0
cosine:0
blocking_strategy:’ilupt’
progressive_aggregation:1
invert_blocks:1

M. Bollhöfer JANUS Guide Page 14

Parameter Setting
isdefinite indicate that your matrix is positive definite (only
used if the matrix is Hermitian)
matching improve diagonal dominance using maximum
weighted matchings
ordering symbolic reordering (e.g. ’amd’ for ’Approximate
Minimum Degree’)
droptol threshold to drop small entries
cosine cosine-based strategy to build blocks
blocking_strategy heuristic approach to improve the given scalar
or block structure preparing the BLOCK ILU
(e.g. ’ilupt’ or ’supernodes’)
progressive_aggregation aggregate blocks during the factorization to ob-
tain even larger blocks
symmetric_structure only use symmetric permutations
invert_blocks compute LD −1 U or LDU

M. Bollhöfer JANUS Guide Page 15

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol, call [PREC,options]=janus(A,options);

M. Bollhöfer JANUS Guide Page 16

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol, call [PREC,options]=janus(A,options);

You prefer to use JANUS inside your own Krylov subspace method
⇒ use sol=janussol(PREC,rhs); or sol=janussolt(PREC,rhs); or
sol=janussolh(PREC,rhs) instead for solving with JANUS, its transpose
or its conjugate transpose.

M. Bollhöfer JANUS Guide Page 16

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol, call [PREC,options]=janus(A,options);

You prefer to use JANUS inside your own Krylov subspace method
⇒ use sol=janussol(PREC,rhs); or sol=janussolt(PREC,rhs); or
sol=janussolh(PREC,rhs) instead for solving with JANUS, its transpose
or its conjugate transpose.

You are interested in some statistics concerning JANUS

⇒ nz=janusnnz(PREC) tells you about the nonzeros. Other information
can be directly accessed from PREC.

M. Bollhöfer JANUS Guide Page 16

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol, call [PREC,options]=janus(A,options);

You prefer to use JANUS inside your own Krylov subspace method
⇒ use sol=janussol(PREC,rhs); or sol=janussolt(PREC,rhs); or
sol=janussolh(PREC,rhs) instead for solving with JANUS, its transpose
or its conjugate transpose.

You are interested in some statistics concerning JANUS

⇒ nz=janusnnz(PREC) tells you about the nonzeros. Other information
can be directly accessed from PREC.

You would like to handle complex–valued or symmetric matrices

⇒ janus will select the correct driver on its own!

M. Bollhöfer JANUS Guide Page 16

How to set up your own parameters
Your iterative solver does not converge
⇒ reduce options.droptol, call [PREC,options]=janus(A,options);

You prefer to use JANUS inside your own Krylov subspace method
⇒ use sol=janussol(PREC,rhs); or sol=janussolt(PREC,rhs); or
sol=janussolh(PREC,rhs) instead for solving with JANUS, its transpose
or its conjugate transpose.

You are interested in some statistics concerning JANUS

⇒ nz=janusnnz(PREC) tells you about the nonzeros. Other information
can be directly accessed from PREC.

You would like to handle complex–valued or symmetric matrices

⇒ janus will select the correct driver on its own!

You would like to display the BLOCK ILU

⇒ janusspy(PREC)
M. Bollhöfer JANUS Guide Page 16
Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 17

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 18

Matchings
Maximum weighted matchings are a combinatorial graph theoretical
approach to improve diagonal dominance

Using matchings a general system A is

1. rescaled
2. permuted
such that A → ΠT Dr ADc satisfies |aijnew | 6 1, |aiinew | = 1.

Matchings can be symmetrized for systems satisfying |A| = |AT | with

similar properties
0 0

50 50

100 100

150 150

200 200

250
−→ 250

300 300

350 350

400 400

450 450

0 50 100 150 200 250 300 350 400 450 0 50 100 150 200 250 300 350 400 450
nz = 1887 nz = 1887

M. Bollhöfer JANUS Guide Page 19

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 20

Symmetric reorderings
’amd’ Approximate Minimum Degree ’rcm’ Reverse Cuthill-McKee

’mtmetis’ multithreaded Metis

M. Bollhöfer JANUS Guide Page 21

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 22

BLOCK ILU
Factorize A ≈ LDU or LD −1 U using several blocking strategies to
compute a BLOCK ILU with blocks of bigger size
block structures allow to exploit dense matrix kernels

M. Bollhöfer JANUS Guide Page 23

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 24

Automatic structure detection
JANUS offers many special purpose drivers for
complex systems
general sparse (with GMRES)
complex symmetric (with SQMR)
complex Hermitian (with SQMR)
complex Hermitian positive definite (with CG)
real systems
general sparse (with GMRES)
real symmetric (with SQMR)
real symmetric positive definite (with CG)
JANUS automatically detects
real/complex systems
symmetry structures

M. Bollhöfer JANUS Guide Page 25

Automatic structure detection

JANUS asks YOU to specify if the system is positive definite

SPD Case — C-Code

A.isdefinite=1;
ierr=JanusFactor(&A,&PREC,options);

SPD Case — MATLAB-Code

 options.isdefinite=1;
 [PREC,options]=janus(A,options);
if no other options are set, the default options are used

M. Bollhöfer JANUS Guide Page 26

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 27

Additional features

JANUS will always compute the main branch of log(det A) (PREC.logdet)

JANUS offers drivers for approximate matrix inversion via Neumann series
based on the given symmetric block LDLT
(DSYMbfspai,ZHERbfspai,ZSYMbfspai for C code, symbfspai for the
MATLAB interface)
JANUS offers drivers for selected matrix inversion series based on the
given symmetric block LDLT (DSYMselbinv,ZHERselbinv,ZSYMselbinv
for C code, symselbinv for the MATLAB interface)

M. Bollhöfer JANUS Guide Page 28

Outline

Introduction — Using JANUS BLOCK ILU

Preconditioning Systems
Getting started with C
Getting started with MATLAB
What’s behind the toolbox
Matchings
Symmetric reorderings
BLOCK ILU
Automatic structure detection
Additional features
Closing Remarks

M. Bollhöfer JANUS Guide Page 29

Closing Remarks
Watch the JANUS website at
http://bilu.tu-bs.de
You may download the sources at
https://github.com/mbollhoefer/JANUS
Matthias Bollhöfer, Olaf Schenk, Fabio Verbosio. A High Performance
Level-Block Approximate LU Factorization Preconditioner Algorithm. Applied
Numerical Mathematics 162:265-282, 2021.
DOI:10.1016/j.apnum.2020.12.023

M. Bollhöfer JANUS Guide Page 30