If computational biology were to be a feasible approach to anything but the

simplest of problems, a first step would involve determining what approaches

are ontologically appropriate.
The following is a brief list of approaches used in computational mathematics
specifically.

Iterative method
Rate of convergence — the speed at which a convergent sequence
approaches its limit
Order of accuracy — rate at which numerical solution of differential equation
converges to exact solution
Series acceleration — methods to accelerate the speed of convergence of a
series
Aitken's delta-squared process — most useful for linearly converging
sequences
Minimum polynomial extrapolation — for vector sequences
Richardson extrapolation
Shanks transformation — similar to Aitken's delta-squared process, but
applied to the partial sums
Van Wijngaarden transformation — for accelerating the convergence of an
alternating series
Abramowitz and Stegun — book containing formulas and tables of many
special functions
Digital Library of Mathematical Functions — successor of book by
Abramowitz and Stegun
Curse of dimensionality
Local convergence and global convergence — whether you need a good
initial guess to get convergence
Superconvergence
Discretization
Difference quotient

Computational complexity of mathematical operations

Smoothed analysis — measuring the expected performance of algorithms
under slight random perturbations of worst-case inputs
Symbolic-numeric computation — combination of symbolic and numeric
methods

Lattice QCD and Numerical Analysis

Collocation method — discretizes a continuous equation by requiring it only

to hold at certain points
Level set method
Level set (data structures) — data structures for representing level sets
Sinc numerical methods — methods based on the sinc function, sinc(x) =
sin(x) / x
ABS methods
Error[edit]
Error analysis (mathematics)
Approximation
Approximation error
Condition number
Discretization error
Floating point number
Guard digit — extra precision introduced during a computation to reduce
round-off error
Truncation — rounding a floating-point number by discarding all digits after
a certain digit
Round-off error
Numeric precision in Microsoft Excel
Arbitrary-precision arithmetic
Interval arithmetic — represent every number by two floating-point numbers
guaranteed to have the unknown number between them
Interval contractor — maps interval to subinterval which still contains the
unknown exact answer
Interval propagation — contracting interval domains without removing any
value consistent with the constraints

Loss of significance
Numerical error
Numerical stability
Error propagation:
Propagation of uncertainty
Significance arithmetic
Residual (numerical analysis)
Relative change and difference — the relative difference between x and y is |
x − y| / max(|x|, |y|)
Significant figures
False precision — giving more significant figures than appropriate
Truncation error — error committed by doing only a finite numbers of steps

Affine arithmetic
Elementary and special functions[edit]
Summation:
Kahan summation algorithm
Pairwise summation — slightly worse than Kahan summation but cheaper
Binary splitting
Multiplication:
Multiplication algorithm — general discussion, simple methods
Karatsuba algorithm — the first algorithm which is faster than
straightforward multiplication
Toom–Cook multiplication — generalization of Karatsuba multiplication
Schönhage–Strassen algorithm — based on Fourier transform, asymptotically
very fast
Fürer's algorithm — asymptotically slightly faster than Schönhage–Strassen
Division algorithm — for computing quotient and/or remainder of two
numbers
Long division
Restoring division
Non-restoring division
SRT division
Newton–Raphson division: uses Newton's method to find the reciprocal of D,
and multiply that reciprocal by N to find the final quotient Q.
Goldschmidt division
Exponentiation:
Exponentiation by squaring
Addition-chain exponentiation
Multiplicative inverse Algorithms: for computing a number's multiplicative
inverse (reciprocal).
Newton's method
Polynomials:

Horner's method
Estrin's scheme — modification of the Horner scheme with more possibilities
for parallelization
Clenshaw algorithm
De Casteljau's algorithm
Square roots and other roots:
Integer square root
nth root algorithm
Shifting nth root algorithm — similar to long division
hypot — the function (x2 + y2)1/2
Alpha max plus beta min algorithm — approximates hypot(x,y)
Fast inverse square root — calculates 1 / √x using details of the IEEE
floating-point system
Elementary functions (exponential, logarithm, trigonometric functions):
Trigonometric tables — different methods for generating them
CORDIC — shift-and-add algorithm using a table of arc tangents
BKM algorithm — shift-and-add algorithm using a table of logarithms and
complex numbers
Gamma function:
Lanczos approximation
Spouge's approximation — modification of Stirling's approximation; easier to
apply than Lanczos
AGM method — computes arithmetic–geometric mean; related methods
compute special functions
FEE method (Fast E-function Evaluation) — fast summation of series like the
power series for ex
Gal's accurate tables — table of function values with unequal spacing to
reduce round-off error
Spigot algorithm — algorithms that can compute individual digits of a real
number
Approximations of π:
Liu Hui's π algorithm — first algorithm that can compute π to arbitrary
precision
Leibniz formula for π — alternating series with very slow convergence
Wallis product — infinite product converging slowly to π/2
Viète's formula — more complicated infinite product which converges faster
Gauss–Legendre algorithm — iteration which converges quadratically to π,
based on arithmetic–geometric mean
Borwein's algorithm — iteration which converges quartically to 1/π, and
other algorithms
Chudnovsky algorithm — fast algorithm that calculates a hypergeometric
series
Bailey–Borwein–Plouffe formula — can be used to compute individual
hexadecimal digits of π
Bellard's formula — faster version of Bailey–Borwein–Plouffe formula

Numerical linear algebra[edit]

Numerical linear algebra — study of numerical algorithms for linear algebra
problems

Types of matrices appearing in numerical analysis:

Sparse matrix
Band matrix
Bidiagonal matrix
Tridiagonal matrix
Pentadiagonal matrix
Skyline matrix
Circulant matrix
Triangular matrix
Diagonally dominant matrix
Block matrix — matrix composed of smaller matrices
Stieltjes matrix — symmetric positive definite with non-positive off-diagonal
entries
Hilbert matrix — example of a matrix which is extremely ill-conditioned (and
thus difficult to handle)
Wilkinson matrix — example of a symmetric tridiagonal matrix with pairs of
nearly, but not exactly, equal eigenvalues
Convergent matrix – square matrix whose successive powers approach the
zero matrix
Algorithms for matrix multiplication:
Strassen algorithm
Coppersmith–Winograd algorithm
Cannon's algorithm — a distributed algorithm, especially suitable for
processors laid out in a 2d grid
Freivalds' algorithm — a randomized algorithm for checking the result of a
multiplication
Matrix decompositions:
LU decomposition — lower triangular times upper triangular
QR decomposition — orthogonal matrix times triangular matrix
RRQR factorization — rank-revealing QR factorization, can be used to
compute rank of a matrix
Polar decomposition — unitary matrix times positive-semidefinite Hermitian
matrix
Decompositions by similarity:
Eigendecomposition — decomposition in terms of eigenvectors and
eigenvalues
Jordan normal form — bidiagonal matrix of a certain form; generalizes the
eigendecomposition
Weyr canonical form — permutation of Jordan normal form
Jordan–Chevalley decomposition — sum of commuting nilpotent matrix and
diagonalizable matrix
Schur decomposition — similarity transform bringing the matrix to a
triangular matrix
Singular value decomposition — unitary matrix times diagonal matrix times
unitary matrix
Matrix splitting – expressing a given matrix as a sum or difference of
matrices

Gaussian elimination
Row echelon form — matrix in which all entries below a nonzero entry are
zero
Bareiss algorithm — variant which ensures that all entries remain integers if
the initial matrix has integer entries
Tridiagonal matrix algorithm — simplified form of Gaussian elimination for
tridiagonal matrices
LU decomposition — write a matrix as a product of an upper- and a lower-
triangular matrix
Crout matrix decomposition
LU reduction — a special parallelized version of a LU decomposition
algorithm
Block LU decomposition
Cholesky decomposition — for solving a system with a positive definite
matrix
Minimum degree algorithm
Symbolic Cholesky decomposition
Iterative refinement — procedure to turn an inaccurate solution in a more
accurate one
Direct methods for sparse matrices:
Frontal solver — used in finite element methods
Nested dissection — for symmetric matrices, based on graph partitioning
Levinson recursion — for Toeplitz matrices
SPIKE algorithm — hybrid parallel solver for narrow-banded matrices
Cyclic reduction — eliminate even or odd rows or columns, repeat

Iterative methods:

Jacobi method
Gauss–Seidel method
Successive over-relaxation (SOR) — a technique to accelerate the Gauss–
Seidel method
Symmetric successive overrelaxation (SSOR) — variant of SOR for symmetric
matrices
Backfitting algorithm — iterative procedure used to fit a generalized additive
model, often equivalent to Gauss–Seidel
Modified Richardson iteration
Conjugate gradient method (CG) — assumes that the matrix is positive
definite
Derivation of the conjugate gradient method
Nonlinear conjugate gradient method — generalization for nonlinear
optimization problems
Biconjugate gradient method (BiCG)
Biconjugate gradient stabilized method (BiCGSTAB) — variant of BiCG with
better convergence
Conjugate residual method — similar to CG but only assumed that the matrix
is symmetric
Generalized minimal residual method (GMRES) — based on the Arnoldi
iteration
Chebyshev iteration — avoids inner products but needs bounds on the
spectrum
Stone's method (SIP – Srongly Implicit Procedure) — uses an incomplete LU
decomposition
Kaczmarz method
Preconditioner
Incomplete Cholesky factorization — sparse approximation to the Cholesky
factorization
Incomplete LU factorization — sparse approximation to the LU factorization
Uzawa iteration — for saddle node problems
Underdetermined and overdetermined systems (systems that have no or
more than one solution):
Numerical computation of null space — find all solutions of an
underdetermined system
Moore–Penrose pseudoinverse — for finding solution with smallest 2-norm
(for underdetermined systems) or smallest residual
Sparse approximation — for finding the sparsest solution (i.e., the solution
with as many zeros as possible)

Eigenvalue algorithm — a numerical algorithm for locating the eigenvalues of

a matrix
Power iteration
Inverse iteration
Rayleigh quotient iteration
Arnoldi iteration — based on Krylov subspaces
Lanczos algorithm — Arnoldi, specialized for positive-definite matrices
Block Lanczos algorithm — for when matrix is over a finite field
QR algorithm
Jacobi eigenvalue algorithm — select a small submatrix which can be
diagonalized exactly, and repeat
Jacobi rotation — the building block, almost a Givens rotation
Jacobi method for complex Hermitian matrices
Divide-and-conquer eigenvalue algorithm
Folded spectrum method
LOBPCG — Locally Optimal Block Preconditioned Conjugate Gradient
Method
Eigenvalue perturbation — stability of eigenvalues under perturbations of the
matrix
Other concepts and algorithms[edit]
Orthogonalization algorithms:
Gram–Schmidt process
Householder transformation
Householder operator — analogue of Householder transformation for general
inner product spaces
Givens rotation
Krylov subspace
Block matrix pseudoinverse
Bidiagonalization
Cuthill–McKee algorithm — permutes rows/columns in sparse matrix to yield
a narrow band matrix
In-place matrix transposition — computing the transpose of a matrix without
using much additional storage
Pivot element — entry in a matrix on which the algorithm concentrates
Matrix-free methods — methods that only access the matrix by evaluating
matrix-vector products
Interpolation and approximation[edit]
Interpolation — construct a function going through some given data points
Nearest-neighbor interpolation — takes the value of the nearest neighbor
Polynomial interpolation[edit]
Polynomial interpolation — interpolation by polynomials
Linear interpolation
Runge's phenomenon
Vandermonde matrix
Chebyshev polynomials
Chebyshev nodes
Lebesgue constant (interpolation)
Different forms for the interpolant:
Newton polynomial
Divided differences
Neville's algorithm — for evaluating the interpolant; based on the Newton
form
Lagrange polynomial
Bernstein polynomial — especially useful for approximation
Brahmagupta's interpolation formula — seventh-century formula for
quadratic interpolation

Bilinear interpolation
Trilinear interpolation
Bicubic interpolation
Tricubic interpolation
Padua points — set of points in R2 with unique polynomial interpolant and
minimal growth of Lebesgue constant
Hermite interpolation
Birkhoff interpolation
Abel–Goncharov interpolation

Spline interpolation — interpolation by piecewise polynomials

Spline (mathematics) — the piecewise polynomials used as interpolants
Perfect spline — polynomial spline of degree m whose mth derivate is ±1
Cubic Hermite spline
Centripetal Catmull–Rom spline — special case of cubic Hermite splines
without self-intersections or cusps
Monotone cubic interpolation
Hermite spline
Bézier curve
De Casteljau's algorithm
composite Bézier curve
Generalizations to more dimensions:
Bézier triangle — maps a triangle to R3
Bézier surface — maps a square to R3
B-spline
Box spline — multivariate generalization of B-splines
Truncated power function
De Boor's algorithm — generalizes De Casteljau's algorithm
Non-uniform rational B-spline (NURBS)
T-spline — can be thought of as a NURBS surface for which a row of control
points is allowed to terminate
Kochanek–Bartels spline
Coons patch — type of manifold parametrization used to smoothly join other
surfaces together
M-spline — a non-negative spline
I-spline — a monotone spline, defined in terms of M-splines
Smoothing spline — a spline fitted smoothly to noisy data
Blossom (functional) — a unique, affine, symmetric map associated to a
polynomial or spline
See also: List of numerical computational geometry topics

Trigonometric interpolation — interpolation by trigonometric polynomials

Discrete Fourier transform — can be viewed as trigonometric interpolation at
equidistant points
Relations between Fourier transforms and Fourier series
Fast Fourier transform (FFT) — a fast method for computing the discrete
Fourier transform
Bluestein's FFT algorithm
Bruun's FFT algorithm
Cooley–Tukey FFT algorithm
Split-radix FFT algorithm — variant of Cooley–Tukey that uses a blend of
radices 2 and 4
Goertzel algorithm
Prime-factor FFT algorithm
Rader's FFT algorithm
Bit-reversal permutation — particular permutation of vectors with 2m entries
used in many FFTs.
Butterfly diagram
Twiddle factor — the trigonometric constant coefficients that are multiplied
by the data
Cyclotomic fast Fourier transform — for FFT over finite fields
Methods for computing discrete convolutions with finite impulse response
filters using the FFT:
Overlap–add method
Overlap–save method
Sigma approximation
Dirichlet kernel — convolving any function with the Dirichlet kernel yields its
trigonometric interpolant
Gibbs phenomenon
Other interpolants[edit]
Simple rational approximation
Polynomial and rational function modeling — comparison of polynomial and
rational interpolation
Wavelet
Continuous wavelet
Transfer matrix
See also: List of functional analysis topics, List of wavelet-related transforms
Inverse distance weighting
Radial basis function (RBF) — a function of the form ƒ(x) = φ(|x−x0|)
Polyharmonic spline — a commonly used radial basis function
Thin plate spline — a specific polyharmonic spline: r2 log r
Hierarchical RBF
Subdivision surface — constructed by recursively subdividing a piecewise
linear interpolant
Catmull–Clark subdivision surface
Doo–Sabin subdivision surface
Loop subdivision surface
Slerp (spherical linear interpolation) — interpolation between two points on a
sphere
Generalized quaternion interpolation — generalizes slerp for interpolation
between more than two quaternions
Irrational base discrete weighted transform
Nevanlinna–Pick interpolation — interpolation by analytic functions in the
unit disc subject to a bound
Pick matrix — the Nevanlinna–Pick interpolation has a solution if this matrix
is positive semi-definite
Multivariate interpolation — the function being interpolated depends on
more than one variable
Barnes interpolation — method for two-dimensional functions using
Gaussians common in meteorology
Coons surface — combination of linear interpolation and bilinear
interpolation
Lanczos resampling — based on convolution with a sinc function
Natural neighbor interpolation
Nearest neighbor value interpolation
PDE surface
Transfinite interpolation — constructs function on planar domain given its
values on the boundary
Trend surface analysis — based on low-order polynomials of spatial
coordinates; uses scattered observations

Approximation theory:

Orders of approximation
Lebesgue's lemma
Curve fitting
Vector field reconstruction
Modulus of continuity — measures smoothness of a function
Least squares (function approximation) — minimizes the error in the L2-norm
Minimax approximation algorithm — minimizes the maximum error over an
interval (the L∞-norm)
Equioscillation theorem — characterizes the best approximation in the L∞-
norm
Unisolvent point set — function from given function space is determined
uniquely by values on such a set of points
Stone–Weierstrass theorem — continuous functions can be approximated
uniformly by polynomials, or certain other function spaces

Approximation by polynomials:
Linear approximation
Bernstein polynomial — basis of polynomials useful for approximating a
function
Bernstein's constant — error when approximating |x| by a polynomial
Remez algorithm — for constructing the best polynomial approximation in
the L∞-norm
Bernstein's inequality (mathematical analysis) — bound on maximum of
derivative of polynomial in unit disk
Mergelyan's theorem — generalization of Stone–Weierstrass theorem for
polynomials
Müntz–Szász theorem — variant of Stone–Weierstrass theorem for
polynomials if some coefficients have to be zero
Bramble–Hilbert lemma — upper bound on Lp error of polynomial
approximation in multiple dimensions
Discrete Chebyshev polynomials — polynomials orthogonal with respect to a
discrete measure
Favard's theorem — polynomials satisfying suitable 3-term recurrence
relations are orthogonal polynomials
Approximation by Fourier series / trigonometric polynomials:
Jackson's inequality — upper bound for best approximation by a
trigonometric polynomial
Bernstein's theorem (approximation theory) — a converse to Jackson's
inequality
Fejér's theorem — Cesàro means of partial sums of Fourier series converge
uniformly for continuous periodic functions
Erdős–Turán inequality — bounds distance between probability and
Lebesgue measure in terms of Fourier coefficients
Different approximations:
Moving least squares
Padé approximant
Padé table — table of Padé approximants
Hartogs–Rosenthal theorem — continuous functions can be approximated
uniformly by rational functions on a set of Lebesgue measure zero
Szász–Mirakyan operator — approximation by e−n xk on a semi-infinite
interval
Szász–Mirakjan–Kantorovich operator
Baskakov operator — generalize Bernstein polynomials, Szász–Mirakyan
operators, and Lupas operators
Favard operator — approximation by sums of Gaussians
Surrogate model — application: replacing a function that is hard to evaluate
by a simpler function
Constructive function theory — field that studies connection between degree
of approximation and smoothness
Universal differential equation — differential–algebraic equation whose
solutions can approximate any continuous function
Fekete problem — find N points on a sphere that minimize some kind of
energy
Carleman's condition — condition guaranteeing that a measure is uniquely
determined by its moments
Krein's condition — condition that exponential sums are dense in weighted
L2 space
Lethargy theorem — about distance of points in a metric space from
members of a sequence of subspaces
Wirtinger's representation and projection theorem

Extrapolation:
Linear predictive analysis — linear extrapolation
Unisolvent functions — functions for which the interpolation problem has a
unique solution
Regression analysis
Isotonic regression
Curve-fitting compaction
Interpolation (computer graphics)
Finding roots of nonlinear equations[edit]
See #Numerical linear algebra for linear equations
Root-finding algorithm — algorithms for solving the equation f(x) = 0

Bisection method — simple and robust; linear convergence

Lehmer–Schur algorithm — variant for complex functions

Newton's method — based on linear approximation around the current

iterate; quadratic convergence
Kantorovich theorem — gives a region around solution such that Newton's
method converges
Newton fractal — indicates which initial condition converges to which root
under Newton iteration
Quasi-Newton method — uses an approximation of the Jacobian:
Broyden's method — uses a rank-one update for the Jacobian
Symmetric rank-one — a symmetric (but not necessarily positive definite)
rank-one update of the Jacobian
Davidon–Fletcher–Powell formula — update of the Jacobian in which the
matrix remains positive definite
Broyden–Fletcher–Goldfarb–Shanno algorithm — rank-two update of the
Jacobian in which the matrix remains positive definite
Limited-memory BFGS method — truncated, matrix-free variant of BFGS
method suitable for large problems
Steffensen's method — uses divided differences instead of the derivative
Secant method — based on linear interpolation at last two iterates
False position method — secant method with ideas from the bisection method
Muller's method — based on quadratic interpolation at last three iterates
Sidi's generalized secant method — higher-order variants of secant method
Inverse quadratic interpolation — similar to Muller's method, but
interpolates the inverse
Brent's method — combines bisection method, secant method and inverse
quadratic interpolation
Ridders' method — fits a linear function times an exponential to last two
iterates and their midpoint
Halley's method — uses f, f' and f''; achieves the cubic convergence
Householder's method — uses first d derivatives to achieve order d + 1;
generalizes Newton's and Halley's method

Methods for polynomials:

Aberth method
Bairstow's method
Durand–Kerner method
Graeffe's method
Jenkins–Traub algorithm — fast, reliable, and widely used
Laguerre's method
Splitting circle method
Analysis:
Wilkinson's polynomial
Numerical continuation — tracking a root as one parameter in the equation
changes
Piecewise linear continuation
Optimization[edit]
Mathematical optimization — algorithm for finding maxima or minima of a
given function
Basic concepts[edit]
Active set
Candidate solution
Constraint (mathematics)
Constrained optimization — studies optimization problems with constraints
Binary constraint — a constraint that involves exactly two variables
Corner solution
Feasible region — contains all solutions that satisfy the constraints but may
not be optimal
Global optimum and Local optimum
Maxima and minima
Slack variable
Continuous optimization
Discrete optimization

Algorithms for linear programming:

Simplex algorithm
Bland's rule — rule to avoid cycling in the simplex method
Klee–Minty cube — perturbed (hyper)cube; simplex method has exponential
complexity on such a domain
Criss-cross algorithm — similar to the simplex algorithm
Big M method — variation of simplex algorithm for problems with both "less
than" and "greater than" constraints
Interior point method
Ellipsoid method
Karmarkar's algorithm
Mehrotra predictor–corrector method
Column generation
k-approximation of k-hitting set — algorithm for specific LP problems (to find
a weighted hitting set)
Linear complementarity problem

Decompositions:

Benders' decomposition
Dantzig–Wolfe decomposition
Theory of two-level planning
Variable splitting
Basic solution (linear programming) — solution at vertex of feasible region
Fourier–Motzkin elimination
Hilbert basis (linear programming) — set of integer vectors in a convex cone
which generate all integer vectors in the cone
LP-type problem
Linear inequality
Vertex enumeration problem — list all vertices of the feasible set
Convex optimization[edit]
Convex optimization
Quadratic programming
Linear least squares (mathematics)
Total least squares
Frank–Wolfe algorithm
Sequential minimal optimization — breaks up large QP problems into a series
of smallest possible QP problems
Bilinear program
Basis pursuit — minimize L1-norm of vector subject to linear constraints
Basis pursuit denoising (BPDN) — regularized version of basis pursuit
In-crowd algorithm — algorithm for solving basis pursuit denoising
Linear matrix inequality
Conic optimization
Semidefinite programming
Second-order cone programming
Sum-of-squares optimization
Quadratic programming (see above)
Bregman method — row-action method for strictly convex optimization
problems
Proximal gradient method — use splitting of objective function in sum of
possible non-differentiable pieces
Subgradient method — extension of steepest descent for problems with a
non-differentiable objective function
Biconvex optimization — generalization where objective function and
constraint set can be biconvex
Nonlinear programming[edit]
Nonlinear programming — the most general optimization problem in the
usual framework
Special cases of nonlinear programming:
See Linear programming and Convex optimization above
Geometric programming — problems involving signomials or posynomials
Signomial — similar to polynomials, but exponents need not be integers
Posynomial — a signomial with positive coefficients
Quadratically constrained quadratic program
Linear-fractional programming — objective is ratio of linear functions,
constraints are linear
Fractional programming — objective is ratio of nonlinear functions,
constraints are linear
Nonlinear complementarity problem (NCP) — find x such that x ≥ 0, f(x) ≥ 0
and xT f(x) = 0
Least squares — the objective function is a sum of squares
Non-linear least squares
Gauss–Newton algorithm
BHHH algorithm — variant of Gauss–Newton in econometrics
Generalized Gauss–Newton method — for constrained nonlinear least-
squares problems
Levenberg–Marquardt algorithm
Iteratively reweighted least squares (IRLS) — solves a weigted least-squares
problem at every iteration
Partial least squares — statistical techniques similar to principal components
analysis
Non-linear iterative partial least squares (NIPLS)
Mathematical programming with equilibrium constraints — constraints
include variational inequalities or complementarities
Univariate optimization:
Golden section search
Successive parabolic interpolation — based on quadratic interpolation
through the last three iterates

Guess value — the initial guess for a solution with which an algorithm starts
Line search
Backtracking line search
Wolfe conditions
Gradient method — method that uses the gradient as the search direction
Gradient descent
Stochastic gradient descent
Landweber iteration — mainly used for ill-posed problems
Successive linear programming (SLP) — replace problem by a linear
programming problem, solve that, and repeat
Sequential quadratic programming (SQP) — replace problem by a quadratic
programming problem, solve that, and repeat
Newton's method in optimization
See also under Newton algorithm in the section Finding roots of nonlinear
equations
Nonlinear conjugate gradient method
Derivative-free methods
Coordinate descent — move in one of the coordinate directions
Adaptive coordinate descent — adapt coordinate directions to objective
function
Random coordinate descent — randomized version
Nelder–Mead method
Pattern search (optimization)
Powell's method — based on conjugate gradient descent
Rosenbrock methods — derivative-free method, similar to Nelder–Mead but
with guaranteed convergence
Augmented Lagrangian method — replaces constrained problems by
unconstrained problems with a term added to the objective function
Ternary search
Tabu search
Guided Local Search — modification of search algorithms which builds up
penalties during a search
Reactive search optimization (RSO) — the algorithm adapts its parameters
automatically
MM algorithm — majorize-minimization, a wide framework of methods
Least absolute deviations
Expectation–maximization algorithm
Ordered subset expectation maximization
Adaptive projected subgradient method
Nearest neighbor search
Space mapping — uses "coarse" (ideal or low-fidelity) and "fine" (practical or
high-fidelity) models
Optimal control and infinite-dimensional optimization[edit]
Optimal control
Pontryagin's minimum principle — infinite-dimensional version of Lagrange
multipliers
Costate equations — equation for the "Lagrange multipliers" in Pontryagin's
minimum principle
Hamiltonian (control theory) — minimum principle says that this function
should be minimized
Types of problems:
Linear-quadratic regulator — system dynamics is a linear differential
equation, objective is quadratic
Linear-quadratic-Gaussian control (LQG) — system dynamics is a linear SDE
with additive noise, objective is quadratic
Optimal projection equations — method for reducing dimension of LQG
control problem
Algebraic Riccati equation — matrix equation occurring in many optimal
control problems
Bang–bang control — control that switches abruptly between two states
Covector mapping principle
Differential dynamic programming — uses locally-quadratic models of the
dynamics and cost functions
DNSS point — initial state for certain optimal control problems with multiple
optimal solutions
Legendre–Clebsch condition — second-order condition for solution of optimal
control problem
Pseudospectral optimal control
Bellman pseudospectral method — based on Bellman's principle of optimality
Chebyshev pseudospectral method — uses Chebyshev polynomials (of the
first kind)
Flat pseudospectral method — combines Ross–Fahroo pseudospectral
method with differential flatness
Gauss pseudospectral method — uses collocation at the Legendre–Gauss
points
Legendre pseudospectral method — uses Legendre polynomials
Pseudospectral knotting method — generalization of pseudospectral methods
in optimal control
Ross–Fahroo pseudospectral method — class of pseudospectral method
including Chebyshev, Legendre and knotting
Ross–Fahroo lemma — condition to make discretization and duality
operations commute
Ross' π lemma — there is fundamental time constant within which a control
solution must be computed for controllability and stability
Sethi model — optimal control problem modelling advertising
Infinite-dimensional optimization
Semi-infinite programming — infinite number of variables and finite number
of constraints, or other way around
Shape optimization, Topology optimization — optimization over a set of
regions
Topological derivative — derivative with respect to changing in the shape
Generalized semi-infinite programming — finite number of variables, infinite
number of constraints

Approaches to deal with uncertainty:

Markov decision process
Partially observable Markov decision process
Probabilistic-based design optimization
Robust optimization
Wald's maximin model
Scenario optimization — constraints are uncertain
Stochastic approximation
Stochastic optimization
Stochastic programming
Stochastic gradient descent

Random optimization algorithms:

Random search — choose a point randomly in ball around current iterate

Simulated annealing
Adaptive simulated annealing — variant in which the algorithm parameters
are adjusted during the computation.
Great Deluge algorithm
Mean field annealing — deterministic variant of simulated annealing
Bayesian optimization — treats objective function as a random function and
places a prior over it
Evolutionary algorithm
Differential evolution
Evolutionary programming
Genetic algorithm, Genetic programming
Genetic algorithms in economics
MCACEA (Multiple Coordinated Agents Coevolution Evolutionary Algorithm)
— uses an evolutionary algorithm for every agent
Simultaneous perturbation stochastic approximation (SPSA)
Luus–Jaakola
Particle swarm optimization
Stochastic tunneling
Harmony search — mimicks the improvisation process of musicians
Convex analysis — function f such that f(tx + (1 − t)y) ≥ tf(x) + (1 − t)f(y) for
t ∈ [0,1]
Pseudoconvex function — function f such that ∇f · (y − x) ≥ 0 implies f(y) ≥
f(x)
Quasiconvex function — function f such that f(tx + (1 − t)y) ≤ max(f(x), f(y))
for t ∈ [0,1]
Subderivative
Geodesic convexity — convexity for functions defined on a Riemannian
manifold
Duality (optimization)
Weak duality — dual solution gives a bound on the primal solution
Strong duality — primal and dual solutions are equivalent
Shadow price
Dual cone and polar cone
Duality gap — difference between primal and dual solution
Fenchel's duality theorem — relates minimization problems with
maximization problems of convex conjugates
Perturbation function — any function which relates to primal and dual
problems
Slater's condition — sufficient condition for strong duality to hold in a convex
optimization problem
Total dual integrality — concept of duality for integer linear programming
Wolfe duality — for when objective function and constraints are differentiable
Farkas' lemma
Karush–Kuhn–Tucker conditions (KKT) — sufficient conditions for a solution
to be optimal
Fritz John conditions — variant of KKT conditions
Lagrange multiplier
Lagrange multipliers on Banach spaces
Semi-continuity
Complementarity — study of problems with constraints of the form 〈u, v 〉 = 0
Mixed complementarity problem
Mixed linear complementarity problem
Lemke's algorithm — method for solving (mixed) linear complementarity
problems
Danskin's theorem — used in the analysis of minimax problems
Maximum theorem — the maximum and maximizer are continuous as
function of parameters, under some conditions

Relaxation (approximation) — approximating a given problem by an easier

problem by relaxing some constraints
Lagrangian relaxation
Linear programming relaxation — ignoring the integrality constraints in a
linear programming problem
Self-concordant function
Reduced cost — cost for increasing a variable by a small amount
Hardness of approximation — computational complexity of getting an
approximate solution

Geometric median — the point minimizing the sum of distances to a given set
of points
Chebyshev center — the centre of the smallest ball containing a given set of
points
Iterated conditional modes — maximizing joint probability of Markov random
field
Response surface methodology — used in the design of experiments
Automatic label placement
Compressed sensing — reconstruct a signal from knowledge that it is sparse
or compressible
Cutting stock problem
Demand optimization
Destination dispatch — an optimization technique for dispatching elevators
Energy minimization
Entropy maximization
Highly optimized tolerance
Hyperparameter optimization
Inventory control problem
Newsvendor model
Extended newsvendor model
Assemble-to-order system
Linear programming decoding
Linear search problem — find a point on a line by moving along the line
Low-rank approximation — find best approximation, constraint is that rank of
some matrix is smaller than a given number
Meta-optimization — optimization of the parameters in an optimization
method
Multidisciplinary design optimization
Optimal computing budget allocation — maximize the overall simulation
efficiency for finding an optimal decision
Paper bag problem
Process optimization
Recursive economics — individuals make a series of two-period optimization
decisions over time.
Stigler diet
Space allocation problem
Stress majorization
Trajectory optimization
Transportation theory
Wing-shape optimization
Miscellaneous[edit]
Combinatorial optimization
Dynamic programming
Bellman equation
Hamilton–Jacobi–Bellman equation — continuous-time analogue of Bellman
equation
Backward induction — solving dynamic programming problems by reasoning
backwards in time
Optimal stopping — choosing the optimal time to take a particular action
Odds algorithm
Robbins' problem
Global optimization:
BRST algorithm
MCS algorithm
Multi-objective optimization — there are multiple conflicting objectives
Benson's algorithm — for linear vector optimization problems
Bilevel optimization — studies problems in which one problem is embedded
in another
Optimal substructure
Dykstra's projection algorithm — finds a point in intersection of two convex
sets
Algorithmic concepts:
Barrier function
Penalty method
Trust region
Test functions for optimization:
Rosenbrock function — two-dimensional function with a banana-shaped
valley
Himmelblau's function — two-dimensional with four local minima, defined by
f(x, y) = (x^2+y-11)^2 + (x+y^2-7)^2
Rastrigin function — two-dimensional function with many local minima
Shekel function — multimodal and multidimensional
Mathematical Optimization Society
Numerical quadrature (integration)[edit]
Numerical integration — the numerical evaluation of an integral
Rectangle method — first-order method, based on (piecewise) constant
approximation
Trapezoidal rule — second-order method, based on (piecewise) linear
approximation
Simpson's rule — fourth-order method, based on (piecewise) quadratic
approximation
Adaptive Simpson's method
Boole's rule — sixth-order method, based on the values at five equidistant
points
Newton–Cotes formulas — generalizes the above methods
Romberg's method — Richardson extrapolation applied to trapezium rule
Gaussian quadrature — highest possible degree with given number of points
Chebyshev–Gauss quadrature — extension of Gaussian quadrature for
integrals with weight (1 − x2)±1/2 on [−1, 1]
Gauss–Hermite quadrature — extension of Gaussian quadrature for integrals
with weight exp(−x2) on [−∞, ∞]
Gauss–Jacobi quadrature — extension of Gaussian quadrature for integrals
with weight (1 − x)α (1 + x)β on [−1, 1]
Gauss–Laguerre quadrature — extension of Gaussian quadrature for
integrals with weight exp(−x) on [0, ∞]
Gauss–Kronrod quadrature formula — nested rule based on Gaussian
quadrature
Gauss–Kronrod rules
Tanh-sinh quadrature — variant of Gaussian quadrature which works well
with singularities at the end points
Clenshaw–Curtis quadrature — based on expanding the integrand in terms of
Chebyshev polynomials
Adaptive quadrature — adapting the subintervals in which the integration
interval is divided depending on the integrand
Monte Carlo integration — takes random samples of the integrand

Quantized state systems method (QSS) — based on the idea of state

quantization
Lebedev quadrature — uses a grid on a sphere with octahedral symmetry
Sparse grid
Coopmans approximation
Numerical differentiation — for fractional-order integrals
Numerical smoothing and differentiation
Adjoint state method — approximates gradient of a function in an
optimization problem
Euler–Maclaurin formula

Numerical methods for ordinary differential equations — the numerical

solution of ordinary differential equations (ODEs)
Euler method — the most basic method for solving an ODE
Explicit and implicit methods — implicit methods need to solve an equation
at every step
Backward Euler method — implicit variant of the Euler method
Trapezoidal rule — second-order implicit method
Runge–Kutta methods — one of the two main classes of methods for initial-
value problems
Midpoint method — a second-order method with two stages
Heun's method — either a second-order method with two stages, or a third-
order method with three stages
Bogacki–Shampine method — a third-order method with four stages (FSAL)
and an embedded fourth-order method
Cash–Karp method — a fifth-order method with six stages and an embedded
fourth-order method
Dormand–Prince method — a fifth-order method with seven stages (FSAL)
and an embedded fourth-order method
Runge–Kutta–Fehlberg method — a fifth-order method with six stages and an
embedded fourth-order method
Gauss–Legendre method — family of A-stable method with optimal order
based on Gaussian quadrature
Butcher group — algebraic formalism involving rooted trees for analysing
Runge–Kutta methods
List of Runge–Kutta methods
Linear multistep method — the other main class of methods for initial-value
problems
Backward differentiation formula — implicit methods of order 2 to 6;
especially suitable for stiff equations
Numerov's method — fourth-order method for equations of the form y'' =
f(t,y)
Predictor–corrector method — uses one method to approximate solution and
another one to increase accuracy
General linear methods — a class of methods encapsulating linear multistep
and Runge-Kutta methods
Bulirsch–Stoer algorithm — combines the midpoint method with Richardson
extrapolation to attain arbitrary order
Exponential integrator — based on splitting ODE in a linear part, which is
solved exactly, and a nonlinear part
Methods designed for the solution of ODEs from classical physics:
Newmark-beta method — based on the extended mean-value theorem
Verlet integration — a popular second-order method
Leapfrog integration — another name for Verlet integration
Beeman's algorithm — a two-step method extending the Verlet method
Dynamic relaxation
Geometric integrator — a method that preserves some geometric structure of
the equation
Symplectic integrator — a method for the solution of Hamilton's equations
that preserves the symplectic structure
Variational integrator — symplectic integrators derived using the underlying
variational principle
Semi-implicit Euler method — variant of Euler method which is symplectic
when applied to separable Hamiltonians
Energy drift — phenomenon that energy, which should be conserved, drifts
away due to numerical errors
Other methods for initial value problems (IVPs):
Bi-directional delay line
Partial element equivalent circuit
Methods for solving two-point boundary value problems (BVPs):
Shooting method
Direct multiple shooting method — divides interval in several subintervals
and applies the shooting method on each subinterval
Methods for solving differential-algebraic equations (DAEs), i.e., ODEs with
constraints:
Constraint algorithm — for solving Newton's equations with constraints
Pantelides algorithm — for reducing the index of a DEA
Methods for solving stochastic differential equations (SDEs):
Euler–Maruyama method — generalization of the Euler method for SDEs
Milstein method — a method with strong order one
Runge–Kutta method (SDE) — generalization of the family of Runge–Kutta
methods for SDEs
Methods for solving integral equations:
Nyström method — replaces the integral with a quadrature rule

Analysis:
Truncation error (numerical integration) — local and global truncation
errors, and their relationships
Lady Windermere's Fan (mathematics) — telescopic identity relating local
and global truncation errors
Stiff equation — roughly, an ODE for which unstable methods need a very
short step size, but stable methods do not
L-stability — method is A-stable and stability function vanishes at infinity
Dynamic errors of numerical methods of ODE discretization — logarithm of
stability function
Adaptive stepsize — automatically changing the step size when that seems
advantageous

Numerical partial differential equations — the numerical solution of partial

differential equations (PDEs)

Finite difference method — based on approximating differential operators

with difference operators
Finite difference — the discrete analogue of a differential operator
Finite difference coefficient — table of coefficients of finite-difference
approximations to derivatives
Discrete Laplace operator — finite-difference approximation of the Laplace
operator
Eigenvalues and eigenvectors of the second derivative — includes
eigenvalues of discrete Laplace operator
Kronecker sum of discrete Laplacians — used for Laplace operator in
multiple dimensions
Discrete Poisson equation — discrete analogue of the Poisson equation using
the discrete Laplace operator
Stencil (numerical analysis) — the geometric arrangements of grid points
affected by a basic step of the algorithm
Compact stencil — stencil which only uses a few grid points, usually only the
immediate and diagonal neighbours
Higher-order compact finite difference scheme
Non-compact stencil — any stencil that is not compact
Five-point stencil — two-dimensional stencil consisting of a point and its four
immediate neighbours on a rectangular grid
Finite difference methods for heat equation and related PDEs:
FTCS scheme (forward-time central-space) — first-order explicit
Crank–Nicolson method — second-order implicit
Finite difference methods for hyperbolic PDEs like the wave equation:
Lax–Friedrichs method — first-order explicit
Lax–Wendroff method — second-order explicit
MacCormack method — second-order explicit
Upwind scheme
Upwind differencing scheme for convection — first-order scheme for
convection–diffusion problems
Lax–Wendroff theorem — conservative scheme for hyperbolic system of
conservation laws converges to the weak solution
Alternating direction implicit method (ADI) — update using the flow in x-
direction and then using flow in y-direction

Finite difference methods for option pricing

Finite-difference time-domain method — a finite-difference method for
electrodynamics
Finite element methods[edit]
Finite element method — based on a discretization of the space of solutions
Finite element method in structural mechanics — a physical approach to
finite element methods
Galerkin method — a finite element method in which the residual is
orthogonal to the finite element space
Discontinuous Galerkin method — a Galerkin method in which the
approximate solution is not continuous
Rayleigh–Ritz method — a finite element method based on variational
principles
Spectral element method — high-order finite element methods
hp-FEM — variant in which both the size and the order of the elements are
automatically adapted
Examples of finite elemets:
Bilinear quadrilateral element — also known as the Q4 element
Constant strain triangle element (CST) — also known as the T3 element
Barsoum elements
Direct stiffness method — a particular implementation of the finite element
method, often used in structural analysis
Trefftz method
Finite element updating
Extended finite element method — puts functions tailored to the problem in
the approximation space
Functionally graded elements — elements for describing functionally graded
materials
Superelement — particular grouping of finite elements, employed as a single
element
Interval finite element method — combination of finite elements with interval
arithmetic
Discrete exterior calculus — discrete form of the exterior calculus of
differential geometry
Modal analysis using FEM — solution of eigenvalue problems to find natural
vibrations
Céa's lemma — solution in the finite-element space is an almost best
approximation in that space of the true solution
Patch test (finite elements) — simple test for the quality of a finite element
MAFELAP (MAthematics of Finite ELements and APplications) —
international conference held at Brunel University
NAFEMS — not-for-profit organisation that sets and maintains standards in
computer-aided engineering analysis
Multiphase topology optimisation — technique based on finite elements for
determining optimal composition of a mixture
Interval finite element
Applied element method — for simulation of cracks and structural collapse
Wood–Armer method — structural analysis method based on finite elements
used to design reinforcement for concrete slabs
Isogeometric analysis — integrates finite elements into conventional NURBS-
based CAD design tools
Stiffness matrix — finite-dimensional analogue of differential operator
Combination with meshfree methods:
Weakened weak form — form of a PDE that is weaker than the standard weak
form
G space — functional space used in formulating the weakened weak form
Smoothed finite element method
List of finite element software packages
Other methods[edit]
Spectral method — based on the Fourier transformation
Pseudo-spectral method
Method of lines — reduces the PDE to a large system of ordinary differential
equations
Boundary element method (BEM) — based on transforming the PDE to an
integral equation on the boundary of the domain
Interval boundary element method — a version using interval arithmetics
Analytic element method — similar to the boundary element method, but the
integral equation is evaluated analytically
Finite volume method — based on dividing the domain in many small
domains; popular in computational fluid dynamics
Godunov's scheme — first-order conservative scheme for fluid flow, based on
piecewise constant approximation
MUSCL scheme — second-order variant of Godunov's scheme
AUSM — advection upstream splitting method
Flux limiter — limits spatial derivatives (fluxes) in order to avoid spurious
oscillations
Riemann solver — a solver for Riemann problems (a conservation law with
piecewise constant data)
Properties of discretization schemes — finite volume methods can be
conservative, bounded, etc.
Discrete element method — a method in which the elements can move freely
relative to each other
Extended discrete element method — adds properties such as strain to each
particle
Movable cellular automaton — combination of cellular automata with
discrete elements
Meshfree methods — does not use a mesh, but uses a particle view of the
field
Discrete least squares meshless method — based on minimization of
weighted summation of the squared residual
Diffuse element method
Finite pointset method — represent continuum by a point cloud
Moving Particle Semi-implicit Method
Method of fundamental solutions (MFS) — represents solution as linear
combination of fundamental solutions
Variants of MFS with source points on the physical boundary:
Boundary knot method (BKM)
Boundary particle method (BPM)
Regularized meshless method (RMM)
Singular boundary method (SBM)
Methods designed for problems from electromagnetics:
Finite-difference time-domain method — a finite-difference method
Rigorous coupled-wave analysis — semi-analytical Fourier-space method
based on Floquet's theorem
Transmission-line matrix method (TLM) — based on analogy between
electromagnetic field and mesh of transmission lines
Uniform theory of diffraction — specifically designed for scattering problems
Particle-in-cell — used especially in fluid dynamics
Multiphase particle-in-cell method — considers solid particles as both
numerical particles and fluid
High-resolution scheme
Shock capturing method
Vorticity confinement — for vortex-dominated flows in fluid dynamics, similar
to shock capturing
Split-step method
Fast marching method
Orthogonal collocation
Lattice Boltzmann methods — for the solution of the Navier-Stokes equations
Roe solver — for the solution of the Euler equation
Relaxation (iterative method) — a method for solving elliptic PDEs by
converting them to evolution equations
Broad classes of methods:
Mimetic methods — methods that respect in some sense the structure of the
original problem
Multiphysics — models consisting of various submodels with different physics
Immersed boundary method — for simulating elastic structures immersed
within fluids
Multisymplectic integrator — extension of symplectic integrators, which are
for ODEs
Stretched grid method — for problems solution that can be related to an
elastic grid behavior.
Techniques for improving these methods[edit]
Multigrid method — uses a hierarchy of nested meshes to speed up the
methods
Domain decomposition methods — divides the domain in a few subdomains
and solves the PDE on these subdomains
Additive Schwarz method
Abstract additive Schwarz method — abstract version of additive Schwarz
without reference to geometric information
Balancing domain decomposition method (BDD) — preconditioner for
symmetric positive definite matrices
Balancing domain decomposition by constraints (BDDC) — further
development of BDD
Finite element tearing and interconnect (FETI)
FETI-DP — further development of FETI
Fictitious domain method — preconditioner constructed with a structured
mesh on a fictitious domain of simple shape
Mortar methods — meshes on subdomain do not mesh
Neumann–Dirichlet method — combines Neumann problem on one
subdomain with Dirichlet problem on other subdomain
Neumann–Neumann methods — domain decomposition methods that use
Neumann problems on the subdomains
Poincaré–Steklov operator — maps tangential electric field onto the
equivalent electric current
Schur complement method — early and basic method on subdomains that do
not overlap
Schwarz alternating method — early and basic method on subdomains that
overlap
Coarse space — variant of the problem which uses a discretization with
fewer degrees of freedom
Adaptive mesh refinement — uses the computed solution to refine the mesh
only where necessary
Fast multipole method — hierarchical method for evaluating particle-particle
interactions
Perfectly matched layer — artificial absorbing layer for wave equations, used
to implement absorbing boundary conditions
Grids and meshes[edit]
Grid classification / Types of mesh:
Polygon mesh — consists of polygons in 2D or 3D
Triangle mesh — consists of triangles in 2D or 3D
Triangulation (geometry) — subdivision of given region in triangles, or
higher-dimensional analogue
Nonobtuse mesh — mesh in which all angles are less than or equal to 90°
Point set triangulation — triangle mesh such that given set of point are all a
vertex of a triangle
Polygon triangulation — triangle mesh inside a polygon
Delaunay triangulation — triangulation such that no vertex is inside the
circumcentre of a triangle
Constrained Delaunay triangulation — generalization of the Delaunay
triangulation that forces certain required segments into the triangulation
Pitteway triangulation — for any point, triangle containing it has nearest
neighbour of the point as a vertex
Minimum-weight triangulation — triangulation of minimum total edge length
Kinetic triangulation — a triangulation that moves over time
Triangulated irregular network
Quasi-triangulation — subdivision into simplices, where vertiсes are not
points but arbitrary sloped line segments
Volume mesh — consists of three-dimensional shapes
Regular grid — consists of congruent parallelograms, or higher-dimensional
analogue
Unstructured grid
Geodesic grid — isotropic grid on a sphere
Mesh generation
Image-based meshing — automatic procedure of generating meshes from 3D
image data
Marching cubes — extracts a polygon mesh from a scalar field
Parallel mesh generation
Ruppert's algorithm — creates quality Delauney triangularization from
piecewise linear data

Apollonian network — undirected graph formed by recursively subdividing a

triangle
Barycentric subdivision — standard way of dividing arbitrary convex
polygons into triangles, or the higher-dimensional analogue
Chew's second algorithm — improves Delauney triangularization by refining
poor-quality triangles
Laplacian smoothing — improves polynomial meshes by moving the vertices
Jump-and-Walk algorithm — for finding triangle in a mesh containing a given
point
Spatial twist continuum — dual representation of a mesh consisting of
hexahedra
Pseudotriangle — simply connected region between any three mutually
tangent convex sets
Simplicial complex — all vertices, line segments, triangles, tetrahedra, …,
making up a mesh

Lax equivalence theorem — a consistent method is convergent if and only if it

is stable
Courant–Friedrichs–Lewy condition — stability condition for hyperbolic PDEs
Von Neumann stability analysis — all Fourier components of the error should
be stable
Numerical diffusion — diffusion introduced by the numerical method, above
to that which is naturally present
False diffusion
Numerical resistivity — the same, with resistivity instead of diffusion
Weak formulation — a functional-analytic reformulation of the PDE necessary
for some methods
Total variation diminishing — property of schemes that do not introduce
spurious oscillations
Godunov's theorem — linear monotone schemes can only be of first order
Motz's problem — benchmark problem for singularity problems
Direct simulation Monte Carlo
Quasi-Monte Carlo method
Markov chain Monte Carlo
Metropolis–Hastings algorithm
Multiple-try Metropolis — modification which allows larger step sizes
Wang and Landau algorithm — extension of Metropolis Monte Carlo
Equation of State Calculations by Fast Computing Machines — 1953 article
proposing the Metropolis Monte Carlo algorithm
Multicanonical ensemble — sampling technique that uses Metropolis–
Hastings to compute integrals
Gibbs sampling
Coupling from the past
Reversible-jump Markov chain Monte Carlo
Dynamic Monte Carlo method
Kinetic Monte Carlo
Gillespie algorithm
Particle filter
Auxiliary particle filter
Reverse Monte Carlo
Demon algorithm
Pseudo-random number sampling
Inverse transform sampling — general and straightforward method but
computationally expensive
Rejection sampling — sample from a simpler distribution but reject some of
the samples
Ziggurat algorithm — uses a pre-computed table covering the probability
distribution with rectangular segments
For sampling from a normal distribution:
Box–Muller transform
Marsaglia polar method
Convolution random number generator — generates a random variable as a
sum of other random variables
Indexed search
Variance reduction techniques:
Antithetic variates
Control variates
Importance sampling
Stratified sampling
VEGAS algorithm
Low-discrepancy sequence
Constructions of low-discrepancy sequences
Event generator
Parallel tempering
Umbrella sampling — improves sampling in physical systems with significant
energy barriers
Hybrid Monte Carlo
Ensemble Kalman filter — recursive filter suitable for problems with a large
number of variables
Transition path sampling
Walk-on-spheres method — to generate exit-points of Brownian motion from
bounded domains

Ensemble forecasting — produce multiple numerical predictions from slightly

differing initial conditions or parameters
Bond fluctuation model — for simulating the conformation and dynamics of
polymer systems
Iterated filtering
Metropolis light transport
Monte Carlo localization — estimates the position and orientation of a robot
Monte Carlo methods for electron transport
Monte Carlo method for photon transport
Monte Carlo methods in finance
Monte Carlo methods for option pricing
Quasi-Monte Carlo methods in finance
Monte Carlo molecular modeling
Path integral molecular dynamics — incorporates Feynman path integrals
Quantum Monte Carlo
Diffusion Monte Carlo — uses a Green function to solve the Schrödinger
equation
Gaussian quantum Monte Carlo
Path integral Monte Carlo
Reptation Monte Carlo
Variational Monte Carlo
Methods for simulating the Ising model:
Swendsen–Wang algorithm — entire sample is divided into equal-spin
clusters
Wolff algorithm — improvement of the Swendsen–Wang algorithm
Metropolis–Hastings algorithm
Auxiliary field Monte Carlo — computes averages of operators in many-body
quantum mechanical problems
Cross-entropy method — for multi-extremal optimization and importance
sampling
Also see the list of statistics topics

Large eddy simulation

Smoothed-particle hydrodynamics
Aeroacoustic analogy — used in numerical aeroacoustics to reduce sound
sources to simple emitter types
Stochastic Eulerian Lagrangian method — uses Eulerian description for
fluids and Lagrangian for structures
Explicit algebraic stress model
Computational magnetohydrodynamics (CMHD) — studies electrically
conducting fluids
Geodesic grid
Quantum jump method — used for simulating open quantum systems,
operates on wave function
Dynamic Design Analysis Method (DDAM) — for evaluating effect of
underwater explosions on equipment

Cell lists
Coupled cluster
Density functional theory
DIIS — direct inversion in (or of) the iterative subspace

Now, if one were to pick any given set of approaches, one would have to first
ensure that the set of approaches is commutative, i.e. many of these
approaches assume slightly different ontological premises and thus cannot
be used together, which is not an issue for mathematics per se, since it
doesn't claim to imply any relation to empirical data. However it becomes an
issue in applying computational approaches to empirical data.
Even within the simplest mathematical systems, for instance regular
arithmetic and simple linear algebra, there is no rational transition between
the two. We tend to assume there is primarily because most of us made the
transition initially as children and habituated the change in the basis of
understanding the symbology involved. In terms of the process of learning,
this is reinforced by the sense that simple algebra is somehow based on
arithmetic, since arithmetic is always an assumed prerequisite. However this
is illusory. The prerequisite practice of arithmetic is simply the habituating
of the ability to manipulate mathematical symbols in the most general sense,
it in no way implies that there is any necessary or even contingent relation
between the symbology of arithmetic and simple algebra. Within
computationl mathematics, which is only a small subset of mathematical
systems in general, there are dozens of underlying mathematical systems
that have no rational transitions, i.e. are non-commutative. Within
computation, which generally runs on a 'good enough' approach, this only
occasionally creates issues. However if one is trying to model a given system
accurately rather than simply using a 'good enough' simulation to provide an
optimization to a purely computational problem, this simultaneous use of
non-commutative approaches cannot be permitted.
Once it is confirmed that the set is fully commutative (which we have no
quick or simple means of doing) one would have to ensure that the operative
ontology in all the approaches is a sensible one in terms of understanding
and manipulating empirical biological data, i.e. determining that the
operative ontology of the mathematical approaches is identical to the actual
operative ontology of real biological systems. We have no theoretical means
of accomplishing this, never mind a practical method.
Even within the small set of biologically inspired computational approaches,
while it is true that the models behave in a manner that is somewhat similar
to the biological systems they were inspired by, it is also true that they do not
do so with any accuracy. This lack of realistic precision could be due to the
model being a relatively closed system when compared with the actual
biological system, or it could be due to the model failing to take into account
or failing to accurately determine initial conditions for all relevant
parameters, or it could be that the model is based on invalid ontological
assumptions and merely mimics a certain aspect of reality without implying
anything ontologically valid about reality. There's no means to distinguish
between these potential origins of inaccuracy except by modeling the system
in question and its entire spatio-temporal environment, which is nothing
other than the rest of reality itself, with all parameters accurately
determined. The only feasible model we can ever have for that is reality
itself.
While even the more complex single celled systems are beyond the modeling
capability of computational mathematics, multicellular cell differentiated
systems are beyond the capability of computation in a more general sense. In
a similar manner to the lack of commutativity between different
mathematical systems, there is no commutativity between a single cell
system and a multicellular, cell differentiated system, i.e. there is no rational
transition, since the generation of a more comprehensive generic view is
itself an ontological, not a rational exercise.