Numerical_Methods_Formula_Sheet
Numerical_Methods_Formula_Sheet
uk k
Sheet ||uk || ; λ = vkT Avk
√
Singular values: Find roots of AT A − λI, return λ=σ
1 Algorithms
1.7 Nonlinear Systems & Optimization
1.1 Nonlinear Equations in 1 Variable Taylor Series for systems: f (x+p) = f (x)+J(x)p+O(||p||2 )
∂f ∂f1
1
Bisection Method: given [a, b] s.t. f (a) ∗ f (b) < 0, com- ∂x ... ∂xn
1
Jacobian Matrix: J(x) = ... ... ...
pute f ( b+a
2 ) and replace the x-val for which sgn(f (x)) == ∂fn∂fn
sgn(f ( b+a b+a
η≤ 1 ... ∂x
2 )) with 2 . , where k is the number of it- n∂x1
2k
∗ b−a Newton’s Method for Systems: solve J(xk )pk = −f (xk ),
erations run. If |x − xn | ≤ atol, then n = ceil(log2 ( 2atol )).
then xk+1 = xk + pk .
FPI: x = g(x) until stopping point. Converges if |g‘(x)| ≤
1 ∀x ∈ (a, b). Rate = −log10 (ρ). =⇒ k = 1/rate.
Newton’s method: xk+1 = xk − f (xk ) 1.8 Approximating Polynomials
f ′ (xk )
f (xk )(xk −xk−1 )
Secant method: xk+1 = xk − f (xk )−f (xk−1 )
Monomial basis: pn (x) = c0 + c1 x+...+cn xn ; Solve Ac = x
The above methods converge if f (x ) = 0, f (x∗ ) ̸= 0.
∗ ′ for c. Lagrange interpolation: pn (x) = Σyi ϕi (x), where
phii (xn ) = 1 iff x = xn , 0 otherwise.
1.2 Linear Algebra Divided Differences: pn (x) = c0 + c1 (x − x0 ) + ... + cn (x −
a·b
xn−1 )(x − xn−2 )...(x − x0 ), solve as in monomial basis.
projb a = projection of A onto B = b·b b f n+1 (ξ) n
η: (n+1)! Π0 (x − xi ); maximize ξ for an upper bound.
||x − y|| = ||z||, zi = xi − yi