Balancing AVL Trees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

16
AVL Illustration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Analysis of AVL Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Chapter 6: Transform and Conquer Horner’s Rule and Fast Exponentiation 19
Horner’s Rule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Horner’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
You can never solve a problem Fast Exponentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
on the level on which it was cre- Fast Exponentiation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
ated. (Albert Einstein)

We cannot solve our problems

with the same thinking we used
when we created them.
(Albert Einstein)

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Presorting Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Gaussian Elimination 4
System of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Example of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Forward Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Forward Elimination Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Backward Substitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Backward Substitution Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Comments on Gaussian Elimination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Balanced Search Trees: AVL Trees 11
Binary Search Tree Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
AVL Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Binary Search Tree Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
R-Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
LR-Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1 2
Introduction Gaussian Elimination 4

Transform-and-conquer is an approach to solving a problem by changing an System of Linear Equations

instance to:
Gaussian Elimination is an algorithm solving a system of n linear equations in n
 a simpler instance of the same problem, or
unknowns.
 a different representation of the same problem, or
 an instance of a different problem. a11 x1 + a12 x1 + · · · + a1n xn = b1
a21 x1 + a22 x1 + · · · + a2n xn = b2
..
.
an1 x1 + an2 x1 + · · · + ann xn = bn
where aij and bi are the coefficients, and xj are the unknowns.
CS 3343 Analysis of Algorithms Chapter 6: Slide – 2

For example, the linear least squares method in statistics needs to solve this kind
Presorting Example of system.
CS 3343 Analysis of Algorithms Chapter 6: Slide – 4
Presorting (sorting the data in advance) makes it easier to:
 Check element uniqueness. Example of Linear Equations
 Compute the mode.
 Search the data. For example, consider this system of 3 equations:
 Find the minimum, maximum, median, or any order statistic.
2x1 − x2 + 3x3 = 4
CS 3343 Analysis of Algorithms Chapter 6: Slide – 3
3x1 + 2x2 + x3 = 6
−5x1 + x2 − 2x3 = 3
This system has the solution x1 = −5/4, x2 = 13/4, and x3 = 13/4.

Gaussian elimination works in two stages: forward elimination and backward

substitution.

Forward elimination adds/subtracts equations from each other.

CS 3343 Analysis of Algorithms Chapter 6: Slide – 5

3 4
Forward Elimination Backward Substitution
 
2 −1 3 4 Solving last equation 4x3 = 13 yields x3 = 13/4.
 3 2 1 6  row 2 − 3/2 row 1
−5 1 −2 3 row 3 + 5/2 row 1 Substitute for x3 in second equation:
  7/2x2 − 7/2x3 = 0
2 −1 3 4
 0 7/2 −7/2 0  7/2x2 = 0 + 7/2x3 = 0 + 91/8 = 91/8
x2 = (91/8)/(7/2) = 13/4
0 −3/2 11/2 13 row 3 + 3/7 ∗ row 2
  Substitute for x2 and x3 in first equation:
2 −1 3 4
 0 7/2 −7/2 0  2x1 − x2 + 3x3 = 4
0 0 4 13 2x1 = 4 + x2 − 3x3 = 4 + 13/4 − 3 ∗ (13/4)
2x1 = −10/4
CS 3343 Analysis of Algorithms Chapter 6: Slide – 6 x1 = −5/4
CS 3343 Analysis of Algorithms Chapter 6: Slide – 8

Forward Elimination Algorithm

Backward Substitution Algorithm
algorithm ForwardElim(A[1..n, 1..n], b[1..n])
// First stage to solve Ax = b by Gaussian Elim.
algorithm BackwardSubst(A[1..n, 1..n + 1])
// Input: A matrix A and column vector b
// Second stage to solve Ax = b by Gaussian Elim.
// Output: Equivalent upper-triangular matrix
// Input: An upper-triangular matrix A
for i ← 1 to n do
// Output: Solution column-vector x
A[i, n + 1] ← b[i]
for i ← n downto 1 do
for i ← 1 to n − 1 do
s ← A[i, n + 1]
for j ← i + 1 to n do
for j ← i + 1 to n do
for k ← i to n + 1 do
s ← s − A[i, j] ∗ x[j]
A[j, k] ← A[j, k] − A[i, k] ∗ A[j, i]/A[i, i]
x[i] ← s/A[i, i]
CS 3343 Analysis of Algorithms Chapter 6: Slide – 7
return x

CS 3343 Analysis of Algorithms Chapter 6: Slide – 9

5 6
Comments on Gaussian Elimination AVL Trees
 Gaussian Elimination requires that A be nonsingular. See your linear algebra  For each node in binary search tree, all keys on its left are ≤ to its key, and all
book. keys on its right are ≥ to its key.
 Gaussian Elimination is Θ(n3). Note triple loop in ForwardElim.  In a balanced search tree, the height is Θ(log n), where n is the number of
 Better numerical properties can be obtained through partial pivoting. See nodes.
book.  For each node in an AVL tree, the heights of the left and right subtree are
 LU decomposition is an extension of Gaussian elimination. After Θ(n3) equal or differ by one.
preprocessing of the A matrix, only Θ(n2) is needed for any b vector. See  The balance factor is the left height minus the right height.
book.  Tree rotations are used to transform the tree into balance.
 Similar processing can be used to compute matrix inverse and determinant. CS 3343 Analysis of Algorithms Chapter 6: Slide – 12

CS 3343 Analysis of Algorithms Chapter 6: Slide – 10

Binary Search Tree Insertion

Balanced Search Trees: AVL Trees 11 algorithm TreeInsert(T, z)
// Inserts a node into a binary search tree
Binary Search Tree Notation // Input: Tree T and node z
y ← null
 T.root is the root of tree T or null.
x ← T.root
 x.key is the key of node x.
while x 6= null do
 x.parent is the parent of x or null.
y←x
 x.left is the left child of x or null.
if z.key < x.key then x ← x.left
 x.right is the right child of x or null.
else x ← x.right
 x.height is the height of x. null.height = −1.
z.parent ← y
if y = null then T.root ← z
else if z.key < y.key then y.left ← z
else y.right ← z

CS 3343 Analysis of Algorithms Chapter 6: Slide – 13

CS 3343 Analysis of Algorithms Chapter 6: Slide – 11

7 8
R-Rotation Balancing AVL Trees
An R-rotation is used when the left child is too high, and the left-left grandchild is
algorithm AVLBalance(x)
as high or higher than the left-right one.
// Input: Node x
// Output: x and nodes above it are balanced
while x 6= null do
if x.left.height > x.right.height + 1 then
if x.left.left.height < x.left.right.height
then LR-Rotation(x)
else R-Rotation(x)
CS 3343 Analysis of Algorithms Chapter 6: Slide – 14
else if x.left.height + 1 < x.right.height then
if x.right.left.height > x.right.right.height
then RL-Rotation(x)
LR-Rotation else L-Rotation(x)
x ← x.parent
An LR-rotation is used when the left child is too high and the left-right grandchild
is higher than the left-left one. CS 3343 Analysis of Algorithms Chapter 6: Slide – 16

AVL Illustration

CS 3343 Analysis of Algorithms Chapter 6: Slide – 15

CS 3343 Analysis of Algorithms Chapter 6: Slide – 17

9 10
Analysis of AVL Trees Horner’s Algorithm
The height h of an AVL tree with n nodes satisfies log2(n/2) < h < 1.5 log2 n
 A binary tree of height h has ≤ 2h+1 −1 nodes. algorithm Horner(P [0..n], x)
// Evaluates a polynomial at x
n < 2h+1 → n/2 < 2h → log2(n/2) < h // Input: A value x and an array P of coefficients
 An AVL tree with height h at worst has subtrees of height h − 1 and h − 2. // Output: The value of the polynomial at x
 Assuming heights h − 1 and h − 2 have at least 1.6h−1 and 1.6h−2 nodes, v ← P [n]
respectively: for i ← n − 1 downto 0 do
v ← x ∗ v + P [i]
n > 1.6h−1 + 1.6h−2 > 1.6h →
return v
1.6h < n → h < 1.5 log2 n
CS 3343 Analysis of Algorithms Chapter 6: Slide – 18 CS 3343 Analysis of Algorithms Chapter 6: Slide – 20

Fast Exponentiation
Horner’s Rule and Fast Exponentiation 19
We can evaluate xn quickly based on the binary expansion of the exponent. For
example,
Horner’s Rule
 210 can be evaluated by 210 = 210102 = 28 ∗ 22 .
Horner’s rule is an efficient algorithm for evaluating a polynomial. For example,  213 can be evaluated by 213 = 211012 = 28 ∗ 24 ∗ 2.
 5x2 − 3x + 8 can be evaluated by (5x + 3)x + 8.  Algorithm on next page corresponds to book’s right to left algorithm.
 7x3 + x2 − 9x − 2 can be evaluated by ((7x + 1)x − 9)x − 2. CS 3343 Analysis of Algorithms Chapter 6: Slide – 21

The generalization of this is Horner’s rule.

It performs n multiplications and n additions/subtractions for an n-degree
polynomial.
CS 3343 Analysis of Algorithms Chapter 6: Slide – 19

11 12
Fast Exponentiation Algorithm

algorithm FastExpt(x, n)
// Evaluates xn
// Input: A value x and an integer n ≥ 0
// Output: The value of xn
term ← x
product ← 1
while n > 0 do
if n mod 2 = 1 then
product ← product ∗ term
term ← term ∗ term
n ← ⌊n/2⌋
return product

CS 3343 Analysis of Algorithms Chapter 6: Slide – 22

13