Practice Questions

1
Recurrence Examples

 0 n0  0 n0
s ( n)   s ( n)  
c  s (n  1) n  0 n  s (n  1) n  0

n 1 
 c  c n 1
 
T ( n)   T ( n)  
2T  n   c n  1  n
  2  aT    cn n  1
 b 2
Substitution Method
• Quick Sort and Merge Sort Evaluation

n
T (n)  2.T ( )  n
2
n n n
T ( )  2.T ( 2 ) 
2 2 2
n n n
T ( 2 )  2.T ( 3 )  2
2 2 2
n n n
T ( 3 )  2.T ( 4 )  3 . . .
2 2 2
3
n n n
T ( k 1 )  2.T ( k )  k 1
2 2 2
Substitution Method
n n
T (n)  2.T ( )  (n)  2 .T ( 2 )  n  n
2

2 2
n
T (n)  2 .T ( 2 )  n  n
2

2
n
T (n)  2 .T ( 3 )  n  n  n
3

2
...
n
T (n)  2 .T ( k )  n
k
n  . n
. .
2 k times
n
T (n)  2 .T ( k )  k .n
k
Let us suppose that : n  2 k  log 2 n  k
2
4
Hence, T (n)  n.T (1)  n. log 2 n  n  n. log 2 n
T (n)  (n. log 2 n)
 Substitution Method
• Quick Sort Worst case: pivot is smallest/largest element all the time.
T(n) = T(n-1) + cn
T(n-1) = T(n-2) + c(n-1)
T(n-2) = T(n-3) + c(n-2)
…
T(n-k) = T(n-k-1) + c(n-k)

T(n) = T(n-1) + cn  T(n) = T(n-2) + c(n-1) + cn

T(n) = T(n-3) + c(n-2) + c(n-1) + cn  T(n) = T(n-k) + c(n + n-1 + … n-k)

Let us suppose that : n  k  1  k  n  1

n 5
T(n)  T(1)  c  i  O(n ) 2

i 1
Substitution Method
Solve the recurrence relation given below.
1 if n  1

T ( n)   n
3  T ( 4 )  n otherwise

Solution:
n
(1) T (n)  3  T ( )  n
4
n n n
(2) T ( )  3  T ( 2 ) 
4 4 4
n n n
(3) T ( 2 )  3  T ( 3 )  2 and so on
4 4 4
6
n n n
(4) T ( k 1 )  3  T ( k )  k 1
4 4 4
 Substitution Method
n
Now substitute the value of T ( ) from (2) to (1)
4
 n n
(5) T (n)  33T ( 2 )    n
 4 4
n 3
 3 T ( 2 )  ( )  n  n
2

4 4
n
substitute the value of T ( 2 ) from (3) to (5) equation, we have
4
 n n 3
T (n)  3 3  T ( 3 )  2   ( )n  n
2

 4 4  4
n 3 k 1 3 k 2 3 3 0
T ( n)  3  T ( k )  ( ) n  ( )       ( ) n  ( ) n
k

4 4 4 4 4
Let us suppose that : n  4 k  log 4 n  k 7
n  3 3 3 
T (n)  3k  T ( )  n 1  ( )  ( ) 2  ....  ( ) k 1 
n  4 4 4 
Substitution Method
n  3 3 2 3 k 1 
T (n)  3  T ( )  n 1  ( )  ( )  ....  ( ) 
k

n  4 4 4 
 3 k 
 1  ( ) 
4 k 1 1 xk
T (n)  3  T (1)  n 1  (
k
) 1  x  x  ...  x
2
 1 ( )
 3  1 x
1
 4 
3k
T (n)  3  1  4n(1  k )
k
 T (1)  1
4
3 k
3 nk
 3 k
T (n)  3  4n  (1  k )  3  4n  (1  )  3 k  4n(
k k
)
4 n n
 3k  4(n  3k )  1  3k  4 n  4  3k

 4 n  33 k
 4n  3  3 log n
4 8

T ( n)  4n  3  n Hence T (n)   (n)

log 34
 Substitution Method A Hard Example
1 if n  1
 where 0  a  b
T ( n)   n
 aT ( )  n otherwise
b
n n n
Solution : T (n)  aT ( b )  n  a  a  T ( 2 )  a   n
b b
 a  T ( 2 )  ( )n  n  a 2 aT ( n3 )  n2   ( a )n  n
2 n a
b b  b b  b
n a 2 a
 a T ( 3 )  ( ) n  ( )n  n
3

b b b
n a k 1 a k 2 a 1
T (n)  a T ( k )  ( ) n  ( ) n  ....  ( ) n  n
k

b b b b
9
n  a a 2 a k 1 
 a T ( k )  n 1  ( )  ( )  ....  ( ) 
k

b  b b b 
 Substitution Method
 a a k 1 
T (n)  a  T (1)  n 1  ( )  ....  ( ) 
k
Let us suppose that, n  b k
 b b 
 a k
 1 ( )  b a k
 a k T (1)  n  b
  a k  T (1)  n( )(1  k )
 1 a  ba b
 b 
b b a k k
b b k
 a k
 a  1  n(
k
)( )  a k
 n( )( )
ba b k
ba n
b b b bab k
a k
b 
k
a k
  b k
 ( )a
ba ba ba ba
b a log bn b a
T ( n)  ( )b 
k
a  n  n log ba
10
ba ba ba ba
 T ( n)   ( n)
Substitution Method A More Hard Example
1 if n  1

T ( n)   n
3  T ( )  cn 2
otherwise
4
Solution : (1) T (n)  3  T ( n )  cn 2
4
n n n
(2) T ( )  3  T ( 2 )  c( ) 2
4 4 4
n n n 2
(3) T ( 2 )  3  T ( 3 )  c( 2 )
4 4 4
n n n 2
(4) T ( 3
)  3  T ( 4
)  c ( 3
) and so on
4 4 4 11
n n n 2
(5) T ( k 1 )  3  T ( k )  c( k 1 )
4 4 4
Substitution Method
n
Now substitute the value of T ( ) from (2) to (1)
4
 n n 2 n n 2
(6) T (n)  33T ( 2 )  c( )   cn  3 T ( 2 )  3c( )  cn 2
2 2

 4 4  4 4
n
substitute the value of T ( 2
) from (3) to (6), we have
4
 n n 2 n 2
T ( n)  3 3T ( 3 )  c( 2 )   3c  ( )  cn 2
2

 4 4  4
n n 2 n 2
 3 T ( 3 )  3 c( 2 )  3c  ( )  cn 2
3 2

4 4 4
n 3 2 2 3 1 2
 3 T ( 3 )  ( 2 ) cn  ( 2 ) cn  cn 2
3

4 4 4
12
n  3 k 1 3 k 2 3 0 2
T ( n )  3 T ( k )  ( 2 )  ( 2 )
k
      ( 2 )  cn
4  4 4 4 
 Substitution Method
Let us suppose that n can be expressed as n  4 k
 3 3 2 3 k 1 
T (n)  3 T (1)  cn 1   ( )      ( ) 
k 2

 16 16 16 
 3 k 
 1 ( )  16 3 k

T (n)  3 k  1  cn 2 ( 16 )  3 k  cn 2  (1  k )
 1 3  13 16
 16 

since 4 k  n  (4 k ) 2  n 2  (4 2 ) k  n 2  16  n
k 2

16 n 2  3 k 16 16 16 k
 3  cn  (
k 2
2
)  3 k
  c c  n  (1  c)3
 ( n 2

2
3 k
)
13 n 13 13 13
16 2 16 log n 16 2 16 13
T (n)  cn  (1  c)3 4  cn  (1 
3
c)n log 4
13 13 13 13
Hence T (n)  (n 2 )
Master Method

14
Example 9

• T(n) = 4T(n/2) + n3
• a=4, b=2, f(n) = n3
• nlogb a = nlog2 4 = O(n2) where  = 1
• Since f(n) = O(nlog2 4 + ), where =1, case III applies:
• Case III if f(n)= (nlogB a + ) for some constant >0, and if
af(n/b)<=cf(n) for some constant c<1 and all sufficient large
n, then T(n) =  (f(n))
• a f(n/b)= 4(n/2)3 <=4/2n3=c f(n) for c=4/2
• af(n/b)=4/8n3<=4/2n3=c f(n)
• af(n/b)=n3/2<=2n3=c f(n) 3
f(n)= n
• Thus the solution is T(n) = (n3)
4f(n/2) = 4(n/2)3
af(n/b)=a (n/b)3
Practice Questions
• T (n) = 3T (n/2)+ n2
• T (n) = 4T (n/2)+ n
• T (n) = T (n/2) + 2n
• T (n) = T (n/2) + 1 Binary Search
• T (n) = 2nT (n/2) + n
• T (n) = 16T (n/4)+ n
• T (n) = 2T (n/2)+ n log n
• T (n) = 2T (n/4)+ n0.51
• T (n) = 0.5T (n/2)+ 1/n
• T (n) = p2T (n/2) + log n
• T (n) = 3T (n/2)+ n
• T (n) = 3T (n/3)+ pn
• T (n) = 4T (n/2)+ cn
• T (n) = 3T (n/4)+ n log n
• T (n) = 3T (n/3)+ n/2
• T (n) = 6T (n/3)+ n2 log n
• T (n) = 4T (n/2)+ n/ log n
• T (n) = 64T (n/8)− n2 log n
• T (n) = 7T (n/3)+ n2
• T (n) = 4T (n/2)+ log n 16
• T(n)=2T(n-1)+O(n)
• T(n) =3T(n/3) + O(n)
Practice Questions

17
  0 n0
s ( n)  
c  s (n  1) n  0
• s(n) =
c + s(n-1)
c + c + s(n-2)
2c + s(n-2)
2c + c + s(n-3)
3c + s(n-3)
…
kc + s(n-k) = ck + s(n-k)
  0 n0
s(n)  
c  s(n  1) n  0
• So far for n >= k we have
• s(n) = ck + s(n-k)
• What if k = n?
• s(n) = cn + s(0) = cn
  0 n0
s(n)  
c  s(n  1) n  0
• So far for n >= k we have
• s(n) = ck + s(n-k)
• What if k = n?
• s(n) = cn + s(0) = cn
• So

 0 n0
s(n)  
c  s(n  1) n  0

• Thus in general
• s(n) = cn
  0 n0
s ( n)  
n  s (n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
=…
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
  0 n0
s(n)  
n  s(n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
=…
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
n

= i
i  n  k 1
 s(n  k )
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have
n

i
i  n  k 1
 s(n  k )
  0 n0
s(n)  
n  s(n  1) n  0
• So far for n >= k we have
n

i
i  n  k 1
 s (n  k )

• What if k = n?
  0 n0
s(n)  
n  s(n  1) n  0
• So far for n >= k we have
n

i
i  n  k 1
 s (n  k )

• What if k = n?
n
n 1
n


i 1
i  s (0)   i  0  n
i 1 2
  0 n0
s(n)  
n  s(n  1) n  0
• So far for n >= k we have
n

i
i  n  k 1
 s (n  k )

• What if k = n?
n
n 1
n


i 1
i  s(0)   i  0  n
i 1 2
• Thus in general
n 1
s ( n)  n
2
  c n 1
 n
T (n)  2T
   c n 1
  2 
• T(n) =
2T(n/2) + c
2(2T(n/2/2) + c) + c
22T(n/22) + 2c + c
22(2T(n/22/2) + c) + 3c
23T(n/23) + 4c + 3c
23T(n/23) + 7c
23(2T(n/23/2) + c) + 7c
24T(n/24) + 15c
…
2kT(n/2k) + (2k - 1)c
  c n 1
 n
T (n)  2T
   c n 1
  2 
• So far for n > 2k we have
• T(n) = 2kT(n/2k) + (2k - 1)c
• What if k = lg n?
• T(n) = 2lg n T(n/2lg n) + (2lg n - 1)c
= n T(n/n) + (n - 1)c
= n T(1) + (n-1)c
= nc + (n-1)c = (2n - 1)c
  c n 1
Example T (n)    n 
aT    cn n  1
  b 
• T(n) =
aT(n/b) + cn
a(aT(n/b/b) + cn/b) + cn
a2T(n/b2) + cna/b + cn
a2T(n/b2) + cn(a/b + 1)
a2(aT(n/b2/b) + cn/b2) + cn(a/b + 1)
a3T(n/b3) + cn(a2/b2) + cn(a/b + 1)
a3T(n/b3) + cn(a2/b2 + a/b + 1)
…
akT(n/bk) + cn(ak-1/bk-1 + ak-2/bk-2 + … + a2/b2 + a/b + 1)
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So we have
• T(n) = akT(n/bk) + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)
• For k = logb n
• n = bk
• T(n) = akT(1) + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)
= akc + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)
= cak + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)
= cnak /bk + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)
= cn(ak/bk + ... + a2/b2 + a/b + 1)
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a = b?
• T(n) = cn(k + 1)
= cn(logb n + 1)
= (n log n)
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a < b?
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a < b?
• Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a < b?
• Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)
• So:

a k a k 1 a
 k 1     1 
a b k 1  1 
1  a b 
k 1

1
k
b b b a b   1 1  a b  1 a b
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a < b?
• Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)
• So:

a k a k 1 a
 k 1     1 
a bk 1  1 
1  a b 
k 1

1
k
b b b a b  1 1  a b  1 a b

• T(n) = cn ·(1) = (n)

  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?

a k a k 1 a
 k 1     1 
a b k 1  1 
  a b 
k

k
b b b a b   1
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?

a k a k 1 a
 k 1     1 
a bk 1  1 
  a b 
k

k
b b b a b  1

• T(n) = cn · (ak / bk)

  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?

a k a k 1 a
 k 1     1 
a bk 1  1 
  a b 
k

k
b b b a b  1

• T(n) = cn · (ak / bk)

= cn · (alog n / blog n) = cn · (alog n / n)
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?

a k a k 1 a
 k 1     1 
a bk 1  1 
  a b 
k

k
b b b a b  1

• T(n) = cn · (ak / bk)

= cn · (alog n / blog n) = cn · (alog n / n)
recall logarithm fact: alog n = nlog a
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?

a k a k 1 a
 k 1     1 
a bk 1  1 
  a b 
k

k
b b b a b  1
• T(n) = cn · (ak / bk)
= cn · (alog n / blog n) = cn · (alog n / n)
recall logarithm fact: alog n = nlog a
= cn · (nlog a / n) = (cn · nlog a / n)
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So with k = logb n
• T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)
• What if a > b?

a k a k 1 a
 k 1     1 
a bk 1  1 
  a b 
k

k
b b b a b  1
• T(n) = cn · (ak / bk)
= cn · (alog n / blog n) = cn · (alog n / n)
recall logarithm fact: alog n = nlog a
= cn · (nlog a / n) = (cn · nlog a / n)
= (nlog a )
  c n 1
 n
T (n)  aT
   cn n  1
  b 
• So…

 n  ab

T (n)  n log b n  ab
 
  n logb a
 ab
 Practice Questions
Substitution Method

44
Example

T(n) = 2 T[(n/2) + 17 ] + n
we neglect 17, because when n is large the difference
between T(n/2) and T[(n/2)+17) is not that large: both cut
n nearly even in half.
T(n) = 2 T(n/2) + n
Suppose T(n)<=nlgn
T(n/2)<=n/2lgn/2
T(n)= 2n/2lgn/2 +n
= nlgn-nlg2+n
=nlgn-n+n
= lgn
 Changing Variables

Sometimes a little algebraic manipulation can make an unknown

recurrence similar to one we have seen before.
Consider the following recurrence:
T(n)= 2T( n ) +lgn which looks difficult, We can simply the recurrence
with a change of variables.
Introduce two new variables. Renaming m=lgn n=2m
T(2m)= 2T(2m/2) + m
Suppose T(2m) = S(m) then T(2m/2)=s(m/2) (by placing m=m/2)
S(m) = 2S(m/2) + m
By solving it iteratively O(mlgm), Changing back from s(m)
to T(n)
T(n)=T(2m)=S(m) = O(mlgm)=O(lgn lg lgn)
Changing Variables
T(n) = 2T( n) +1

4/16/2024
n=2m  n1/2=2m/2

48
m=logn

M. Shoaib Farooq
T(2m) = 2T(2m/2) +1

T(2m)=S(m) then T(2m/2)=S(m/2) (by placing m=m/2)

S(m) = 2S(m/2) +1 which is much like recurrence

after solving iteratively O(m)
S(m) = O(m)
= O(lgn)
Recursion-tree method
• A recursion tree models the costs (time) of a
recursive execution of an algorithm.
• The recursion tree method is good for
generating guesses for the substitution method.
• The recursion-tree method can be unreliable,
just like any method that uses ellipses (…).
• The recursion-tree method promotes intuition,
however.
Recursion Tree Method
T(n) = 2T(n/3) + n.

50
Recursion Tree Method

51
52
53