L4+ +Characterizing+Running+Times
L4+ +Characterizing+Running+Times
Times
CSC 204 - Algorithm Design and Analysis
Corey Elowsky
Asymptotic Analysis
2
n + 50 n+100 grows asymptotically the same asn
2
Asymptotic Analysis
2
100 10,000 1,000,000 100
100 n
1,000 1,000,000 100,000,000 100
2 3
As the input size (n) becomes very large
n n n Ratio
the functional dependency on n
10 100 1,000 10 does matter!!!
50 2,500 125,000 50
3 2
1,000 1,000,000 1,000,000,000 1,000
n grows asymptotically faster than n
1,000,000,000,0
10,000 100,000,000 10,000
00
1,000,000,000,000,
100,000 10,000,000,000 000
100,000
Asymptotic Analysis
• recall that the worst-case running time of insertion sort was this complicated expression
c5 c6 c7 2 c5 c6 c7
T (n)=( + + )n +( c 1+ c 2+ c 4 + − − + c 8) n−( c 2+ c 4 + c 5 +c 8 )
2 2 2 2 2 2
• as becomes very large (asymptotically), the lower-order linear and constant terms
have negligible effects relative to the quadratic term, so they can be discarded
c5 c6 c7 2
T (n) ≈ ( + + )n
2 2 2
• we can also ignore coefficients as becomes very large
How can we
formalize this??
2
T (n) ≈ n
Asymptotic Notation - Big O
• -notation characterizes an upper bound
on the asymptotic behavior of a function
• pronounced “big-oh of ”
4
O(n )
3
3 n +4 n − 8 n +5
l g (n)
4
2
100 100 n +n l o g (n)
f (n)=O ( g( n))
Asymptotic Notation
• How can we state that a function is in ?
f (n) ∈O ( g (n))
f (n) i s a n e l e m e n t o f O ( g (n))
f (n) i s O ( g (n))
f (n)=O ( g( n))
Asymptotic Notation - Big O
• How can we formally prove that a
function is in ?
f (n)=O ( g( n))
Asymptotic Notation - Big O
Prove is
2
n
3 n +4
for all
Method 1 - Algebraic
+ - +
2
n +4 ≤ n
−3 n − 4 ≥ 0 -1 4
− 4 )( n +1 )≥ 0l e t c=1 a n d n0= 4
n
for all
4 a n d n=−1 t
herefore
Asymptotic Notation - Big O 7n
2
3 n +4
Prove is
for all
Method 2
2 2 2
n +4 ≤ 3 n + 4 n =7 n n≥1
l e t c=7 a n d n0 =1
for all
t
herefore
Asymptotic Notation - Big Ω
• Ω-notation characterizes an lower
bound on the asymptotic behavior of a
function
• pronounced “big-omega of ”
4
Ω(n )
5 2
3 n −n n! 2n
4
9
n −3 100 n +n l o g (n)
f (n)=Ω ( g (n))
Asymptotic Notation - Big Ω
• How can we formally prove that a
function is in ?
f (n)=Ω ( g (n))
Asymptotic Notation - Big Ω 2
3 n −n−8
n
Prove is
for all
Method 1 - Algebraic
2 2 ± √ 100
3 n −n −8 n=
6
2
−2 n −8 ≥ 0 n=2 a n d n=
−4
3
l e t c =1 a n d n0= 2
n=−(−2)± √ ¿¿¿ + - +
for all
therefore
-4/3 2
Asymptotic Notation - Big 𝚯
• 𝚯-notation characterizes a tight bound
on the asymptotic behavior of a function
• pronounced “big-theta of ”
4
Θ(n )
4 2
3 n −n 4
10000 n −8 n+ 9
4
9n
f (n)=Θ ( g( n))
Asymptotic Notation - Big 𝚯
• How can we formally prove that a function is in ?
for all
f (n)=Θ ( g( n))
proof
8 7
5 n −17 n ≥ 0 8 4 8 4
6 n −5 n −12 n ≤ 6 n −5 n + 3 n −12 n+8
2
remove subdominant terms with positive coefficient
8 7 8 7 7
5 n ≥ 17 n 6 n −5 n −12 n ≤ increase exponents of subdominant negative terms
5 n ≥ 17 8
6 n −17 n =¿
7 combine like terms
8 8 7
17 n +(5 n −17 n )=¿ split leading term
n≥ =3.4
5 8 solve for when subdominant terms are ≥ 0
n ≤
l e t c =1 a n d n0= 4
for all
therefore
Asymptotic Notation - Big 𝚯
Prove is
proof
8 4 2 8 2
6 n −5 n + 3 n −12 n+8 ≤ 6 n +3 n + 8 remove subdominant terms with negative coefficient
8 8 8
≤ 6 n +3 n +8 n increase exponents of subdominant positive terms
l e t c=17 a n d n0 =1
for all
therefore
Asymptotic Notation - Big 𝚯
Prove is
c 1= 1 , c 2 =17 , n0 = 4
for all
Asymptotic Notation - Induction
Proof Prove is
Need to show for
and so
2¿ 2 ≤
k +
⋅
¿ 2 2≤⋅
k 1
⋅
2 2(kk+
≤!1(
)k⋅ k+
! 1) !
1 k
t c =1 a n d n0= 4
for all
therefore
Asymptotic Notation - Limit Proof
Prove is d 1
l n ( x )=
dx x
¿ lim ¿ ¿
3
lim l o g (n)
n→∞
n n→∞
Since the limit of the ration is
equal to 0 this means that the
function in the denominator grows
¿ lim 3 ¿ ¿ ¿ ¿ lim 3 ¿ ¿
asymptotically faster than the
function in the numerator, so
therefore
n→∞ n→∞
is
6 l n(n) lim 6 l n ( n )
lim
n→∞ n ¿
n→∞
¿ n This proof can easily be
1 generalized to show that
6 lim 6 is
lim
n→∞ n ¿ 0
n→∞
¿ ¿
1 n
COOL DESMOS DEMO
Asymptotic Notation
Each of the three asymptotic notations can be used for any case of the
algorithms (best, worse, average, etc…), since they characterize functions
in general. It just so happens we are using them to characterize functions
that represent running times. It is always best to use the asymptotic
notation that is as precise as possible.
Asymptotic Notation
4 2
3 n +4 10 n −8 n 5 n + 8 n−4 l o g (n)+2n 8 n!
l o g (n )
2 2 2
Ω(n ) Θ(n ) O(n )
Asymptotic Notation
4
10 n −8 n
Ω ( 1) Θ (1) O (1)
Ω ( n) Θ ( n) O ( n)
2 2 2
Ω(n ) Θ(n ) O(n )
4 4 4
Ω(n ) Θ(n ) O(n )
n n n
Ω(2 ) Θ (2 ) O (2 )
Ω ( n !) Θ ( n!) O ( n!)
Asymptotic Notation
Give the most accurate and precise asymptotic classification of the
following functions…
n
n l o g n +5 n !+ 2 i s Θ ( n!)
8 3
2 n + 11l o g (n)+100 n i s Θ(n )
8
5 ¿ n
Θ (2 )
Asymptotic Analysis - Insertion sort
• Recall the worst-case running time of insertion sort
c5 c6 c7 2 c5 c6 c7
T (n)=( + + )n +( c 1+ c 2+ c 4 + − − + c 8) n−( c 2+ c 4 + c 5 +c 8 )
2 2 2 2 2 2
2
Θ(n )
• is also another commonly used characterization since big-O notation is
commonly used for worst-case running times, and for worst-case running
times, we usually are only concerned with establishing an upper bound.
However is more precise!
Asymptotic Analysis - Insertion sort
• Recall the best-case running time of insertion sort
T (n)=( c1 + c 2+ c 4 + c 5 +c 8 ) n−(c 2 +c 4 +c 5 + c 8)
Θ ( n)
Asymptotic Analysis
• Which statements are valid?
1. The worst-case running time of insertion sort is
1 result = 0
2 for i = 1 to n Θ ( n)
3 i=i+1
func(n)
1 for i = 1 to n
3
Θ(n )
2 for j = 1 to n
3 for k = 1 to n
4 z= i+j+k
Some Practice
func(n)
Θ (l o g 2 (n))
1 i=n
2 while i > 0
3 i=i/2
or
4 i =⌊i ⌋
Θ (l o g (n))
Homework Prep