ARTIFICIAL INTELLIGENCE

Chapter 1: Complexity Analysis (third part )

Montassar Ben Messaoud, PhD
Big-O,
big-Ω and
big-Θ notation
 𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)
Translation: 𝑓(𝑛) is 𝑂(𝑔 𝑛 ) if there exist two
constants 𝑐 and 𝑛0 such that for every 𝑛 greater than
or equal to 𝑛0 , 𝑓(𝑛) is smaller than or equal to 𝑐𝑔(𝑛)
𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)

𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 =𝑂 𝑛 ?
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 𝑓 𝑛 = 2𝑛 + 10
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 =𝑂 𝑛 ?
𝑔 𝑛 =𝑛
𝑐=1
𝑐𝑔 𝑛 = 𝑛
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 𝑓 𝑛 = 2𝑛 + 10
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 =𝑂 𝑛 ?
𝑔 𝑛 =𝑛
𝑐=3
𝑐𝑔 𝑛 = 3𝑛

𝑛0
𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)

𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 =𝑂 𝑛 ?
𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)

𝑔 𝑛 is an upper
bound of 𝑓(𝑛)
𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)

2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
𝑓 𝑛 =𝑂 𝑛 ?
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 =𝑂 𝑛 ?
𝑔 𝑛 =𝑛
𝑐=3
𝑐𝑔 𝑛 =
3𝑛
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 =𝑂 𝑛 ?
𝑔 𝑛 =𝑛
𝑐 = 10
𝑐𝑔 𝑛 =
10𝑛
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 =𝑂 𝑛 ?
𝑔 𝑛 =𝑛
𝑐 = 100
𝑐𝑔 𝑛 =
100𝑛
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 =𝑂 𝑛 ?
𝑔 𝑛 =𝑛
𝑐 = 1000
𝑐𝑔 𝑛 =
1000𝑛
𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)

2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
𝑓 𝑛 = 𝑂 𝑛² ?
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 = 𝑂 𝑛² ?
𝑔 𝑛 = 𝑛²
𝑐=2
𝑐𝑔 𝑛 = 2𝑛²

𝑛0
𝑓 𝑛 = 𝑂(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)
2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
𝑓 𝑛 = 𝑂 𝑛² ?
𝑛² is an upper bound
of 𝑓(𝑛)
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 = 𝑂 𝑛! ?
𝑔 𝑛 = 𝑛!
𝑐=1
𝑐𝑔 𝑛 = 𝑛!

𝑛0
 𝑛² and 𝑛! are both
upper bounds of 𝑓 𝑛 =
2
𝑛 + 10𝑛 − 6, but the
tight
one is 𝑛²
𝑂(1)
𝑂(𝑙𝑜𝑔𝑛)
𝑂(𝑛)
𝑂(𝑛𝑙𝑜𝑔𝑛)
 𝑘 3
𝑂(𝑛 ) (e.g.: 𝑂(𝑛²) and 𝑂(𝑛 ))
 𝑛 𝑛
𝑂(𝑘 ) (e.g.: 𝑂(2 ))
𝑂(𝑛!)
𝑂 1 < 𝑂 𝑙𝑜𝑔𝑛 < 𝑂 𝑛 < 𝑂 𝑛𝑙𝑜𝑔𝑛 < 𝑂 𝑛𝑘 < 𝑂 𝑘 𝑛 < 𝑂(𝑛!)
with 𝑘 > 1
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 𝑓 𝑛 = 2𝑛 + 10
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0 𝑓 𝑛 =𝑂 𝑛 ?
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)
𝑔 𝑛 =𝑛
𝑐=1
𝑐𝑔 𝑛 = 𝑛

𝑔 𝑛 =𝑛
𝑐=3
𝑐𝑔 𝑛 = 3𝑛
𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 = 𝑂(𝑛)
 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
 2
𝑓 𝑛 = 𝑛 + 10𝑛 − 6
2
𝑓 𝑛 = 𝑂(𝑛 )
 4 𝑛 2
𝑓 𝑛 = 𝑛 + 2 + 3𝑛 + 7
 4 𝑛 2
𝑓 𝑛 = 𝑛 + 2 + 3𝑛 + 7

𝑂 1 < 𝑂 𝑙𝑜𝑔𝑛 < 𝑂 𝑛 < 𝑂 𝑛𝑙𝑜𝑔𝑛 < 𝑂 𝑛𝑘 < 𝑂 𝑘 𝑛 < 𝑂(𝑛!)

with 𝑘 > 1
 4 𝑛 2
𝑓 𝑛 = 𝑛 + 2 + 3𝑛 + 7

𝑂 1 < 𝑂 𝑙𝑜𝑔𝑛 < 𝑂 𝑛 < 𝑂 𝑛𝑙𝑜𝑔𝑛 < 𝑂 𝑛𝑘 < 𝑂 𝑘 𝑛 < 𝑂(𝑛!)

with 𝑘 > 1
 4 𝑛 2
𝑓 𝑛 = 𝑛 + 2 + 3𝑛 + 7
𝑛
𝑓 𝑛 = 𝑂(2 )
𝑂 1 < 𝑂 𝑙𝑜𝑔𝑛 < 𝑂 𝑛 < 𝑂 𝑛𝑙𝑜𝑔𝑛 < 𝑂 𝑛𝑘 < 𝑂 𝑘 𝑛 < 𝑂(𝑛!)
with 𝑘 > 1
𝑓 𝑛 = 𝑂(𝑔 𝑛 ) 𝑓 𝑛 = 100𝑛²
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛) 𝑓 𝑛 = 𝑂 𝑛² ?
𝑔 𝑛 = 𝑛²
𝑐 = 110
𝑐𝑔 𝑛 =
110𝑛²

𝑛0
 2
𝑇 𝑛 = 3𝑛 + 5𝑛 + 2
 2
𝑇 𝑛 = 3𝑛 + 5𝑛 + 2

⇒
𝑔 𝑛 = 𝑛²
𝑐=4
𝑐𝑔 𝑛 = 4𝑛²

𝑛0

⇒
Time complexity: 𝑂(𝑛²)
The big-O notation is a mathematical
concept that we used in algorithms to
analyse the time and space complexity
of an algorithm, where 𝑓(𝑛) is 𝑇(𝑛),
the function that expresses the
resources usage in terms of input size
Big-Ω
notation
Big-O 𝑓 𝑛 = 𝑂(𝑔 𝑛 )
notation:
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)
Translation: 𝑓(𝑛) is 𝑂(𝑔 𝑛 ) if there exist two
constants 𝑐 and 𝑛0 such that for every 𝑛 greater than
or equal to 𝑛0 , 𝑓(𝑛) is smaller than or equal to 𝑐𝑔(𝑛)
Big-Ω 𝑓 𝑛 = Ω(𝑔 𝑛 )
notation:
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≥ 𝑐𝑔(𝑛)
Translation: 𝑓(𝑛) is Ω(𝑔 𝑛 ) if there exist two
constants 𝑐 and 𝑛0 such that for every 𝑛 greater than
or equal to 𝑛0 , 𝑓(𝑛) is greater than or equal to 𝑐𝑔(𝑛)
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≥ 𝑐𝑔(𝑛)

2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
𝑓 𝑛 = Ω 𝑛² ?
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≥ 𝑐𝑔(𝑛)

2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
𝑓 𝑛 = Ω 𝑛² ?
𝑓 𝑛 = Ω(𝑔 𝑛 ) 2
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
𝑓 𝑛 ≥ 𝑐𝑔(𝑛) 𝑓 𝑛 = Ω 𝑛² ?
𝑔 𝑛 = 𝑛²
𝑐=3
𝑐𝑔 𝑛 = 3𝑛²
𝑓 𝑛 = Ω(𝑔 𝑛 ) 2
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
𝑓 𝑛 ≥ 𝑐𝑔(𝑛) 𝑓 𝑛 = Ω 𝑛² ?
𝑔 𝑛 = 𝑛²
𝑐=1
𝑐𝑔 𝑛 = 𝑛²

𝑛0
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≥ 𝑐𝑔(𝑛)

𝑓 𝑛 = 2𝑛 − 5
𝑓 𝑛 = Ω 𝑛² ?
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 = 2𝑛 − 5
𝑓 𝑛 ≥ 𝑐𝑔(𝑛) 𝑓 𝑛 = Ω 𝑛² ?
𝑔 𝑛 = 𝑛²
𝑐=1
𝑐𝑔 𝑛 = 𝑛²
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 = 2𝑛 − 5
𝑓 𝑛 ≥ 𝑐𝑔(𝑛) 𝑓 𝑛 = Ω 𝑛² ?
𝑔 𝑛 = 𝑛²
𝑐 = 0,01
𝑐𝑔 𝑛 =
0,01𝑛²
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≥ 𝑐𝑔(𝑛)

𝑓 𝑛 = 2𝑛 − 5
𝑓 𝑛 =Ω 𝑛 ?
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 = 2𝑛 − 5
𝑓 𝑛 ≥ 𝑐𝑔(𝑛) 𝑓 𝑛 =Ω 𝑛 ?
𝑔 𝑛 =𝑛
𝑐=1
𝑐𝑔 𝑛 = 𝑛

𝑛0
𝑓 𝑛 = Ω(𝑔 𝑛 )
if ∃𝑐, 𝑛0 ∀𝑛 ≥ 𝑛0
𝑓 𝑛 ≥ 𝑐𝑔(𝑛)

𝑓 𝑛 = 2𝑛 − 5
𝑓 𝑛 =Ω 𝑛 ?
𝑛 is a lower bound of
𝑓(𝑛)
 2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3 is in Ω(1), but it's not the
tight lower bound
𝑔 𝑛 = 𝑛² is the tight lower bound of
2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
 2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
 2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
 2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
 2
𝑓 𝑛 = 2𝑛 − 5𝑛 + 3
𝑓 𝑛 = Ω(𝑛²)
1.Find T(n)
2.Deduce the
acomplexity
1.Find T(n)
2.Deduce the
acomplexity
𝑂(𝑔 𝑛 )
Ω(𝑔 𝑛 )
 best case
worst case
best case

We want to search for

num = 4
4 5 1 7 2 9 3 8
best case

We want to search for

num = 4
4 5 1 7 2 9 3 8

Time complexity: 𝑂(1)

worst case

We want to search for

num = 6
4 5 1 7 2 9 3 8
worst case

We want to search for

num = 6
4 5 1 7 2 9 3 8

Time complexity: 𝑂(𝑛)

best case
 best case

average case
 best case

average case

worst case
 tight upper bound
best case

tight upper bound

average case

tight upper bound

worst case
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case
tight lower bound
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case
tight lower bound
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound 𝑂(𝑛)

worst case
tight lower bound
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound 𝑂(𝑛)

worst case
tight lower bound Ω(𝑛)
 tight upper bound 𝑂(1)
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound 𝑂(𝑛)

worst case
tight lower bound Ω(𝑛)
 tight upper bound 𝑂(1)
best case tight lower bound Ω(1)

tight upper bound

average case
tight lower bound

tight upper bound 𝑂(𝑛)

worst case
tight lower bound Ω(𝑛)
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case
tight lower bound
Big-Θ
notation
 We say that 𝑓(𝑛) is in Θ(𝑔(𝑛)) if
𝑂(𝑔(𝑛)) is the tight upper bound of 𝑓(𝑛)
and Ω(𝑔(𝑛)) is the tight lower bound of
𝑓(𝑛), which means that when 𝑓(𝑛) is in
𝑂(𝑔(𝑛))
and Ω(𝑔(𝑛)), then we say
that it's in Θ(𝑔(𝑛))
 𝑓 𝑛 = Θ(𝑔 𝑛 )
if ∃𝑐1 , 𝑐2 , 𝑛0 ∀𝑛 ≥ 𝑛0
𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔(𝑛)

Translation: 𝑓(𝑛) is Θ(𝑔 𝑛 ) if there exist three

constants 𝑐1 , 𝑐2 , and 𝑛0 such that for every 𝑛 greater
than or equal to 𝑛0 , 𝑓(𝑛) is greater than or equal to
𝑐1 𝑔(𝑛) and smaller than or equal to 𝑐2 𝑔(𝑛)
𝑓 𝑛 = Θ(𝑔 𝑛 )
if ∃𝑐1 , 𝑐2 , 𝑛0 ∀𝑛 ≥ 𝑛0
𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔(𝑛)

𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 =Θ 𝑛 ?
𝑓 𝑛 = Θ(𝑔 𝑛 )
if ∃𝑐1 , 𝑐2 , 𝑛0 ∀𝑛 ≥ 𝑛0
𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔(𝑛)

𝑓 𝑛 = 2𝑛 + 10
𝑓 𝑛 =Θ 𝑛 ?
𝑓 𝑛 = Θ(𝑔 𝑛 )
𝑓 𝑛 = 2𝑛 + 10
if ∃𝑐1 , 𝑐2 , 𝑛0 ∀𝑛 ≥ 𝑛0
𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔(𝑛) 𝑓 𝑛 =Θ 𝑛 ?
𝑔 𝑛 =𝑛
𝑐1 = 1
𝑐2 = 3
𝑐1 𝑔 𝑛 = 𝑛
𝑐2 𝑔 𝑛 = 3𝑛

𝑛0
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound 𝑂(𝑛)

worst case
tight lower bound Ω(𝑛)
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case 𝑂(𝑛)
Θ(𝑛)
Ω(𝑛)
tight lower bound
 tight upper bound 𝑂(1)
best case tight lower bound Ω(1)

tight upper bound

average case
tight lower bound

tight upper bound

worst case 𝑂(𝑛)
Θ(𝑛)
Ω(𝑛)
tight lower bound
 tight upper bound
best case 𝑂(1)
Θ(1)
Ω(1)
tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case 𝑂(𝑛)
Θ(𝑛)
Ω(𝑛)
tight lower bound
𝑓 𝑛 = Θ 𝑛 ⇒ 𝑓 𝑛 = 𝑂(𝑛)
𝑓 𝑛 = Θ 𝑛 ⇒ 𝑓 𝑛 = 𝑂(𝑛)
𝑓 𝑛 = 𝑂 𝑛 ⇏ 𝑓 𝑛 = Θ(𝑛)
𝑓 𝑛 = Θ 𝑛 ⇒ 𝑓 𝑛 = Ω(𝑛)
𝑓 𝑛 = Θ 𝑛 ⇒ 𝑓 𝑛 = Ω(𝑛)
𝑓 𝑛 = Ω 𝑛 ⇏ 𝑓 𝑛 = Θ(𝑛)
 upper
bounds
𝑛 tight upper
𝑂(2 ) 𝑂(𝑛²) bound

3
𝑂(𝑛 ) 𝑂(𝑛)
 upper
bounds
𝑛 tight upper
𝑂(2 ) 𝑂(𝑛²) bound

3
𝑂(𝑛 ) 𝑂(𝑛)
Ω(𝑛) Ω(𝑙𝑜𝑔𝑛)
tight lower
bound Ω(𝑙𝑜𝑔𝑙𝑜𝑔𝑛) Ω(1)
lower bound
 tight upper
bound

𝑂(𝑛)
Θ(𝑛)
Ω(𝑛)
tight lower
bound
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case
tight lower bound
 tight upper bound
best case tight lower bound

tight upper bound

average case
tight lower bound

tight upper bound

worst case
tight lower bound
The big-Θ notation is more
meaningful, it gives us the
tight upper bound and the
tight lower bound
Thank you for your attention
montassar.benmessaoud@tbs.u-tunis.tn