ECS 36C

Data Structures
Spring 2024 - Instructor Siena Saltzen
 Administrative
https://canvas.ucdavis.edu/courses/877554/assignments/syllabus
● No need to email me if missing class, make your best scheduling decisions
● Discussion/OH are now up on the syllabus
● Welcome Survey is worth 2 points, up until the 13th
● HW 1 Releases tonight, you will have a week to complete it
● And a practice homework assignment - Does not have autograder, but Is listed
● SDC - Acknowledged, Keep in mind you can always take the normal offering, working
on accommodations will be handled through the SDC, including weekly Quizzes
● Discord Rules - Please remember this is a professional forum and act accordingly
● Testing Dates will remain as posted
● SLAC Night! Wed 5-7 come code together!
Common Complexities and Their Names
● We can assume that the Big-O notation gives us the
algorithm’s approximate run time in the worst case.
There is a lot more math involved in the formal definition
○ ( Luckily I will only make you do some of it…)

● When using the Big-O notation, we describe the

algorithm’s efficiency based on the increasing size of the
input data (n).

● We will keep talking about algorithms efficiency like this

through the class while showing you some of the flagship
examples.
Visual Representation
Fib Cont
How long does this take? A second? A minute? It depends on n So
we’ll approximate the runtime as a function of the value of input, n.
There are lots of factors we could consider:
● What machine is it running on?
● What language is it written in?
● What compiler was used?
● How was it programmed?
But we want a basis for comparison that’s independent of these
implementation details. Consequently, we want to count just “basic
operations” like arithmetic, and memory access, while ignoring the
details given above.
How many operations when this algorithm is run? 2 going
into the loop
4 each time the loop is executed
the loop is executed n - 2 times
1 returning from function
2 + (4)(n-2) + 1
If we’re ignoring details, does it make sense to be so
precise? It’s educational now, but later we’ll see that we
can do this more simply and ignore the details.
 Run as fun() of input
The run time of iterative Fibonacci is (depending on the details of how we count and our
implementation):
2 + (4)(n - 2) + 1
which simplifies to
4n - 5
Since we’ve abstracted away exactly how long the different operations take, and on
what computer we’re running, does it make sense to say 4n - 5 instead of 4n - 6 or 5n -
3 or ...? What matters here is n. As n gets bigger, the run time grows linearly in
proportion to n. (We’ll formalize this in the very near future.) The 4 and the 5 are just
details
What if there are many possible inputs, as in the case of linear search in a
list? Does that change anything?

What if the item is the first in the list? The last in the list? Not in the list?
Which run time should we report?
There are different kinds of analysis we could report:
• Best Case
• Worst Case
• Average Case Amortized:
In accounting, amortization refers to expensing the acquisition cost minus the residual value of intangible assets in a systematic

• Common Case manner over their estimated "useful economic lives" so as to reflect their consumption, expiry, and obsolescence, or other
decline in value as a result of use or the passage of time. Wikipedia

In computer science and algorithms, amortized analysis is a technique used to estimate the average time complexity of an
• Amortized algorithm over a sequence of operations, rather than the worst-case complexity of individual operations.

For example, for a dynamic array that doubles in size when needed, normal asymptotic analysis would only conclude that adding
• and so on.. an item to it costs O(n), because it might need to grow and copy all elements to the new array. Amortized analysis takes into
account that in order to have to grow, n/2 items must have been added without causing a grow since the previous grow, so
adding an item really only takes O(1) (the cost of O(n) is amortized over n/2 actions).
 Big-O notation

Going back to the iterative Fibonacci example, we calculated the function for the run
time to be 4n - 5. Then we argued that the 4 and the 5 are noise for our purposes.
One way to say all that is “The running time for the Fibonacci algorithm finding the
nth Fibonacci number is on the order of n.”
In Big-O notation, that’s just T(n) is O(n)
T(n) is our shorthand for the runtime of the function being analyzed.
The O in O(n) means “order of magnitude”, so Big-O notation is clearly, and
intentionally, not precise. It’s a formal notation for the approximation of the time (or
space) requirements of running algorithms.
 Formal Definition

There’s a formal mathematical definition for Big-O that we’re obliged to

discuss:

T(n) is O(f(n)) if there are two positive constants, n0 and c, and a

function f(n) such that cf(n) >= T(n) for all n > n0
In other words, as n gets sufficiently large (larger than n0), there is some
constant c for which the processing time will always be less than or equal
to cf(n), so cf(n) is an upper bound on the performance. The performance
will never be worse than cf(n) and may be better.

positive means greater than zero...it does not include zero

https://www.freecodecamp.org/news/big-o-notation-why-it-matters-and-why-it-doesnt-1674cfa8a23c/
One more time. Here’s what this

T(n) is O(f(n)) if there are two positive constants, n0 and c, and a function f(n)
such that cf(n) >= T(n) for all n > n0
really means in a practical sense:

If you want to show that T(n) is O(f(n)) then find two positive constants, n0 and c,
and a function f(n) that satisfy the constraints above. For example...
 Big-O arithmetic

For our iterative Fibonacci example, T(n) = 4n - 5. We

want to show that T(n) is O(n). So we set up our inequality
like this:
4n - 5 <= cn
For our iterative Fibonacci example, T(n) = 4n - 5. We
want to show that T(n) is O(n). Then we do some algebra
to isolate n:
4n - 5 <= cn
4n <= cn + 5
n <= cn/4 + 5/4
 Now pick some value for c. Let’s use 4 to cancel the denominator.
n <= 4n/4 + 5/4
n <= n + 5/4
c=4
Great.
Now we pick some value for n0, substitute it for n, and see if the inequality holds.

* Desmos
Hmmm, let’s try 1.
1 <= 1 + 5/4

1 is always less than or equal to 1 + 5/4, so in choosing c = 4 and n0 = 1,

we have shown that 4n - 5 is O(n)

because 4n - 5 <= 4n for n >= 1. T(n) is O(n). Yippee!

This is an easy example, but what we expect you to do on your homework.

For the first section: For each problem in code, find T(n).

The same way we did for the fibonacci sequence and search examples.

Then decide and prove O(n) by providing C and n0.

Let do another Example to find T(n):

Okay. Next Steps again:
Examining some code, we determine that T(n) = 3n^2 + 6n and we think
that T(n) is O(n^2). How do we prove it?
3n^2 + 6n <= cn^2
3n (n + 2) <= cn^2
n + 2 <= cn/3
n <= cn/3 - 2
 n <= cn/3 - 2
Now we let c = 3 to cancel the denominator. That gives us
n <= n - 2. That won’t work.
Can we pick another c? Remember proof is “a” c exists that allows the inequality.
Let’s try c = 6, a multiple of 3,
that will also simplify things by cancelling the denominator.
That gives n <= 2n - 2. That’s much more promising...
 3n^2 + 6n <= cn^2
3n (n + 2) <= cn^2
n + 2 <= cn/3
n <= cn/3 - 2
n <= 2n - 2 with c = 6
If we let n0 = 1 and substitute it for n, we get
1 <= 2 - 2
1 <= 0
That won’t do.
 3n^2 + 6n <= cn^2
3n (n + 2) <= cn^2
n + 2 <= cn/3
n <= cn/3 - 2
n <= 2n - 2 with c = 6
If we let n0 = 2 and substitute it for n, we get
2 <= 4 - 2
2 <= 2
That will do nicely.
 Recap
Examining some code, we determine that T(n) = 3n2 + 6n and we think that T(n) is
O(n2). How do we prove it?
3n2 + 6n <= cn2
3n (n + 2) <= cn2
n + 2 <= cn/3
n <= cn/3 - 2
n <= 2n - 2 with c = 6
Choosing c = 6 and n0 = 2, we have shown that 3n2 + 6n is O(n2) because 3n2 + 6n
<= 6n2 for n >= 2.
But wait! This is a little bit misleading, as was the previous example. Are we sure that
this works for all n >= 2?

It is possible that you find values for n and c where it all appears to work, but if you
then try out larger values of n and the inequality no longer holds. Your proof isn't a
proof.

So in this class, make sure that, once you have found a working n and c, you try some
larger values of n. There's an example of how you might be deceived coming up.
The Big-O definition simply(?) says that there is a point n0
such that for all values of n that are past this point, T(n) is
bounded by some multiple of f(n). Thus, if the running time
T(n) of an algorithm is O(n^2), we are guaranteeing that at
some point we can bound the running time by a quadratic
function (a function whose high-order term involves n^2).
Thus, if the running time T(n) of an algorithm is O(n2),

we are guaranteeing that at some point we can bound the running time by a
quadratic function (a function whose high-order term involves n2).

Big-O says there’s a function that is an upper bound to the worst-case performance
for the algorithm.
Note however that if T(n) is linear and not quadratic, you
could still say that the running time is O(n^2).

It’s technically correct because the inequality holds. However, O(n) would
be the more precise claim because it’s an even lower upper bound.

Desmos Example
Big-O is for expressing how run time or memory requirements grow as a function of the
problem size. Your book* has a nice table listing commonly-encountered rates.
O(1) Constant
O(log n) Logarithmic
O(n) Linear
O(n log n) Log-linear
O(n^2) Quadratic
O(n^3) Cubic
O(n^k) Polynomial – k is constant
O(2^n) Exponential
O(n!) Factorial
https://alyssaq.github.io/stl-complexities/
Recap

Asymptotic analysis means that we’re studying the behavior of

functions f(n) as n gets very large, approaching ∞. Asymptotic analysis
is based on the idea that as the problem size grows, the complexity
can be described as a simple proportionality to some known function.
This idea is incorporated in the Big-O notation for asymptotic
performance. When we say T(n) is O(n^2), we mean that the growth of
the run time is proportional to n^2, and that there is at least one
specific function, cn^2, that is an upper bound to the growth of the run
time.
What exactly are we trying to do?

O(f(n)) is the set of functions with smaller or same

order of growth as f(n) ...and so on...

For any f(n) with run time T(n), we’re trying to

categorize the run time behavior by placing it in
the smallest possible group that contains a
function that is an upper bound on T(n)
However, there may be a limited range of values of n for which the
O(n2) algorithm gives better performance than the O(n lg n) algorithm.
Constants

That’s the story of constants.

When we’re analyzing space or time complexity, we disregard the

constants. That doesn’t mean the constants aren’t important if we
want to talk about real performance (e.g., clock time, CPU time, actual
memory usage), but when we want to talk just about the complexity
inherent in the algorithm, regardless of the details, constants are one of
the details we ignore
Low Order Terms

Let’s say we’ve calculated the run time for some bit of code as

T(n) = 42n^3 + 3n^2 - 17n - 8

We know we can disregard the constants:

T(n) = n^3 + n^2 - n But we don’t say that T(n) is O(n^3 + n^2 - n).

Why?(Bonus)
But we don’t say that T(n) is O(n^3 + n^2 - n).

Why?

Because as n gets very big, the growth of n^3 dominates the growth of the other terms
– the low-order terms – and those terms have no noticeable impact as n gets very big.
Low-order terms

So if the run time is this: T(n) = 42n^3 + 3n^2 - 17n - 8 Then T(n) is O(n^3) because the
dominant or high-order term is O(n^3).
● Desmos example
 Order or Dominance
The “order of dominance” is in this same table. Read from the bottom up. O(n!)
dominates O(2^n). O(2^n) dominates O(n^k). And so on...
Big-O name
O(1) Constant
O(log n) Logarithmic
O(n) Linear
O(n log n) Log-linear
O(n^2) Quadratic
O(n^3) Cubic
O(n^k) Polynomial – k is constant
O(2^n) Exponential
O(n!) Factorial
 Analysing Code
• Basic operations - constant time
• consecutive stmts - sum of times
• conditionals - sum of branches + condition
• loops - sum of iterations
• function calls - cost of function body*

Above all, use your head!

*These can be sneaky

Big Omega

Big-Ω
What if you want to find a lower bound to the performance of an algorithm? That’s
where Big-Ω (Big-Omega) comes in.
Big-Ω says there’s a function that is a lower bound to the worst case performance
for the algorithm. In other words, the algorithm’s performance will never get
better than that lower bound.
 Formal Definition
The formal mathematical definition for Big-Ω:
T(n) is Ω(f(n)) if there are two constants, n0 and c, both > 0, and a function
f(n) such that cf(n) <= T(n) for all n > n0
If T(n) is Ω(f(n)), then we say “T(n) is at least order of f(n)” or “T(n) is bound
from below by f(n)” or “T(n) is Big-Omega of f(n)”.

For example, searching for an item in an unsorted array of n elements must

be Ω(n) in time complexity because we must look at all n elements in the
worst case.
 Big Theta

Big-Θ
If we know that T(n) is both O(f(n)) and Ω(f(n)), then we may say that T(n) is Θ(f(n)) (i.e., Big-Theta of f(n)).
The formal mathematical definition for Big-Θ:
T(n) is Θ(f(n)) if and only if T(n) is O(f(n)) and T(n) is Ω(f(n)).
This says that the growth rate of T(n) equals the growth rate of f(n). If we say that T(n) is Θ(n2), we’re
saying that T(n) is bounded above and below by a quadratic function.
It won’t get worse than the upper bound, and it can’t get better than the lower bound. Big-Θ assures the
tightest possible bound on the algorithm’s performance.
 Wait. Aren’t These all the same?
Big O notation (O):
It is defined as upper bound and upper bound on an algorithm is the most amount of time required ( the worst
case performance).
Big O notation is used to describe the asymptotic upper bound. An algorithm will take at MOST a certain order
of time.
Big Omega notation (Ω) :

It is define as lower bound and lower bound on an algorithm is the least amount of time required. In other words
an algorithm takes AT LEAST a certain amount of time.

Just like O notation provide an asymptotic upper bound, Ω notation provides asymptotic lower bound.

Big Theta notation (Θ) :

It is define as tightest bound and tightest bound is the best of all the worst case times that the algorithm can
take.
Big O

https://www.geeksforgeeks.org/difference-between-big-oh-big-omega-and-big-theta/
Big Ω

https://www.geeksforgeeks.org/difference-between-big-oh-big-omega-and-big-theta/
Big Θ

https://www.geeksforgeeks.org/difference-between-big-oh-big-omega-and-big-theta/
S. Big O Big Omega (Ω) Big Theta (Θ)
No
.

1. It is like (<=) It is like (>=) It is like (==)

rate of growth of an algorithm is less than or equal rate of growth is greater than or equal to a meaning the rate of growth is equal to a
to a specific value. specified value. specified value.

2. The upper bound of algorithm is represented by The algorithm’s lower bound is represented by The bounding of function from above and
Big O notation. Only the above function is Omega notation. The asymptotic lower bound is below is represented by theta notation. The
bounded by Big O. Asymptotic upper bound is given by Omega notation. exact asymptotic behavior is done by this
given by Big O notation. theta notation.

3. Big O – Upper Bound Big Omega (Ω) – Lower Bound Big Theta (Θ) – Tight Bound

4. It is define as upper bound and upper bound on It is define as lower bound and lower bound on It is define as tightest bound and tightest
an algorithm is the most amount of time required ( an algorithm is the least amount of time bound is the best of all the worst case times
the worst case performance). required ( the most efficient way possible, in that the algorithm can take.
other words best case).

5. Mathematically: Big Oh is 0 <= f(n) <= Cg(n) for Mathematically: Big Omega is 0 <= Cg(n) <= Mathematically – Big Theta is 0 <= C2g(n)
all n >= n0 f(n) for all n >= n0 <= f(n) <= C1g(n) for n >= n0
 Example
Show that T(n) = 48n + 8 is Θ(n):
for O(n):
48n + 8 <= cn
8(6n + 1) <= cn
6n + 1 <= cn/8
6n <= cn/8 – 1
n <= cn/48 – 1/6
n <= n – 1/6? (if c = 48) no
n <= 2n – 1/6? (if c = 96) maybe
1 <= 2 – 1/6? (if n0 = 1) yes
T(n) is O(n)
for Ω(n):
48n + 8 >= cn
8(6n + 1) >= cn
6n + 1 >= cn/8
6n >= cn/8 – 1
n >= cn/48 – 1/6
n >= n – 1/6? (if c = 48) maybe
1 >= 1 – 1/6? (if n0 = 1) yes
T(n) is Ω(n) and T(n) is O(n) so T(n) is Θ(n).
 A Note

We write this: T(n) is O(f(n))

or this: T(n) ∈ O(f(n))
but not this: T(n) = O(f(n))
because O(f(n)) describes a set of functions (think back to the slides with
the nested ovals)
Some books use = to mean is or ∈ in this context, but it’s not technically
correct.
I will not dock points for this in class, but you should be aware of proper
mathematical statements.
 Need more review?

I personally really like the descriptions on this website. They reference some algorithms
we haven’t covered yet but will.

https://www.freecodecamp.org/news/big-o-notation-why-it-matters-and-why-it-doesn
t-1674cfa8a23c/

It also goes more in depth for those of you who are interested.
Quick Demo!

Time Your Code! Timing.py In Files