data structures and algorithms
data structures and algorithms
Examples of algorithms:
Finding the fastest route in a GPS navigation system
The algorithms we will look at in this tutorial are designed to solve specific problems, and are often
made to work on specific data structures. For example, the 'Bubble Sort' algorithm is designed to sort
values, and is made to work on arrays.
Data structures and algorithms (DSA) go hand in hand. A data structure is not worth much if you
cannot search through it or manipulate it efficiently using algorithms, and the algorithms in this
tutorial are not worth much without a data structure to work on.
DSA is about finding efficient ways to store and retrieve data, to perform operations on data, and to
solve specific problems.
To understand how to approach complex problems and solve them in a systematic way.
For planning routes, like in a GPS system to find the shortest path from A to B.
For optimizing processes, such as arranging tasks so they can be completed as quickly as
possible.
2
For solving complex problems: From finding the best way to pack a truck to making a
computer 'learn' from data.
Applications of DSA
Operating Systems
Database Systems
Web Applications
Machine Learning
Video Games
Cryptographic Systems
Data Analysis
Search Engines
As we go along in this tutorial, new theoretical concepts and terminology (new words) will be needed
so that we can better understand the data structures and algorithms we will be working on.
These new words and concepts will be introduced and explained properly when they are needed, but
here is a list of some key terms, just to get an overview of what is coming:
Term Description
Data Structure A way of organizing data so it can be used efficiently. Common data structures include a
and binary trees.
3
Time Complexity A measure of the amount of time an algorithm takes to run, depending on the amount of
is working on.
Space Complexity A measure of the amount of memory an algorithm uses, depending on the amount of da
working on.
Big O Notation A mathematical notation that describes the limiting behavior of a function when the
towards a particular value or infinity. Used in this tutorial to describe the time complexi
Divide and Conquer A method of solving complex problems by breaking them into smaller, more manageab
solving the sub-problems, and combining the solutions. Recursion is often used when usin
algorithm.
Brute Force A simple and straight forward way an algorithm can work by simply trying all possible s
choosing the best one.
Where to Start?
In this tutorial, you will first learn about a data structure with matching algorithms, before moving on
to the next data structure.
Further into the tutorial the concepts become more complex, and it is therefore a good idea to learn
DSA by doing the tutorial step-by-step from the start.
And as mentioned on the previous page, you should be comfortable in at least one of the most
common programming languages, like for example JavaScript, C or Python, before doing this tutorial.
On the next page we will look at two different algorithms that prints out the first 100 Fibonacci
numbers using only primitive data structures (two integer variables). One algorithm uses a loop, and
one algorithm uses something called recursion.
4
A Simple Algorithm
Fibonacci Numbers
The Fibonacci numbers are very useful for introducing algorithms, so before we continue, here is a
short introduction to Fibonacci numbers.
The Fibonacci numbers are named after a 13th century Italian mathematician known as Fibonacci.
The two first Fibonacci numbers are 0 and 1, and the next Fibonacci number is always the sum of the
two previous numbers, so we get 0, 1, 1, 2, 3, 5, 8, 13, 21, ...
Create
1
5
This tutorial will use loops and recursion a lot. So before we continue, let's implement three different
versions of the algorithm to create Fibonacci numbers, just to see the difference between
programming with loops and programming with recursion in a simple way.
How it works:
a. Add the two previous numbers together to create a new Fibonacci number.
Loops vs Recursion
To show the difference between loops and recursion, we will implement solutions to find Fibonacci
numbers in three different ways:
It can be a good idea to list what the code must contain or do before programming it:
Update the variables that hold the previous two fibonacci numbers
6
Example
prev2 = 0
prev1 = 1
print(prev2)
print(prev1)
print(newFibo)
prev2 = prev1
prev1 = newFibo
To implement the Fibonacci algorithm we need most of the same things as in the code example
above, but we need to replace the for loop with recursion.
To replace the for loop with recursion, we need to encapsulate much of the code in a function, and we
need the function to call itself to create a new Fibonacci number as long as the produced number of
Fibonacci numbers is below, or equal to, 19.
Example
print(0)
print(1)
count = 2
global count
print(newFibo)
prev2 = prev1
prev1 = newFibo
count += 1
fibonacci(prev1, prev2)
else:
return
fibonacci(1,0)
Run Example »
To find the nnth Fibonacci number we can write code based on the mathematic formula for Fibonacci
number nn:
F(n)=F(n−1)+F(n−2)F(n)=F(n−1)+F(n−2)
This just means that for example the 10th Fibonacci number is the sum of the 9th and 8th Fibonacci
numbers.
Note: This formula uses a 0-based index. This means that to generate the 20th Fibonacci number, we
must write F(19)F(19).
When using this concept with recursion, we can let the function call itself as long as nn is less than, or
equal to, 1. If n≤1n≤1 it means that the code execution has reached one of the first two Fibonacci
numbers 1 or 0.
Example
def F(n):
if n <= 1:
return n
else:
print(F(19))
Notice that this recursive method calls itself two times, not just one. This makes a huge difference in
how the program will actually run on our computer. The number of calculations will explode when we
increase the number of the Fibonacci number we want. To be more precise, the number of function
calls will double every time we increase the Fibonacci number we want by one.
To better understand the code, here is how the recursive function calls return values so
that F(5)F(5) returns the correct value in the end:
9
There are two important things to notice here: The amount of function calls, and the amount of times
the function is called with the same arguments.
So even though the code is fascinating and shows how recursion work, the actual code execution is
too slow and ineffective to use for creating large Fibonacci numbers.
Summary
Recursion and loops are two different programming techniques that can be used to
implement algorithms.
It is time to move on to the first data structure we will look at, the array.
print(0)
print(1)
count = 2
global count
print(newFibo)
prev2 = prev1
prev1 = newFibo
count += 1
(prev1, prev2)
else:
return
fibonacci(1,0)
10
Submit Answer »
11
DSA Arrays
Arrays
An array is a data structure used to store multiple elements.Arrays are used by many algorithms.
For example, an algorithm can be used to look through an array to find the lowest value, like the
animation below shows:
Speed:
Find Lowest
Lowest value:
Note: The Python code above actually generates a Python 'list' data type, but for the scope of this
tutorial the 'list' data type can be used in the same way as an array. Learn more about Python
lists here.
Arrays are indexed, meaning that each element in the array has an index, a number that says where in
the array the element is located. The programming languages in this tutorial (Python, Java, and C) use
zero-based indexing for arrays, meaning that the first element in an array can be accessed at index 0.
In Python, this code use index 0 to write the first array element (value 7) to the console:
Example
Python:
print( my_array[0] )
Run Example »
Let's create our first algorithm using the array data structure.
How it works:
2. Check if the current value is the lowest so far, and if it is, store it.
12
3. After looking at all the values, the stored value will be the lowest of all values in the array.
Try the simulation below to see how the algorithm for finding the lowest value works (the animation
is the same as the one on the top of this page):
Speed:
Find Lowest
Lowest value:
This next simulation also finds the lowest value in an array, just like the simulation above, but here we
can see how the numbers inside the array are checked to find the lowest value:
7,
12,
9,
11,
Implementation
Before implementing the algorithm using an actual programming language, it is usually smart to first
write the algorithm as a step-by-step procedure.
If you can write down the algorithm in something between human language and programming
language, the algorithm will be easier to implement later because we avoid drowning in all the details
of the programming language syntax.
1. Create a variable 'minVal' and set it equal to the first value of the array.
3. If the current element has a lower value than 'minVal', update 'minVal' to this value.
4. After looking at all the elements in the array, the 'minVal' variable now contains the lowest
value.
You can also write the algorithm in a way that looks more like a programming language if you want to,
like this:
Note: The two step-by-step descriptions of the algorithm we have written above can be called
'pseudocode'. Pseudocode is a description of what a program does, using language that is something
between human language and a programming language.
After we have written down the algorithm, it is much easier to implement the algorithm in a specific
programming language:
Example
Python:
minVal = i
Run Example »
When exploring algorithms, we often look at how much time an algorithm takes to run relative to the
size of the data set.
14
In the example above, the time the algorithm needs to run is proportional, or linear, to the size of the
data set. This is because the algorithm must visit every array element one time to find the lowest
value. The loop must run 5 times since there are 5 values in the array. And if the array had 1000
values, the loop would have to run 1000 times.
Try the simulation below to see this relationship between the number of compare operations needed
to find the lowest value, and the size of the array.
See this page for a more thorough explanation of what time complexity is.
Each algorithm in this tutorial will be presented together with its time complexity.
Random
Descending
Ascending
10 Random
Operations: 0
Run Clear
DSA Exercises
Exercise:
print(my_array[])
Bubble Sort
Bubble Sort is an algorithm that sorts an array from the lowest value to the highest value.
Speed:
Bubble Sort
Run the simulation to see how it looks like when the Bubble Sort algorithm sorts an array of values.
Each value in the array is represented by a column.
The word 'Bubble' comes from how this algorithm works, it makes the highest values 'bubble up'.
How it works:
2. For each value, compare the value with the next value.
3. If the value is higher than the next one, swap the values so that the highest value comes last.
4. Go through the array as many times as there are values in the array.
Continue reading to fully understand the Bubble Sort algorithm and how to implement it yourself.
Before we implement the Bubble Sort algorithm in a programming language, let's manually run
through a short array only one time, just to get the idea.
Step 2: We look at the two first values. Does the lowest value come first? Yes, so we don't need to
swap them.
Step 3: Take one step forward and look at values 12 and 9. Does the lowest value come first? No.
7,
12,
9,
11,
We must understand what happened in this first run through to fully understand the algorithm, so
that we can implement the algorithm in a programming language.
Can you see what happened to the highest value 12? It has bubbled up to the end of the array, where
it belongs. But the rest of the array remains unsorted.
So the Bubble Sort algorithm must run through the array again, and again, and again, each time the
next highest value bubbles up to its correct position. The sorting continues until the lowest value 3 is
17
left at the start of the array. This means that we need to run through the array 4 times, to sort the
array of 5 values.
And each time the algorithm runs through the array, the remaining unsorted part of the array
becomes shorter.
Bubble Sort
7,
12,
9,
11,
We will now use what we have learned to implement the Bubble Sort algorithm in a programming
language.
2. An inner loop that goes through the array and swaps values if the first value is higher than the
next value. This loop must loop through one less value each time it runs.
3. An outer loop that controls how many times the inner loop must run. For an array with n
values, this outer loop must run n-1 times.
Example
n = len(my_array)
for i in range(n-1):
for j in range(n-i-1):
18
Run Example »
Imagine that the array is almost sorted already, with the lowest numbers at the start, like this for
example:
In this case, the array will be sorted after the first run, but the Bubble Sort algorithm will continue to
run, without swapping elements, and that is not necessary.
If the algorithm goes through the array one time without swapping any values, the array must be
finished sorted, and we can stop the algorithm, like this:
Example
n = len(my_array)
for i in range(n-1):
swapped = False
for j in range(n-i-1):
swapped = True
if not swapped:
break
Run Example »
For a general explanation of what time complexity is, visit this page.
For a more thorough and detailed explanation of Bubble Sort time complexity, visit this page.
The Bubble Sort algorithm loops through every value in the array, comparing it to the value next to it.
So for an array of nn values, there must be nn such comparisons in one loop.
And after one loop, the array is looped through again and again nn times.
This means there are n⋅nn⋅n comparisons done in total, so the time complexity for Bubble Sort is:
O(n2)––––––––––––––O(n2)__
The graph describing the Bubble Sort time complexity looks like this:
As you can see, the run time increases really fast when the size of the array is increased.
Luckily there are sorting algorithms that are faster than this, like Quicksort, that we will look at later.
You can simulate Bubble Sort below, where the red and dashed line is the theoretical time
complexity O(n2)O(n2). You can choose a number of values nn, and run an actual Bubble Sort
implementation where the operations are counted and the count is marked as a blue cross in the plot
below. How does theory compare with practice?
20
Random
Worst Case
Best Case
10 Random
Operations: 0
Run Clear
DSA Exercises
Exercise:
[7,14,11,8,9]
How does the array look like after the FIRST run through?
[,,,,]
Submit Answer »
DSA Arrays
❮ PreviousNext ❯
21
Arrays
For example, an algorithm can be used to look through an array to find the lowest value, like the
animation below shows:
Speed:
Find Lowest
Lowest value:
Note: The Python code above actually generates a Python 'list' data type, but for the scope of this
tutorial the 'list' data type can be used in the same way as an array. Learn more about Python
lists here.
Arrays are indexed, meaning that each element in the array has an index, a number that says where in
the array the element is located. The programming languages in this tutorial (Python, Java, and C) use
zero-based indexing for arrays, meaning that the first element in an array can be accessed at index 0.
In Python, this code use index 0 to write the first array element (value 7) to the console:
Example
Python:
print( my_array[0] )
Run Example »
Let's create our first algorithm using the array data structure.
How it works:
2. Check if the current value is the lowest so far, and if it is, store it.
22
3. After looking at all the values, the stored value will be the lowest of all values in the array.
Try the simulation below to see how the algorithm for finding the lowest value works (the animation
is the same as the one on the top of this page):
Speed:
Find Lowest
Lowest value:
This next simulation also finds the lowest value in an array, just like the simulation above, but here we
can see how the numbers inside the array are checked to find the lowest value:
7,
12,
9,
11,
Implementation
Before implementing the algorithm using an actual programming language, it is usually smart to first
write the algorithm as a step-by-step procedure.
If you can write down the algorithm in something between human language and programming
language, the algorithm will be easier to implement later because we avoid drowning in all the details
of the programming language syntax.
1. Create a variable 'minVal' and set it equal to the first value of the array.
3. If the current element has a lower value than 'minVal', update 'minVal' to this value.
4. After looking at all the elements in the array, the 'minVal' variable now contains the lowest
value.
You can also write the algorithm in a way that looks more like a programming language if you want to,
like this:
Note: The two step-by-step descriptions of the algorithm we have written above can be called
'pseudocode'. Pseudocode is a description of what a program does, using language that is something
between human language and a programming language.
After we have written down the algorithm, it is much easier to implement the algorithm in a specific
programming language:
Example
Python:
minVal = i
Run Example »
When exploring algorithms, we often look at how much time an algorithm takes to run relative to the
size of the data set.
24
In the example above, the time the algorithm needs to run is proportional, or linear, to the size of the
data set. This is because the algorithm must visit every array element one time to find the lowest
value. The loop must run 5 times since there are 5 values in the array. And if the array had 1000
values, the loop would have to run 1000 times.
Try the simulation below to see this relationship between the number of compare operations needed
to find the lowest value, and the size of the array.
See this page for a more thorough explanation of what time complexity is.
Each algorithm in this tutorial will be presented together with its time complexity.
Random
Descending
Ascending
10 Random
Operations: 0
Run Clear
DSA Exercises
Exercise:
print(my_array[])
Submit Answer »
25
❮ PreviousNext ❯
Bubble Sort
Bubble Sort is an algorithm that sorts an array from the lowest value to the highest value.
Speed:
Bubble Sort
Run the simulation to see how it looks like when the Bubble Sort algorithm sorts an array of values.
Each value in the array is represented by a column.
The word 'Bubble' comes from how this algorithm works, it makes the highest values 'bubble up'.
How it works:
2. For each value, compare the value with the next value.
3. If the value is higher than the next one, swap the values so that the highest value comes last.
4. Go through the array as many times as there are values in the array.
Continue reading to fully understand the Bubble Sort algorithm and how to implement it yourself.
Before we implement the Bubble Sort algorithm in a programming language, let's manually run
through a short array only one time, just to get the idea.
Step 2: We look at the two first values. Does the lowest value come first? Yes, so we don't need to
swap them.
Step 3: Take one step forward and look at values 12 and 9. Does the lowest value come first? No.
7,
12,
9,
11,
We must understand what happened in this first run through to fully understand the algorithm, so
that we can implement the algorithm in a programming language.
27
Can you see what happened to the highest value 12? It has bubbled up to the end of the array, where
it belongs. But the rest of the array remains unsorted.
So the Bubble Sort algorithm must run through the array again, and again, and again, each time the
next highest value bubbles up to its correct position. The sorting continues until the lowest value 3 is
left at the start of the array. This means that we need to run through the array 4 times, to sort the
array of 5 values.
And each time the algorithm runs through the array, the remaining unsorted part of the array
becomes shorter.
Bubble Sort
7,
12,
9,
11,
We will now use what we have learned to implement the Bubble Sort algorithm in a programming
language.
2. An inner loop that goes through the array and swaps values if the first value is higher than the
next value. This loop must loop through one less value each time it runs.
3. An outer loop that controls how many times the inner loop must run. For an array with n
values, this outer loop must run n-1 times.
Example
n = len(my_array)
28
for i in range(n-1):
for j in range(n-i-1):
Run Example »
Imagine that the array is almost sorted already, with the lowest numbers at the start, like this for
example:
In this case, the array will be sorted after the first run, but the Bubble Sort algorithm will continue to
run, without swapping elements, and that is not necessary.
If the algorithm goes through the array one time without swapping any values, the array must be
finished sorted, and we can stop the algorithm, like this:
Example
n = len(my_array)
for i in range(n-1):
swapped = False
for j in range(n-i-1):
swapped = True
if not swapped:
break
29
Run Example »
For a general explanation of what time complexity is, visit this page.
For a more thorough and detailed explanation of Bubble Sort time complexity, visit this page.
The Bubble Sort algorithm loops through every value in the array, comparing it to the value next to it.
So for an array of nn values, there must be nn such comparisons in one loop.
And after one loop, the array is looped through again and again nn times.
This means there are n⋅nn⋅n comparisons done in total, so the time complexity for Bubble Sort is:
O(n2)––––––––––––––O(n2)__
The graph describing the Bubble Sort time complexity looks like this:
As you can see, the run time increases really fast when the size of the array is increased.
Luckily there are sorting algorithms that are faster than this, like Quicksort, that we will look at later.
30
You can simulate Bubble Sort below, where the red and dashed line is the theoretical time
complexity O(n2)O(n2). You can choose a number of values nn, and run an actual Bubble Sort
implementation where the operations are counted and the count is marked as a blue cross in the plot
below. How does theory compare with practice?
Random
Worst Case
Best Case
10 Random
Operations: 0
Run Clear
DSA Exercises
Exercise:
[7,14,11,8,9]
How does the array look like after the FIRST run through?
[,,,,]
Submit Answer »