Q-4:

1. Understanding the Harmonic Series

The harmonic series, H(n), is defined as:

H(n) = Σ (from i = 1 to n) 1/i

A famous result in calculus (and algorithm analysis) shows that as n grows large, H(n)
behaves very similarly to the natural logarithm function. More precisely, it can be
approximated by:

H(n) ≈ ln(n) + γ

where γ (gamma) is the Euler–Mascheroni constant (approximately 0.5772). The constant

γ does not affect the asymptotic behavior.

2. Using the Integral Test to Find Bounds

A common method for approximating sums like H(n) is to compare them to an integral.
Consider the integral of 1/x from 1 to n:

∫₁ⁿ (1/x) dx

We compute: ∫₁ⁿ (1/x) dx = [ln(x)]₁ⁿ = ln(n) – ln(1) = ln(n)

Next, we compare this integral to the sum.

Lower Bound

For i ≥ 2, notice that the function 1/x is decreasing. Therefore, for each term in the series
(except the first):

∫ᵢⁱ⁺¹ (1/x) dx ≤ 1/i

Summing from i = 1 to n – 1, we get:

⇒ ln(n) ≤ H(n)
∫₁ⁿ (1/x) dx ≤ H(n)

Upper Bound

Similarly, for i ≥ 2 we also note:

1/i ≤ ∫ᵢ₋₁ⁱ (1/x) dx

For i = 2 to n, summing up gives:

⇒ H(n) ≤ 1 + ln(n)
H(n) – 1 ≤ ∫₁ⁿ (1/x) dx

Thus we have the inequality:

ln(n) ≤ H(n) ≤ 1 + ln(n)

This is crucial because the constant 1 doesn't affect asymptotic growth.

3. Deriving Big-O, Big-Omega, and Big-Theta

Big-O (Upper Bound)

Big-O notation provides an upper bound. From the inequality we derived:

H(n) ≤ 1 + ln(n)

For sufficiently large n, ln(n) dominates the constant 1. In fact, we can say there exists a
constant c (for example, c = 2) and a value n₀ such that for all n ≥ n₀:

H(n) ≤ 2 ln(n)

Thus, we express this as:

H(n) = O(ln(n))

Since logarithms in any base differ only by a constant factor, it’s common to express this
simply as O(log n).

Big-Omega (Lower Bound)

Big-Omega gives us the lower bound. We observed:

H(n) ≥ ln(n)

This directly tells us that for all n ≥ 1, H(n) is at least ln(n). Therefore, there exists a
constant, say c' = 1, such that:

H(n) = Ω(ln(n))

Again, by convention we express this as Ω(log n).

Big-Theta (Tight Bound)

Since we have both:

ln(n) ≤ H(n) ≤ 1 + ln(n)

the growth of H(n) is sandwiched between two functions that differ only by a constant.
Hence, H(n) has a tight bound:

H(n) = Θ(ln(n)) = Θ(log n)

4. Summary

 Big-O: H(n) = O(ln(n)) (or O(log n)).

There exists c and n₀ such that H(n) ≤ c·ln(n) for all n ≥ n₀.
 Big-Omega: H(n) = Ω(ln(n)) (or Ω(log n)).
There exists c' and n₀ such that H(n) ≥ c'·ln(n) for all n ≥ n₀.
 Big-Theta: H(n) = Θ(ln(n)) (or Θ(log n)).
This indicates that ln(n) tightly bounds H(n) from above and below up to constant
factors.

5. Final Answer

For the harmonic series H(n):

H(n) = 1 + 1/2 + 1/3 + … + 1/n

Big-O: O(log n)
Big-Omega: Ω(log n)
Big-Theta: Θ(log n)

This completes the detailed breakdown for Q4.

 Q-10
1. Constant Time – O(1)

Definition & Characteristics:

An algorithm is said to run in constant time if its running time does not depend on the input size. No matter how large
the input is, the time required remains the same.

 Key insight:
o The algorithm performs a fixed number of operations.
o Constant time operations are ideal for efficiency.

Example:

 Accessing an element in an array by index:

int x = array[5];

No matter how big the array is, this operation always takes the same amount of time.

Practical Impact:

 Such operations are extremely efficient. When used properly, constant time routines form the building blocks of
more complex algorithms.

2. Logarithmic Time – O(log n)

Definition & Characteristics:

Logarithmic time means that as the input size (n) increases, the number of steps increases very slowly—proportional to
the logarithm of n.

 Key insight:
o This typically happens when the problem size is reduced by a constant fraction in each step (for
example, halving).
o Logarithms “compress” large increases in n into relatively small increases in work.
Example:

 Binary Search:
When searching for an element in a sorted array, binary search repeatedly divides the interval in half.

int binarySearch(int[] array, int target) {

int lo = 0, hi = array.length - 1;
while (lo <= hi) {
int mid = lo + (hi - lo) / 2;
if (array[mid] == target) return mid;
else if (array[mid] < target) lo = mid + 1;
else hi = mid - 1;
}
return -1;
}

For an array of size n, the worst-case number of comparisons is roughly log₂(n).

Practical Impact:

 Algorithms with logarithmic growth are very scalable. Even huge datasets can be handled efficiently if the
algorithm reduces the problem size rapidly.

3. Linear Time – O(n)

Definition & Characteristics:

An algorithm that runs in linear time performs a number of operations proportional to the input size n.

 Key insight:
o Every element is processed a fixed number of times, so doubling n roughly doubles the execution time.

Example:

 Linear Search:
Scanning an unsorted list to find a target element.

int linearSearch(int[] array, int target) {

for (int i = 0; i < array.length; i++) {
if (array[i] == target) return i;
}
return -1;
}

Each element is compared once in the worst-case.

Practical Impact:

 Linear algorithms are straightforward and perform well for moderate input sizes. They are essential when each
element must be examined.
4. Linearithmic Time – O(n log n)

Definition & Characteristics:

Linearithmic (or “n log n”) time represents a scenario where an algorithm takes linear time per level of a logarithmic
number of levels (often arising from divide-and-conquer techniques).

 Key insight:
o The problem is divided (which gives the logarithmic factor) and then combined or processed linearly for
each division.

Example:

 Merge Sort:
Merge sort divides an array into halves recursively (log n levels) and then merges these halves in O(n) time.

void mergeSort(int[] array, int lo, int hi) {

if (lo < hi) {
int mid = (lo + hi) / 2;
mergeSort(array, lo, mid);
mergeSort(array, mid + 1, hi);
merge(array, lo, mid, hi);
}
}

Combining these two parts yields a total running time of O(n log n).

Practical Impact:

 This is considered optimal for comparison-based sorting algorithms. Algorithms with O(n log n) behavior strike a
balance between division and merging steps.

5. Quadratic Time – O(n²)

Definition & Characteristics:

Quadratic time algorithms perform about n² operations, meaning that if the input size doubles, the execution time
roughly quadruples.

 Key insight:
o This usually results from having two nested loops iterating over the input.
Example:

 Bubble Sort:
In bubble sort, each element is compared with almost every other element.

void bubbleSort(int[] array) {

int n = array.length;
for (int i = 0; i < n - 1; i++) {
for (int j = 0; j < n - i - 1; j++) {
if (array[j] > array[j + 1]) {
// Swap
int temp = array[j];
array[j] = array[j + 1];
array[j + 1] = temp;
}
}
}
}

Practical Impact:

 Quadratic time can quickly become inefficient as n grows, but is acceptable for very small datasets. Many naive
algorithms for problems like finding duplicates or checking for pair sums are quadratic.

6. Cubic Time – O(n³)

Definition & Characteristics:

Cubic time algorithms involve three levels of nested loops, yielding a running time proportional to n³.

 Key insight:
o Like quadratic algorithms, but with an extra level of iteration, making them even less scalable.

Example:

 Naïve Matrix Multiplication:

Multiplying two n×n matrices involves three nested loops.

void matrixMultiply(int[][] A, int[][] B, int[][] C, int n) {

for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
C[i][j] = 0;
for (int k = 0; k < n; k++) {
C[i][j] += A[i][k] * B[k][j];
}
}
}
}

Practical Impact:

 While cubic algorithms are more computationally intensive than quadratic ones, in areas like 3D simulations or
solving certain combinatorial problems, they may still be practical for small n.
7. Exponential Time – O(2ⁿ) (or similar)

Definition & Characteristics:

Exponential time algorithms have running times that double (or more) with each additional input element. These are
highly inefficient for even moderately sized n.

 Key insight:
o Often arise in brute-force approaches, recursive algorithms without memoization, or problems with
solutions requiring examination of every possible combination.

Example:

 Recursive Computation of Fibonacci Numbers (naively):

int fibonacci(int n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}

The above implementation has exponential time complexity, as it recomputes subproblems many times.

 Subset Sum / Power Set Generation:

Generating all subsets of a set of n elements naturally requires O(2ⁿ) time since there are 2ⁿ possible subsets.

Practical Impact:

 Exponential algorithms are mostly limited to problems with very small input sizes. They often appear in
theoretical contexts or as baseline brute-force solutions before optimizations (like dynamic programming) are
applied.

Summary & Overall Perspectives

 O(1) (Constant): Best-case scenario for independent operations regardless of input.

 O(log n) (Logarithmic): Highly efficient algorithms (e.g., binary search), useful for quickly reducing problem size.
 O(n) (Linear): Common for algorithms that iterate through every element once.
 O(n log n) (Linearithmic): Best possible bound for many sorting tasks.
 O(n²) (Quadratic): Common in simple nested loops; acceptable for small n.
 O(n³) (Cubic): Often results from three nested loops; practical for very small instances.
 O(2ⁿ) (Exponential): Computationally intensive; feasible only for very small inputs.
 Q-29
Proving the Recurrence Equation of the Tower of Hanoi Using Repeated
Substitution

Problem Statement:
Prove the recurrence equation of the Tower of Hanoi problem using the repeated
substitution method. The recurrence is:

T(n)=2T(n−1)+1withT(1)=1.T(n)=2T(n−1)+1withT(1)=1.

Derive the closed-form solution T(n)=2n−1T(n)=2n−1.

Step-by-Step Proof

1. Recurrence Relation

The Tower of Hanoi problem for nn disks requires:

1. Moving n−1n−1 disks from the source rod to the auxiliary rod
(T(n−1)T(n−1) moves).
2. Moving the largest disk from the source to the target rod (11 move).
3. Moving n−1n−1 disks from the auxiliary rod to the target rod
(T(n−1)T(n−1) moves).

This gives the recurrence:

T(n)=2T(n−1)+1with base caseT(1)=1.T(n)=2T(n−1)+1with base caseT(1)=1

2. Repeated Substitution (Iteration)

Expand the recurrence step by step to identify a pattern:

 First substitution (k=1k=1):

T(n)=2T(n−1)+1.T(n)=2T(n−1)+1.

 Second substitution (k=2k=2):

Substitute T(n−1)=2T(n−2)+1T(n−1)=2T(n−2)+1:

T(n)=2[2T(n−2)+1]+1=22T(n−2)+2+1.T(n)=2[2T(n−2)+1]+1=22T(n−2)+2+1.
  Third substitution (k=3k=3):
Substitute T(n−2)=2T(n−3)+1T(n−2)=2T(n−3)+1:

T(n)=22[2T(n−3)+1]+2+1=23T(n−3)+22+2+1.T(n)=22[2T(n−3)+1]+2+1=23T(n−3)+
22+2+1.

 After kk substitutions:

T(n)=2kT(n−k)+2k−1+2k−2+⋯+21+20⏟Geometric series.T(n)=2kT(n−k)+Geometric se
ries2k−1+2k−2+⋯+21+20.

3. Geometric Series Simplification

The sum of the geometric series is:

∑i=0k−12i=2k−1.i=0∑k−12i=2k−1.

Thus, after kk substitutions:

T(n)=2kT(n−k)+(2k−1).T(n)=2kT(n−k)+(2k−1).

4. Substitute Until Base Case

The base case T(1)=1T(1)=1 occurs when n−k=1n−k=1, i.e., k=n−1k=n−1.

Substitute k=n−1k=n−1:

T(n)=2n−1T(1)+(2n−1−1).T(n)=2n−1T(1)+(2n−1−1).

Since T(1)=1T(1)=1:

T(n)=2n−1⋅1+2n−1−1=2n−1.T(n)=2n−1⋅1+2n−1−1=2n−1.

5. Verification with Small nn

 n=1n=1:
T(1)=21−1=1T(1)=21−1=1. ✔️
 n=2n=2:
T(2)=22−1=3T(2)=22−1=3.
Using recurrence: T(2)=2T(1)+1=2⋅1+1=3T(2)=2T(1)+1=2⋅1+1=3. ✔️
 n=3n=3:
T(3)=23−1=7T(3)=23−1=7.
Using recurrence: T(3)=2T(2)+1=2⋅3+1=7T(3)=2T(2)+1=2⋅3+1=7. ✔️
Final Conclusion

Using repeated substitution, we derived the closed-form solution for the Tower of Hanoi
recurrence:

T(n)=2n−1.T(n)=2n−1.

This matches the known minimum number of moves required to solve the puzzle
for nn disks.

Key Takeaways:

 The method leverages the geometric series formed during substitution.

 The base case anchors the solution, ensuring correctness for all n≥1n≥1.
 This approach generalizes to similar linear recurrences with constant coefficients.