VIETNAM GENERAL CONFEDERATION

OF LABOR
TON DUC THANG UNIVERSITY
FACULTY OF INFORMATION
TECHNOLOGY

HUYNH QUOC DAI-524C0016

LE DUC ANH-524C0006

FINAL REPORT

APPLIED CALCULUS FOR IT

 HO CHI MINH CITY, 2023
VIETNAM GENERAL CONFEDERATION OF
LABOR
TON DUC THANG UNIVERSITY
FACULTY OF INFORMATION
TECHNOLOGY

HUYNH QUOC DAI-524C0016

LE DUC ANH-524C0006

FINAL REPORT

APPLIED CALCULUS FOR IT

Advised by
Mrs.Truong Thi Kim Tien
HO CHI MINH CITY, 2025
 i

ACKNOWLEDGEMENT

We would like to express our sincere gratitude to Mrs.Truong Thi Kim Tien, our
instructor and mentor, for her valuable guidance and support throughout the final report of
our project. She has been very helpful and patient in providing us with constructive
feedback and suggestions to improve our work.We have learned a lot from her expertise
and experience. We are honored and privileged to have her as our teacher and supervisor.

Ho Chi Minh city, 25th May 2025.

Author
(Signature and full name)

Dai
Huynh Quoc Dai

Anh
Le Duc Anh
 i

DECLARATION OF AUTHORSHIP

We hereby declare that this is our own project and is guided by Mrs.Truong Thi
Kim Tien; The content research and results contained herein are central and have not been
published in any form before. The data in the tables for analysis, comments and evaluation
are collected by the main author from different sources, which are clearly stated in the
reference section.

In addition, the project also uses some comments, assessments as well as data
of other authors, other organizations with citations and annotated sources.

If something wrong happens, We’ll take full responsibility for the content of
my project. Ton Duc Thang University is not related to the infringing rights, the
copyrights that We give during the implementation process (if any).

Ho Chi Minh city, 25th May 2025.

Author
(Signature and full name)

Dai
Huynh Quoc Dai

Anh
Le Duc Anh
 1

TABLE OF CONTENT

CHAPTER 1:SOLUTION

1. Singular Values: Definition and Formulas……………..…1

2. Singular Value Decomposition (SVD): Definition and
Formulas……………………………………………………2
3. Steps to Find a Singular Value Decomposition (SVD)……4
4. Five Applications of SVD in Real Life and Data Science...6
5. Example 1: SVD of a 2×3 Matrix of Rank 2………………7
6. Example 2: SVD of a 2×3 Matrix of Rank 1………………8
7. Compact (Reduced) SVD: Definition and Formulas……..9
8. Compact SVDs of the Two Example Matrices…………..10
9. Truncated SVD: Definition and Formulas………………12.
10.Truncated SVD of the Example Matrix with Rank 2 (Keep
1 Singular Value)…………………………………………13
11.Recommendation System Using Truncated SVD –
Approach 1 (Non-Centering)……………………………..15
12.Recommendation System Using Truncated SVD –
Approach 2 (Centering)………………………………...…17
13.Recommendation System: Analysis and Conclusions……19
14.References ………………………………………………...20
 2

1:Singular Values: Definition and Formulas

-Let A be an m×n real matrix. Before introducing the concept of Singular Value
Decomposition (SVD), we first define the singular values of A, which are
fundamental components of the SVD.

-Definition:

The singular values of A are defined as the square roots of the eigenvalues of the
matrix ATA (if m ≤ n) or AAT (if m > n). Since ATA and AAT are symmetric and
positive semi-definite matrices, all of their eigenvalues are real and non-negative.

+Let λ1,λ2,…,λn be the eigenvalues of ATA , ordered such that:

λ1≧λ2≧…λn≧0

+Then the singular values of A are defined as:

σᵢ = √λᵢ, for i = 1, 2, ..., n

+We usually arrange the singular values in descending order:

σ₁ ≥ σ₂ ≥ ... ≥ σᵣ > 0, where r is the rank of A

-Key Property: Non-Negativity

if x is an eigenvector of ATA with eigenvalue λ, we have:

‖Ax‖² = xTATAx = λ‖x‖²

Since ‖Ax‖² ≥ 0 and ‖x‖² > 0, it follows that λ ≥ 0, hence σᵢ ≥ 0 for all i.

-Rank and Singular Values

+According to Proposition 1.3 in the notes:

“The number of non-zero singular values of A equals the rank of A.”

+This is because A and ATA share the same kernel, so their ranks are equal. Therefore,
the number of non-zero singular values (i.e., positive σᵢ) equals rank(A)

Geometric Interpretation

+The singular values describe how A stretches or compresses unit vectors in ℝⁿ.
Specifically:
 3

+The largest singular value σ₁ equals the maximum value of ‖Ax‖ when x is a
unit vector:

σ₁ = max { ‖Ax‖ : ‖x‖ = 1 }

+This maximum is attained when x is the eigenvector of ATA corresponding to the

largest eigenvalue λ₁:

‖Ax‖ = √λ₁ = σ₁

+Similarly, σ₂ is the maximum of ‖Ax‖ for unit vectors orthogonal to the one that
gives σ₁, and so on.

2:Singular Value Decomposition (SVD): Definition and Formulas

Let A be an m × n real matrix. A Singular Value Decomposition (SVD) of A is a

factorization of the form:

A = U Σ VT

- Where:

 U is an m × m orthogonal matrix, whose columns are called the left singular

vectors.
 Σ (Sigma) is an m × n diagonal matrix with non-negative real numbers on the
diagonal. These are the singular values σ₁ ≥ σ₂ ≥ ... ≥ σᵣ > 0.
 V is an n × n orthogonal matrix, whose columns are the right singular vectors.

- The superscript T denotes matrix transpose.

Structure of the Σ Matrix

- The matrix Σ has the following form (assuming rank r):

Σ = [ σ₁ 0 ... 0]

[0 σ₂ ... 0]

[… … … …]

[0 0 ... σᵣ ] ∈ ℝ^(m×n)

- All off-diagonal entries are zero, and singular values appear on the diagonal in
descending order.

Interpretation

 The columns of V are eigenvectors of ATA.

 4

 The columns of U are eigenvectors of AAT

 Each non-zero singular value σᵢ satisfies: A vᵢ = σᵢ uᵢ

+ Where vᵢ is a right singular vector and uᵢ is the corresponding left singular vector.

Special Case: Symmetric Matrices

If A is a symmetric square matrix (i.e., A = AT), it admits the eigenvalue

decomposition:

A = PDPT

Where:

 P is an orthogonal matrix of eigenvectors.

 D is a diagonal matrix of eigenvalues.

+ In this case, the singular values of A are the absolute values of the eigenvalues:

σᵢ = |λᵢ|

+ So the SVD is essentially a rearranged eigenvalue decomposition.

3:Steps to Find a Singular Value Decomposition (SVD)

To compute the Singular Value Decomposition of a real matrix A ∈ Rr*m, we follow

the standard procedure as outlined in the course material. Assume the rank of A is r,
and we want to find orthogonal matrices U, V, and a diagonal matrix Σ such that:

A = U Σ VT

Step-by-Step Procedure

Step 1. Compute AT A:

 The matrix AT A is symmetric and of size n×n

 All its eigenvalues are real and non-negative.

Step 2. Find the eigenvalues and eigenvectors of AT A:

 Let λ1 ≥ λ2,…, ≥ λ n ≥0 be the eigenvalues of AT A.

 Let v1,v2,..,vn be the corresponding orthonormal eigenvectors.

Step 3. Compute the singular values σ i:

 The singular values are defined as σ i =√λ i , for all i.

 Arrange them in decreasing order.
 5

Step 4. Form the matrix V:

 Let V = [v1,v2,….,vn]
 Then V is an n × n orthogonal matrix.

Step 5. Compute the left singular vectors ui :

�
 For each i such that σ i ≠ 0, compute: ui = Avi
σi
 These vectors ui are orthonormal and form the first r columns of U.

Step 6. Complete matrix U:

 If r < m, extend the set { u1 , . . . , u r }to an orthonormal basis of Rm to form a

full m × m orthogonal matrix U.

Step 7. Construct the matrix Σ :

 Build the diagonal matrix Σ ∈ Rm*n , where the diagonal entries are:

σ1. σ2,…, σr and all other entries are zero.

- Justification from the Course Notes

+ From Theorem 3.2 in the lecture notes:

 If A is an m × n matrix with rank r, and vi are orthonormal eigenvectors of AT A,

�
then the vectors ui = Avi are orthonormal, and we can write:
σi

A = U Σ VT

+This confirms that the factorization is valid and that the resulting matrices are
consistent with the definition of SVD.
 6

4:Five Applications of SVD in Real Life and Data Science

+The Singular Value Decomposition (SVD) is a powerful and versatile tool in linear
algebra with wide applications across mathematics, computer science, and
engineering. Below are five practical and theoretical applications of SVD, as
discussed in the course.

Rank Estimation

+SVD helps determine the numerical rank of a matrix by identifying how many
singular values are significantly greater than zero. In real-world data (such as
measurements), noise may cause all singular values to be non-zero. However, only a
few large singular values indicate the true dimensional structure, while the rest
correspond to noise.

 For example, if only two singular values are large and the rest are near zero, the
matrix is close to having rank 2. This is especially useful when analyzing noisy
data.

Low-Rank Approximation

Using only the top s < r singular values and corresponding vectors, we can construct a
low-rank approximation of the matrix:
 7

+This technique is commonly used to compress data while preserving the most
important features. It provides the best possible approximation (in 2-norm and
Frobenius norm) of rank s, known as the Eckart–Young–Mirsky theorem.

Image Compression

-Images can be represented as matrices of pixel intensities. By applying SVD and

keeping only the largest singular values, we can reconstruct a visually similar image
with much fewer data points. This significantly reduces storage requirements while
maintaining image quality.
+For instance, an image of rank 2000 can often be approximated well using only 100–
200 singular values.

+This application is demonstrated in detail in Lab 6 of the course (SVD_other2

2025.pdf).

Latent Semantic Analysis (LSA) in Information Retrieval

+SVD is used in Natural Language Processing (NLP) to identify hidden (latent)

relationships between words and documents. A term-document matrix is decomposed
using SVD to reduce its dimensionality, which improves search accuracy, document
clustering, and topic modeling.

 This forms the mathematical foundation of Latent Semantic Indexing (LSI) in

search engines.

Solving Ill-Conditioned Linear Systems

+In systems where the matrix is nearly singular (ill-conditioned), direct methods like
Gaussian elimination are unstable. SVD allows for a numerically stable solution by
inverting only the significant singular values and ignoring those close to zero.

+Given A = U Σ VT , the least-squares solution to Ax = b is:

+This is especially useful in computational applications like regression, data fitting,

and inverse problems.

5:Example 1: SVD of a 2×3 Matrix of Rank 2

Let us consider the matrix:

We will now compute the Singular Value Decomposition A1= U Σ VT

 8

Step 1:Compute AT1 A1

Step 2: Compute the eigenvalues of AT1 A1

Using Python or symbolic tools (or numerical approximation), the eigenvalues are
approximately:

λ1≈48.44, λ2≈7.86, λ3≈0.70

Step 3: Compute singular values

σ1= 48,44≈6.96, σ2= 7,86≈2.80, σ3= 0,70≈0.84
+These are sorted in descending order:
σ₁ ≈ 6.96, σ₂ ≈ 2.80, σ₃ ≈ 0.84
+Because all three singular values are nonzero (and A has only 2 rows), only the first
two will be used to form the compact SVD

Step 4: Compute right singular vectors (V)

These are eigenvectors of AT A. Let:
 V = [v1,v2,v3] (orthonormal eigenvectors of AT A)
Step 5: Compute left singular vectors (U)

Use the relation:

Normalize u1 and u2 to forms columns of U ∈ R2*2

6:Example 2: SVD of a 2×3 Matrix of Rank 1

Let us define a rank-1 matrix:

+Note that the second row is exactly half of the first row, so rank = 1.

Step 1:Compute AT2 A2

 9

Step 2: Compute eigenvalues

+One nonzero eigenvalue:

λ1=70, λ2=0, λ3=0

Step 3: Compute singular values

σ1= 70≈8.37 , σ2=σ3=0

Step 4: Compute V and U

+Only one singular vector pair contributes:

All other singular values are zero, and corresponding vectors can be chosen arbitrarily
(to complete orthonormal bases)

SVD Result for Rank 1 Matrix

We have:

7:Compact (Reduced) SVD: Definition and Formulas

+The compact or reduced Singular Value Decomposition (SVD) is a simplified

version of the full SVD that retains only the non-zero singular values and their
corresponding singular vectors. It is especially useful for matrices with low rank, as it
provides a more efficient and minimal representation.

+Definition

Let A∈Rm*n be a matrix of rank r. Its compact SVD is:

 10

A = U1 Σ1 V1T

+Where:

U1 ∈Rm*r ;: the first r columns of the orthogonal matrix U

Σ1∈Rm*r : a diagonal matrix containing the r non-zero singular values

V1∈Rm*r :the first r columns of the orthogonal matrix V

+This decomposition is a more compact form of the full SVD, which includes zero
singular values and additional orthonormal vectors that are not essential for
reconstructing the original matrix.

Matrix Shapes Summary

Matrix Shape Description

U1 m*r Left singular vectors (reduced)
Σ1 r*r Non-zero singular values on diagonal
V1 r*n Right singular vectors (reduced, transposed)

When to Use Compact SVD

 When the matrix has low rank compared to its dimensions

 When performing low-rank approximation
 When working with compressed representations

+In practical applications like image compression, noise filtering, recommendation

systems

8:Compact SVDs of the Two Example Matrices

+We now compute the compact (reduced) singular value decomposition for both
example matrices from Sections 5 and 6. The compact SVD retains only the non-zero
singular values and their corresponding singular vectors.

8.1: Compact SVD of Matrix A₁ (Rank 2)

 11

Step 1: Compute AT1 A1

Step 2: Compute eigenvalues and eigenvectors of AT A

+ Using a numerical tool (e.g., NumPy), we compute:

Eigenvalues:

λ1 ≈48.44, λ2 ≈7.86, λ 3≈0.70

Singular values:

Step 3: Keep only the top r = 2 singular values

We retain:

 σ1 ≈ 6.96
 σ2 ≈ 2.80

Step 4: Compute the corresponding eigenvectors to form

Let:

 v1 and v2 : eigenvectors of AT A for σ1, σ2.

+Assemble V1 ∈ R3*2 using v1 and v2 as a columns

Step 5: Compute

Normalize both to obtain orthonormal vectors u1,u2 then form matrix U1∈R2*2.

+Compact SVD Result for A₁

 12

A = U1 Σ1 V1T

Where:

8.2: Compact SVD of Matrix A₂ (Rank 1)

Let:

As shown in Section 6, the matrix has rank r = 1. Its compact SVD keeps only the
first singular value and associated vectors:

Hence, the compact SVD of A2 is:

A = U1 Σ1 V1T

+This reduced form captures the exact structure of the rank-1 matrix using minimal
components.

9:Truncated SVD: Definition and Formulas

+The truncated singular value decomposition (truncated SVD) is an approximation of

the full or compact SVD in which only the top s < r singular values and their
corresponding singular vectors are retained. This technique is widely used for
dimensionality reduction, data compression, and noise filtering.
 13

Definition

Let A ∈Rm*n be a matrix of rank r ,with its compact SVD given by :

A = U1 Σ1 V1T

Where:

 U1 ∈ Rm*r is a matrix whose columns are the left singular vectors corresponding
to the nonzero singular values.
 Σ1 ∈ Rr*ris a diagonal matrix with singular values σ1 ≥σ 2≥ … ≥ σr >0
 V1 ∈ Rn*s contains the corresponding right singular vectors.

The truncated SVD of rank s ( s < r) is:

As = Us Σs VsT

Where:

 Us ∈ Rm*s consists of the first s columns of U1,

 Σs ∈ Rm*s contains the top s singular values,
 Vs ∈ Rn*s contains the first s columns of V1

+Matrix Shapes Summary:

Component size Description

Us m×s Top s left singular vectors
Σs s×s Top s singular value on diagonal
Vs s×n Top s right singular vectors (transposed)

Best Low-Rank Approximation:

+According to the Eckart–Young–Mirsky Theorem, the truncated SVD of rank s

provides the best approximation to A of rank s, minimizing the following:

Frobenius norm error:

2-norm error:
 14

+This property makes truncated SVD highly useful in data science and machine
learning applications.

10:Truncated SVD of the Example Matrix with Rank 2 (Keep 1

Singular Value)

In this section, we perform a truncated SVD of the matrix A1 from Example 1 by

retaining only the largest singular value and its associated singular vectors. This
results in a rank-1 approximation of the matrix.

Given matrix A1 :

+From the full or compact SVD (see Section 5), we previously found:

 Singular values: σ 1≈6.96, σ2 ≈2.80

 Right singular vector v1 associated with σ1.
1
 Left singular vector u1= σA1 v1

Truncated SVD of Rank 1

To perform a rank-1 truncated SVD, we keep only the largest singular value σ1 and
the corresponding vectors u1,v1:

This gives us a rank-1 matrix that approximates A1.

Numerical Example (approximate values)

Assume (from prior computations):

 15

Then the approximation becomes:

Resulting Approximate Matrix

This matrix is the best rank-1 approximation of A1 in terms of both Frobenius and
spectral norms.

Conclusion:

+By keeping only the largest singular value, we obtain a low-rank version of A1 that
captures its most significant structure. Although some details are lost (since rank is
reduced from 2 to 1), the overall pattern remains recognizable.

+This approach is commonly used in applications such as:

 Image compression
 Dimensionality reduction
 Recommendation systems

11:Recommendation System Using Truncated SVD – Approach 1 (Non-

Centering)

Step 1: Original Rating Matrix

 16

+We are given the following user–movie rating matrix, where missing values are
indicated as red boxes (or "?"):

User Transformers The Avengers Titanic Casablanca

Liam 5 5 1 ?

Noah 4 5 1 1

Oliver 4 4 1 2

James 5 4 1 1

Olivia 1 2 5 5

Emma ? 1 5 5

Mia ? 1 5 5

Luna 5 4 1 2

+We denote this matrix as R ∈ R8*4

Step 2: Fill in missing values using mean imputation

+For each missing value, we take the average of the corresponding row mean and
column mean:

For example:

 Liam’s missing rating for Casablanca:

Row mean ≈ 3.67, Column mean ≈ 3.38 → Fill = (3.67 + 3.38)/2 ≈ 3.52

 Emma’s missing value for Transformers:

Row mean ≈ 3.67, Column mean ≈ 4.0 → Fill = (3.67 + 4.0)/2 ≈ 3.83

 Mia’s missing value for Transformers:

 17

Same as Emma → ≈ 3.83

We now obtain the filled matrix (no missing values).

Step 3: Apply Truncated SVD (rank 1)

1,Compute the SVD of the filled matrix:

R≈U1 Σ1 V1T, keeping only 1 singular value

2,Multiply:

This gives us an approximate version of the rating matrix, where missing values are
predicted via reconstruction.

Step 4: Extract Predictions

From the approximated matrix:

 Liam’s rating for Casablanca ≈ 3.42

 Emma’s rating for Transformers ≈ 3.85
 Mia’s rating for Transformers ≈ 3.85

+These values are used as predicted ratings for the missing entries.

12:Recommendation System Using Truncated SVD – Approach 2

(Centering)

In this section, we use Approach 2: Centering to estimate missing values in the

movie rating matrix. Unlike Approach 1, this method centers the data by row,
meaning we subtract the user’s mean rating before performing SVD. This technique
helps reduce personal rating bias and often leads to better predictions.

Step 1:Input Matrix with Missing Values

 18

We start with the same rating matrix from Section 11:

User Transformers The Avengers Titanic Casablanca

Liam 5 5 1 ?

Noah 4 5 1 1

Oliver 4 4 1 2

James 5 4 1 1

Olivia 1 2 5 5

Emma ? 1 5 5

Mia ? 1 5 5

Luna 5 4 1 2

Step 2: Fill Missing Values (Same as in Approach 1)

 As before, fill missing entries using the average of the row and column means.
 The filled matrix has no missing entries.

Step 3: Center the Data by Row

For each row (user), subtract their row mean from all entries in that row. Let:

Now, each user has zero mean.

Step 4: Apply Truncated SVD (Keep Only 1 Singular Value)

Compute:
 19

Only keep the largest singular value (rank-1 approximation).

Step 5: Reconstruct the Centered Matrix

Compute the approximation:

This is the best rank-1 approximation of the centered matrix.

Step 6: Recover the Final Predictions

To recover predicted ratings, add back the row means that were subtracted in Step 3:

Now the matrix returns to its original rating scale, and missing values are predicted.

Real Test Ratings (Examples) :

Based on computation on google collab, we obtain:

 Liam → Casablanca ≈ 2,44

 Emma → Transformers ≈ 2,53
 Mia → Transformers ≈ 2,53

13:Recommendation System: Analysis and Conclusions

+In this section, we compare and reflect on the two different approaches used for
predicting missing ratings in the recommendation system: Approach 1 (Non-
Centering) and Approach 2 (Centering). Both approaches utilized truncated SVD
with rank 1, but differed in how the data was preprocessed.
 20

Predicted Rating (Non-

User Movie Predicted Rating (Centering)
Centering)

Liam Casablanca ≈ 3.42 ≈ 2.44

Emma Transformers ≈ 3.85 ≈ 2.53

Mia Transformers ≈ 3.85 ≈ 2.53

 The non-centered approach assumes that all users rate movies similarly, without
adjusting for personal rating habits.
 The centered approach subtracts each user’s average rating before SVD, helping
to remove user bias (e.g., generous vs. strict raters).

+As a result, centered predictions are generally lower, especially for users who tend to
give high ratings (like Liam). This adjustment often leads to more balanced and
realistic estimates.

Interpretation

 Liam’s predicted rating for Casablanca dropped from 3.42 to 2.44 after
centering, suggesting that although Liam tends to rate movies highly, Casablanca
may not align with his true preferences.
 Emma and Mia have identical observed ratings, so both methods gave them
the same prediction. However, the centered values are lower, reflecting the
adjustment for their high average ratings.

14:References

 SVD-notes 2025.pdf – Notes on Singular Value Decomposition for Math 54,

internal course material, 2025.
 SVD_other2 2025.pdf – Lab 6: The SVD and Image Compression, Applied
Linear Algebra Course Material, Faculty of Information Technology, 2025.
 David C. Lay, Linear Algebra and Its Applications, 4th Edition, Pearson, 2011.
 Lloyd N. Trefethen and David Bau III, Numerical Linear Algebra, SIAM, 1997.
NumPy Documentation – https://numpy.org/doc/
 SciPy Documentation – https://docs.scipy.org/doc/
 Matplotlib Documentation – https://matplotlib.org/stable/contents.html
21
22