Scribd
Upload
59 views4 pages

If You Find Any Errors, Please Let Your Instructor Know ASAP

This document contains solutions to 4 problems from a math homework assignment: 1) It shows that for orthogonal matrices U and V, the Frobenius and 2-norms of the matrix UAV are equal to the norms of A, but the 1-norm is not necessarily equal. 2) It proves formulas for computing the 1-norm and infinity-norm of a matrix A. 3) It finds the equation of the line that best fits 3 given data points using the least squares method. 4) It discusses solving a regularized least squares problem that minimizes ||b - Ax||^2 + λ||x||^2, and how the SVD and standard least squares

Uploaded by

Kathy Lee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
59 views4 pages

If You Find Any Errors, Please Let Your Instructor Know ASAP

This document contains solutions to 4 problems from a math homework assignment: 1) It shows that for orthogonal matrices U and V, the Frobenius and 2-norms of the matrix UAV are equal to the norms of A, but the 1-norm is not necessarily equal. 2) It proves formulas for computing the 1-norm and infinity-norm of a matrix A. 3) It finds the equation of the line that best fits 3 given data points using the least squares method. 4) It discusses solving a regularized least squares problem that minimizes ||b - Ax||^2 + λ||x||^2, and how the SVD and standard least squares

Uploaded by

Kathy Lee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Fall 2019 Math 104 - HW 6 Solutions

(If you find any errors, please let your instructor know ASAP)

Problem 1. Let U ∈ Rm×m and V ∈ Rn×n be orthogonal matrices. Show that for any A ∈ Rm×n ,
||U AV ||F = ||A||F and ||U AV ||2 = ||A||2 . Is it true that that we also always have ||U AV ||1 =
||A||1 ? Why or why not?

Solution. Let B = U AV , then

||B||2F = Tr(B T B) = Tr(V T AT U T U AV )


= Tr(V T (AT AV )) = Tr((AT AV )V T )
= Tr(AT A) = ||A||2F .

Thus ||B||F = ||A||F . Since

||B||2 ≤ ||U ||2 ||A||2 ||V ||2 = ||A||2 ,

and
||A||2 ≤ ||U T ||2 ||B||2 ||V T ||2 = ||B||2 ,
we know ||B||2 = ||A||2 .

||U AV ||1 = ||A||1 is not always true. For example, when m = n = 2, let A = V = I,
"√ √ #
2 − 2
U= √2 √2 .
2 2
2 2
√
Let x = (1, 0)T , then ||U ||1 ≥ ||U x||1 = 2 > 1 = ||A||1 .
Problem 2. Let A ∈ Rm×n

(a) Show that


m
X
||A||1 = max cj , cj = |aij |
1≤j≤n
i=1

(b) Show that


n
X
||A||∞ = max ri , ri = |aij |
1≤i≤m
j=1

Solution. (a) Denote A = [A1 A2 · · · An ]. Then cj = ||Aj ||1 . For any x = [x1 x2 · · · xn ]T ,
n n n
X 
X X
||Ax||1 =
xj A j ≤
|xj |||Aj ||1 ≤ |xj | max ||Aj ||1 = ||x||1 max cj .
1≤j≤n 1≤j≤n
j=1 1 j=1 j=1

Thus ||A||1 ≤ max cj .


1≤j≤n

Moreover, denote ej as the j-th unit vector. Then for any j,

||A||1 ≥ ||Aej ||1 = ||Aj ||1 = cj ,

so ||A||1 ≥ max cj . Thus ||A||1 = max cj .


1≤j≤n 1≤j≤n

(b) If A = 0, the assertion is clearly true. Now consider A 6= 0. Assume rk = max ri . Let
1≤i≤m
 T
x̃ = sgn(ak1 ), sgn(ak2 ), · · · , sgn(akn ) . Since A 6= 0, we have ||x̃||∞ = 1, and

n n
X X

||Ax̃||∞ ≥ akj sgn(akj ) = |akj | = rk = max ri .
1≤i≤m
j=1 j=1

Thus ||A||∞ ≥ max ri .


1≤i≤m

Moreover, for any x = [x1 x2 · · · xn ]T such that ||x||∞ = 1,


n n n
X X X
||Ax||∞ = max aij xj ≤ max
|aij ||xj | ≤ max |aij | = max ri ,
1≤i≤m 1≤i≤m 1≤i≤m 1≤i≤m
j=1 j=1 j=1

so ||A||∞ ≤ max ri . Thus ||A||∞ = max ri .


1≤i≤m 1≤i≤m
Problem 3. Very simple linear regression. Consider the three points
" # " # " #
0 1 2
, ,
1 −1 −2

What is the equation of the line that best fits those three points, in the sense of least squares?

Solution. Assume that the equation of the line is y = ax + b. Let


   
0 1 1
A = 1 1 , v = −1
   
2 1 2
" # " # " #
a 5 3 −5
Then AT A = AT v. Since AT A = , AT v = , we have
b 3 3 −2
" # " #−1 " # " #
a 5 3 −5 −3
= = 52
b 3 3 −2 6

Thus the equation of the line is y = − 23 x + 65 .


Problem 4. The least-squares solution minimizes ||b − Ax||2 . In some applications, we would like
to favor ‘low- energy’ solutions and minimize instead

||b − Ax||2 + λ||x||2 ,

where λ is a positive parameter chosen by the scientist/engineer.

(a) Find the solution to this problem.

(b) Suppose you have a routine that returns the SVD of any matrix A; that is, it returns U , Σ and
V . Can you use it to compute the solution? If yes, explain how.

(c) Suppose you have a routine that returns the solution to any least-squares problem; that is, it
takes as input an arbitrary pair A, b and returns an x minimizing ||b − Ax||2 . Can you use it
to compute the solution? If yes, explain how.
" # " #
A b
Solution. (a) Let à = √ , b̃ = , then
λI 0

b − Ax 2
" #
2
||b̃ − Ãx|| =
√ = ||b − Ax||2 + λ||x||2 ,

− λx

2
so the original problem is equivalent to minimizing ||b̃ − Ãx|| . Thus the solution is given
by ÃT Ãx = ÃT b̃. Since ÃT Ã = (AT A + λI) is positive definite, it is invertible (or see (b),
where we get its inverse by SVD). Thus there is a unique solution x∗ = (AT A + λI)−1 ÃT b̃ =
(AT A + λI)−1 AT b.

(b) If we have an SVD A = U ΣV T , then

x∗ = (AT A + λI)−1 AT b = (V Σ2 V T + λI)−1 V ΣU T b


= (V (Σ2 + λI)V T )−1 V ΣU T b = V (Σ2 + λI)−1 ΣU T b.
" # " #
A b
(c) As is explained in part (a), you input à = √ , b̃ = , then the output is the solution.
λI 0

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy