Homework I

Due on Jan 23, 2024

Use single precision (24 bits) unless specified otherwise. Whenever you are asked to explain
some results, you should explain the observation in a quantitative manner. For example, if
a curve has a bend at say n = 1000, you need to explain why you would expect the bend to
happen roughly around this value of n.

1. (10 marks) Consider the function f : ℜ2 → ℜ defined by f (x, y) = x · y. Measure

the size of the input (x, y) by |x| + |y|. What is the condition number of this function?
When is the condition number very high ? Can you intuitively explain why it is high ?
Soln: By definition the condition number is
|(x+h)(y+k)−xy|
|xy| |x| + |y| |hy + xk + hk|
max |h|+|k|
= · .
h,k→0
|x|+|y|
|xy| |h| + |k|

Assume without loss of generality that |x| > |y|. Then |hy+xk+hk|
|h|+|k|
can be made as large
as |x| by setting h to 0 and moving k towards 0. It is also easy to check that this ratio
is at most |x|. Therefore, the condition number is equal to
|x| + |y| |x|
=1+ .
|y| |y|

The problem is ill-conditioned if |x| ≫ |y|. Intuitively, suppose x = L, where L is a

large number and y = 1. Then if we change y from 1 to 2, the vector (x, y) changes
from (1, L) to (2, L). This is a tiny relative change in the input vector (of the order
1/L). But the product changes from L to 2L, which is a 100 percent change.
2. In class, we defined the notion of a backward stable algorithm as follows: an algorithm
for computing a function f (x) computes f (xb). Then the algorithm is backward stable if
|b
x−x|
|x|
is O(εmach ). Sometimes, it is more convenient to work with a slightly more general
definition. We say that the algorithm for computing f (x) is stable if it computes a
quantity y which is close to f (xb) for some vector xb in the following sense:
|y − f (xb)| |xb − x|
= O(εmach ), where = O(εmach ).
|f (xb)| |x|
(i) (5 marks) Show that if an algorithm for computing f is stable, then the relative
error in the output is roughly O(kf (x)·εmach ), where kf (x) is the condition number
of f at x. Hence, we can use either the notion of stability or backward stability.
Soln: We need to estimate |y−f (x)|
|f (x)|
. Now,

|y − f (x)| |y − f (xb) + f (xb) − f (x)| |y − f (xb)| |f (xb) − f (x)|

= ≤ + .
|f (x)| |f (x)| |f (x)| |f (x)|

1
 By definition of condition number,
|f (xb) − f (x)| |xb − x|
= O(kf (x)) · = O(kf (x) · εmach ).
|f (x)| |x|
Therefore the above can be written as
|y − f (xb)| |f (xb)|
· + O(kf (x) · εmach ) ≤ O(εmach ) · (1 + O(kf (x) · εmach )) + O(kf (x) · εmach )
|f (xb)| |f (x)|
= O((kf (x) + 1) · εmach )).

(ii) (5 marks) Consider the function f (x) = 1 + x. Show that if we compute this
function using this expression, then the algorithm may not be backward stable,
but is always stable.
Soln: The number x gets rounded to x′ , where x′ = x(1 + ε1 ) for some ε1 , |ε1 | ≤
εmach . And the final output is

(1 + x′ )(1 + ε2 ) = 1 + ε2 + x(1 + ε1 )(1 + ε2 )

where |ε2 | ≤ εmach . If this were backward stable, then we will need to express this
as 1 + xb. The only choice for xb is ε2 + x(1 + ε1 )(1 + ε2 ). But now,
|xb − x| |ε2 + x(ε1 + ε2 + ε1 · ε2 )| |ε2 | |ε2 |
= ≤ +|ε1 +ε2 +ε1 ·ε2 | ≤ +2εmach +O(εmach 2 ).
|x| |x| |x| |x|
The above can be very large if |x| ≪ εmach . Therefore, the computation is not
backward stable. Note that this will be backward stable if |x| ≥ 0.5.
Now we want to show that the computation is stable. Since backward stability
implies stability, we only need to worry about the case when |x| ≤ 0.5. Note that
the output is y = 1 + ε2 + x(1 + ε1 )(1 + ε2 ). Thus,
|y − f (x)| |ε2 + x(ε1 + ε2 + ε1 ε2 ) |x|
= ≤ |ε2 | + |ε1 ||1 + ε2 |.
|f (x)| |1 + x| |1 + x|
|x|
When |x| ≤ 0.5, |1+x| ≤ 1 and hence, the above is at most O(εmach ). So the
computation is stable.
(iii) (5 marks) Suppose we compute f (x) = sin(x) for x ∈ [0, 2π] using an algorithm
that outputs a value in the range sin(x)(1 ± εmach ). Show that such an algorithm
may not be backward stable, but is always stable.
Soln: Suppose it is backward stable. When x is close to π/2, then the output
value sin(x)(1 + ε) needs to be expressed as sin(x′ ). Now suppose x′ = x + δ.
Then sin(x′ ) ≈ sin(x) + δ cos(x). This implies that δ cos(x) = ε sin(x). Now,
|x′ − x| δ |ε sin(x)|
= = .
|x| |x| |x cos(x)|
Now, sin(x) ∼ 1, cos(x) ∼ 0. Therefore, the above can be much larger than εmach .
Therefore the computation is not backward stable.

2
 This can be easily shown to be stable. Indeed, consider xb = x. Then the computed
value y = sin(x)(1 + ε), and
|y − f (x)|
≤ εmach .
|f (x)|
ex −1
3. Consider the function f (x) = x
.

(i) (2 marks) Show that the function is well-conditioned for x close to 0.

Soln: We use the formula for condition number:
|xf ′ (x)|
kf (x) = .
|f (x)|
For x close to 0, f (x) ≈ 1, f ′ (x) ≈ 1/2. Therefore, the condition number is small.
(ii) (5 marks) Suppose we compute f (x) as in the expression above. Assume that the
computation of ex has relative error at most εmach . Prove that this computation
is not backward stable or stable (i.e., you need to show that the relative error can
be large and since the problem is well-conditioned, it must be the case that the
algorithm is unstable).
Soln: The computed value (again all the epsilons have absolute value at most
εmach ) can be written as
(ex (1 + ε1 ) − 1)(1 + ε2 )
.
x
Now suppose ε1 = εmach and ε2 = 0. Further x is very small (but positive) and
|x| ≪ εmach . Now the computed value above is approximately
(1 + x)(1 + εmach ) − 1 εmach
=1+ .
x x
The actual value should be close to 1. Thus, the relative error can be very large
if |x| ≪ εmach . Therefore the computation is not stable or backward stable.
e −1 x
(iii) (3 marks) Suppose we compute f (x) as ln(e x ) . Assume that the computation

of the exponential and the logarithm function have relative error at most εmach .
Prove that this algorithm is backward stable or stable.
Soln: Again (all epsilons have absolute value at most εmach ). The computed value
of ex is ex (1 + ε1 ). Therefore, the computed value of f (x) is
ex (1 + ε1 ) − 1
· (1 + ε3 ),
ln(ex (1 + ε1 ))(1 + ε2 )
where ε2 appears because of ln and ε3 appears because of division.
Now we set xb = x and let y be the expression above. Let z denote the intermediate
ex (1+ε1 )−1 |y−f (x)|
expression ln(e x (1+ε )) . Now, we want to show that
1 |f (x)|
= O(εmach ), i.e., y =
u−1
f (x)(1 + ε) for some ε, |ε| ≤ εmach . Define a function g(u) = ln(u) . Then f (x) =
g(e ) and z = g(e (1 + ε1 )). Now we want to argue that g(e ) and g(ex (1 + ε1 ))
x x x

3
 are close. For this we find the condition number of g at u ≈ 1 (since ex ≈ 1 when
x ≈ 0). By definition of condition number,

kg (y) = |g ′ (y)y|/|g(y)|.

Evaluating this at y ≈ 1, we see that kg (y) ≈ 1. Therefore, if a = ex ,

|g(a) − g(a(1 + ε1 ))| = O(εmach ) · |g(a)| ≈ O(εmach ),

because g(a) ≈ 1 for a ≈ 1. Thus,

ex (1 + ε1 ) − 1 z(1 + ε3 )
y = · (1 + ε 3 ) = = z(1 + O(εmach ))
ln(ex (1 + ε1 ))(1 + ε2 ) 1 + ε2
= g(a(1 + ε1 ))(1 + O(εmach )) = g(a)(1 + O(εmach ))(1 + O(εmach ))
= f (x)(1 + O(εmach )),

which is what we wanted to prove.

4. Consider the function

1 1
f (x) = − ,
1−x 1+x
assuming x ̸= ±1.

(a) (2 marks) When is the above function ill-conditioned?

|f ′ (x)x|
Soln: We use the definition of kf (x) = |f (x)|
. Evaluating this, we see that

1 + x2
kf (x) = .
1 − x2
Therefore the problem is ill-conditioned when x ≈ ±1.
(b) (3 marks) Suppose we compute this function using the expression above. Show
that the algorithm is unstable for values close to 0.
Soln: Note that the function is well-condition for x ≈ 0. We now show that the
computation has high relative error, and hence, must be unstable. First observe
that f (x) ≈ 0 for x ≈ 0. The computed value can be written as (assume that x
can be represented exactly, even then this computation is unstable)
1 + ε1 1 + ε2 ε1 1 + ε2
y= − = f (x) + − .
1−x 1+x 1−x 1+x

For x ≈ 0,
ε1 1 + ε2
− ≈ ε1 (1 + x) − ε2 (1 + x).
1−x 1+x
The above can be close to εmach . Thus, y could be f (x) + εmach . Since f (x) can be
very close to 0, this is high relative error.

4
 2x
(c) (5 marks) Suppose we compute the function as f (x) = 1−x 2 . Show that this

computation is stable or backward stable for all x.

Soln: Again it is easier to show stability. The computed value can be written as

2x(1 + ε1 )
y= .
1 − x2 (1 + ε2 )
√
Define xb = x 1 + ε2 ≈ x(1 + ε2 /2). Clearly |bx|x| −x|
= O(εmach ). Now we need to
show that y and f (xb) are close (i.e., y = f (xb)(1 + O(εmach ))). Observe that

2xb(1 + ε3 )
y= = z(1 + ε3 ),
1 − xb2
and so this shows the desired result.

5. The fibonacci numbers fk are defined by f0 = 1, f1 = 1, and

fk+1 = fk + fk−1 (1)

for any integer k > 1. A small perturbation of them, the pib numbers, pk , are defined
by p0 = 1, p1 = 1 and

pk+1 = c · pk + pk−1 (2)

√
3
for any integer k > 1 where c = 1 + 100
.

(a) (5 marks) Plot the numbers fn and pn together in one log scale plot. On the
plot, mark 1/ϵmach for single and double precision arithmetic.
Plots are shown. The interesting thing to note is that both grow exponentially at
about the same rate.
(b) (4 marks) Rewrite (1) to express fk−1 in terms of fk and fk+1 . Use the computed
fn and fn−1 to recompute fk for k = n−2, n−3, . . . , 0. Make a plot of the difference
between the original f0 = 1 and the recomputed f0 as a function of n. What n

5
 values result in low accuracy for the recomputed f0 ? How do the results in single
and double precision differ?
In single precision, machine precision can store up to 16 digits. The f (n) values
reach 1016 around n = 35. Since computation involves integer addition, this
means that there would not be any error in computation till n = 35. After that
digits will start getting chopped and errors will grow exponentially. As the plot
shows, till n = 35, there is no cancellation error. Similarly, for double precision,
there are roughly 35 digits, and f (n) reaches 1035 around n = 75. Hence, till
f (75) or so, there are no errors.
(c) (6 marks) Repeat part (b) for the pib numbers. Comment on the striking dif-
ference in the way precision is lost in the two cases. Explain the results.
In the computation of p(n), we have rounding errors from the very beginning.
This is because the value c is not stored precisely. Since errors double at every
step, it roughly means that one would lose one digit of accuracy after every step.
Since single precision corresponds to about 15 digits, we see high error around
n = 15. Similarly for double precision, one has about 35 digits of accuracy, and
so one starts seeing high errors around n = 35.
6. Write a program to generate the first n terms in the sequence given by the difference
equation:
xk+1 = 111 − (1130 − 3000/xk−1 )/xk ,

6
with starting values x1 = 11/2, x2 = 61/11 (2 marks). Use n = 10 for single precision.

(i) (2 marks) The exact solution is monotonically converging to 6, but what do you
observe?
After about n = 8, the values start deviating from 5. Roughly the reason is as
follows: each computation of xn requires two divisions by xk values, which may
lead to a loss of about 2 digits in precision. Since single precision is about 15
digits, it implies accuracy till about n = 7 only.
(ii) In order to explain the results, consider the function F : ℜ2 → ℜ2 as
" #! " #
x 111 − (1130 − 3000/y)/x
F = .
y x
" # " #!
xk+1 xk
The recurrence above can be expressed as = F . A vector
xk xk−1
z is a fixed point of F if F (z) = z. What are the possible fixed points of this
function? (3 marks)
We see that if (z, u) is a fixed point then u = z and so, u2 = 111u − (1130u −
3000/u). Solving this, we see that the solutions are u = 5, 6, 100.
(iii) (5 marks) To understand
" # the behaviour of the above recurrence, give " 2D# plots
xk 5
of how the vector evolves as we start from points close to . Use
xk−1 5
high precision here so that rounding errors do not affect your conclusion.
As can be seen from the figure, starting from points close to (5, 5), the recurrence
leads to a trajectory which always goes away from (5, 5).