Lecture 4

Closed Functions

September 4, 2007
 Lecture 4

Outline

Log-Transformation

Jensens Inequality

Level Sets

Closed Functions

Convexity and Continuity

Convex Optimization 1
 Lecture 4

Log-Transformation of Variables

Useful for transforming a nonconvex function to a convex one

A posynomial is a function of y1 > 0, . . . , yn > 0 of the form

a
g(y1, . . . , yn) = by11 ynan

with scalars b > 0 and ai > 0 for all i.

A posynomial need not be convex
Log-transformation of variables
xi = ln yi for all i
We have a convex function

0
f (x) = bea1x1 eanxn = bea x

Convex Optimization 2
 Lecture 4

Log-Transformation of Functions

Replacing f with ln f [when f (x) > 0 over domf ]

Useful for:
Transforming non-separable functions to separable ones

Example: (Geometric Mean) f (x) = (ni=1xi)1/n for x with xi > 0 for

all i is non-separable. Using F (x) = ln f (x), we obtain a separable F ,

n
1X
F (x) = ln xi
n i=1

Separable structure of objective function is advantageous in distributed

optimization

Convex Optimization 3
 Lecture 4

General Convex Inequality

Basic convex inequality: For a convex f , we have for x, y domf and
(0, 1),

f (x + (1 )y) f (x) + (1 )f (y)

General convex inequality: For a convex f and any convex combination of

the points in domf , we have

m m
!
X X
f i xi if (xi)
i=1 i=1

Pm
(xi domf and i > 0 for all i, i=1 i = 1, m > 0 integer)
A convex combination m i=1 i xi can be viewed as the expectation of a
P

random vector z having outcomes z = xi with probability i

Convex Optimization 4
 Lecture 4

Jensens Inequality

General convex inequality can be interpreted as:

For a convex f and a (finite) discrete random variable z with outcomes
zi domf , we have
f (Ez) E [f (z)]

General Jensens inequality: The above relation holds for a convex f and,
a random variable z with outcomes in domf and a finite expectation Ez

Convex Optimization 5
 Lecture 4

Level Sets

Def. Given a scalar c R and a function f : Rn R,

a (lower) level set of f associated with c is given by

Lc(f ) = {x domf | f (x) c}

Examples: f (x) = kxk2 for x Rn, f (x1, x2) = ex1

Every level set of a convex function is convex

Converse is false: Consider f (x) = ex for x R

Def. A function g is concave when g is convex

Every (upper) level set of a concave function is convex

Convex Optimization 6
 Lecture 4

Convex Extended-Value Functions

The definition of convexity that we have used thus far is applicable to

functions mapping from a subset of Rn to Rn. It does not apply to
extended-value functions mapping from a subset of Rn to the extended
set R {, +}.
The general definition of convexity relies on the epigraph of a function
Let f be a function taking values in R {, +}. The epigraph of
f is the set given by

epif = {(x, w) Rn R | f (x) w for some x Rn}.

Def. The extended-value function f is convex when its epigraph epif is

convex set (in Rn R).
This definition coincides with the one we have used for a function f
with values in R.

Convex Optimization 7
 Lecture 4

Closed Functions

Def. A function f is closed if its epigraph epif is a closed set in Rn R,

i.e.,
for every sequence {(xk , wk )} epif converging to some (x, w),
we have (x, w) epif
Examples
Affine functions are closed [f (x) = a0x + b]
Quadratic functions are closed [f (x) = x0P x + a0x + b]
Continuous functions are closed
A class of closed functions is larger than the class of continuous functions

For example
f (x) = 0 for x = 0, f (x) = 1 for x > 0, and f (x) = + otherwise
This f is closed but not continuous

Convex Optimization 8
 Lecture 4

Closed Function Properties

Lower-Semicontinuity Def. A function f is lower-semicontinuous at

a given vector x0 if for every sequence {xk } converging to x0, we have
f (x0) lim inf k0 f (xk )
We say that f is lower-semicontinuous over a set X if f is lower-
semicontinuous at every x X

Th. For a function f : Rn R {, +} the following statements

are equivalent
(i) f is closed

(ii) Every level set of f is closed

(iii) f is lower-semicontinuous (l.s.c.) over Rn

Convex Optimization 9
 Lecture 4

Proof of Theorem
(i)(ii) Let c be any scalar and consider Lc(f ). If Lc(f ) = , then Lc(f )
is closed. Suppose now that Lc(f ) 6= . Pick {xk } Lc(f ) such that
xk x for some x Rn. We have f (xk ) c for all k, implying that
(xk , c) epif for all k. Since (xk , c) (x, c) and epif is closed, it follows
that (x, c) epif . Consequently f (x) c, showing that x Lc(f ).
(ii)(iii) Let x0 Rn be arbitrary and let {xk } be a sequence such
that xk x0. To arrive at a contradiction, assume that f is not l.s.c. at
x0, i.e.,
lim inf k f (xk ) < f (x0)
Then, there exist a scalar and a subsequence {xk }K {xk } such that
f (xk ) < f (x0) for all k K
yielding that {xk }K L (f ). Since xk x0 and the set L (f ) is closed,
it follows that x0 L (f ). Hence, f (x0) - a contradiction. Thus, we
must have
f (x0) lim inf k f (xk )

Convex Optimization 10
 Lecture 4

Proof of Theorem - continues

(iii)(i) To arrive at a contradiction assume that epif is not closed.

Then, there exists a sequence {(xk , wk )} epif such that
(xk , wk ) (x, w) and (x, w) 6 epif .
Since (xk , wk ) epif for all k, we have
f (xk ) wk for all k
Taking the limit inferior as k , and using wk w, we obtain
lim inf f (xk ) lim wk = w.
k k
Since (x, w) 6 epif , we have f (x) > w, implying that
lim inf f (xk ) w < f (x)
k
On the other hand, because xk x, and f is l.s.c. at x, we have
f (x) lim inf f (xk )
k
- a contradiction. Hence, epif must be closed.

Convex Optimization 11
 Lecture 4

Operations Preserving Closedness

Positive Scaling
For a closed function f : Rn R {, +} and > 0, the function
g(x) = f (x) is closed

Sum
For closed functions fi : Rn R {, +}, i = 1, . . . , m, the sum
g(x) = m i=1 fi (x) is closed
P

Composition with Affine Mapping

For an m n matrix A, a vector b Rm, and a closed function
f : Rm R {, +}, the function
g(x) = f (Ax + b) is closed

This generalizes to a Composition with Continuous Mapping

Convex Optimization 12
 Lecture 4

Pointwise Supremum

For a collection of closed functions fi : Rn R {, +} over an

arbitrary index set I , the function

g(x) = sup fi(x) is closed

iI

Example: Piecewise-linear function (polyhedral function)

f (x) = max{a01x + b1, . . . , a0mx + bm} is closed

where ai Rn and bi R for all i

Convex Optimization 13
 Lecture 4

Convexity and Continuity

Let f : Rn R with domf = Rn.

If f is convex, then f is continuous over Rn

In this case, a convex f is l.s.c

Theorem Let f : Rn R be convex and such that int(domf ) 6= .

Then, f is continuous over int(domf ).
In this case, a convex f need not be l.s.c over Rn

Example:
1

for x = 0
f (x) =

0 for x > 0
+ otherwise

Convex Optimization 14
 Lecture 4

Proof of the Theorem

Using the translation if necessary, we may assume without loss of

generality that the origin is in the interior of the domain of f .

It is sufficient to show that f is continuous at the origin

By scaling the unit box if necessary, we may assume without loss of

generality that the unit box {x Rn | kxk 1} is contained in domf

Let vi, i I = {1, . . . , 2n} be vertices of the unit box (i.e., each vi has
entries 1 or 1). The unit box can be viewed as a simplex generated
by these vertices, i.e.,

every x with kxk 1 is a convex combination of vertices vi, i I

Convex Optimization 15
 Lecture 4

or equivalently: every x with kxk 1 is given by

X
with i 0 and
P
x= i vi iI i = 1
iI

Note that by convexity of f , we have

f (x) max f (vi) = M (1)

iI

Let xk 0 and assume that xk 6= 0 for all k

We introduce yk = kxxkk and zk = kxxkk
k k
Note that we can write 0 as a convex combination of yk and zk , as
follows

1 kxk k
0= xk + zk for all k
kxk k + 1 kxk k + 1

Convex Optimization 16
 Lecture 4

By convexity of f it follows that

1 kxk k
f (0) f (xk ) + f (zk ) for all k
kxk k + 1 kxk k + 1

By letting k and using Eq. (1), we have

f (0) lim inf f (xk ) (2)

k0

Note that we can write xk = (1 kxk k) 0 + kxk k yk for all k

By using convexity, we obtain

f (xk ) (1 kxk k) f (0) + kxk k f (yk )

Convex Optimization 17
 Lecture 4

Taking the limsup as k and using Eq. (1), we see that

lim sup f (xk ) f (0)

k

From this relation and Eq. (2), we see that

lim f (xk ) = f (0),

k

showing that f is continuous at 0

Convex Optimization 18