Bayesian Optimization

-
-

Hyperparameter optimization
Pr , Pa ,
- -

,
Pre - each has a
range of values

Grid Search

Enumerate combinations and choose the best

force
" "

brute
computationally expensive
-

L
can
parallelize
 "
performance
"
the
Optimize
Costly to evaluate
-

fly ,
-
>
Vw) Vi is setting fr pi

Cost black box

function is a

kind to the shape of this

"

of
"

exotrmte
regression
-

fukin
-

Normally ,
we make assumption dont type of fukin
linear → Nonlinear ?
variable
Single
ease

a → (iii. iii. so .
.
.

>
¥7
la ☐
tho
polynomial
function #
Linear
regression
of re here

More
expansive
version .

if this idea
Regression
dh.dz An
bet if basis function
-
.
,

4,4 Ln
Compute coeffuo
- -

f- =
E. ✗ ili
Fourier transform
Same idea
Infinite basis
Multwaiiabelyausian
f- Ger ,
-
-

, Nut
L Means
( Covariance matrix

Gaussoanproce.ee#
Infinite dimensional version
of a
multivariate Ganson
( ✗rinz .
-
-
- )
For
any finite subset - multivariate Gaussian
 The mean & covariance matrix is the same for

every subset
A funhm if Chainz - -

Mind

Gaussian Process Regression

Basis is a
family if fmehois
Find the best
fit
Bayesvnoptimriahmatwolestartm.mx
, ,
Xz, - - ✗n (each Xiii a vector )

Evaluate my model at each Xi & compute

fGi)

Fit a multivariate Gaussian to these observations

&
format for
" "

-
Choose a reasonable mean
a
covariance typically 0

higher
" "

✗ is are dose covariance is

nj ,
Given this § fitted to (Xr .
-
> Mn)

t
Estimate Icann) for any unit
to
92+1 NI re -
mining
µn+ , ,

ML model
for anti

Acquisition step
" "

-
Choose the best anti

's
Optimize based regimen
-

over Nuss on
-

prediction