Scribd
Upload
47 views

Estimation and Detection Lec 01

This document provides an introduction to statistical inference and hypothesis testing. It discusses three examples where estimation and detection are important: communication systems, radar communication, and automatic control systems. It then reviews probability concepts like expectation and defines key terms for hypothesis testing such as the null and alternative hypotheses, decision rules, rejection regions, and risks. Finally, it covers Bayesian hypothesis testing, showing that the optimal decision rule is a likelihood ratio test that minimizes unconditional risk.

Uploaded by

Monu Mahali
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
47 views

Estimation and Detection Lec 01

This document provides an introduction to statistical inference and hypothesis testing. It discusses three examples where estimation and detection are important: communication systems, radar communication, and automatic control systems. It then reviews probability concepts like expectation and defines key terms for hypothesis testing such as the null and alternative hypotheses, decision rules, rejection regions, and risks. Finally, it covers Bayesian hypothesis testing, showing that the optimal decision rule is a likelihood ratio test that minimizes unconditional risk.

Uploaded by

Monu Mahali
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Lecture 1: Introduction to Statistical Inference

Introduction

We are interested in processing of information-bearing signals to extract information. There are two types of problems of fundamental interest.
1. Detection: Finite number of possible situations
2. Estimation: Nearest to the possible situation
We look at three examples of interest.
Example 1.1 (Communication System). We need to estimate unknown analog
signal from the received signal that is distorted and corrupted.
Source

Transducer

Transmitter
Channel

Output

Transducer

Receiver

Figure 1: Block diagram for analog communication.


Example 1.2 (Radar communication). Pulse electromagnetic waves are sent
and received after possible reflection from a target if it exists. Target could be an
aircraft, ship, spacecraft, missile etc. If the target is detected, then one is interested
in estimating range, angle, and velocity of the target. One may be interested in
tracking the mobile target trajectory or even controlling it.
Example 1.3 (Automatic Control). In automatic control problem, given a
linear time invariant system H, we need to design a controller C for achieving
a desired response through output signal y(t). Typically, reference signal x(t) is
unknown in such systems, and only a noisy version of the state may be observable.

x(t)

e(t)

u(t)

y(t)

F
Figure 2: Block diagram for automatic control.
Other applications of estimation and detection theory are in seismology, radio
astronomy, sonar, speech, signal, and image processing, biomedical signal processing, optimal communications etc.

Probability Review

We denote observation space by equipped with a -algebra G, that is a measurable collection of sets. Further, for all elements of A G, we have a non-negative
set function P : [0, 1] that satisfies the following axioms of probability:
1. P () = 1,
2. P
for any disjoint countable collection of sets {An : n N}, we have P (n An ) =
n P (An ).
Example 2.1 (Finite Observations). When observation space has finitely
many elements, we can take G = P(). Further, specifying P ({}) for all
completely specifies the probability set function.
Example 2.2 (Euclidean Space). For the case when observation space = Rn ,
we take G = B n , Borel -algebra on Rn . For this case, it suffices to specify the set
function P (A) for sets A G of the form { : i xi , i [n]}.
Definition 2.3 (Expectation). For a real valued function g : R, we denote
its expectation by E[g(Y )] and define it as
Z
E[g(Y )] =
g(y)dP (y).
y

Hypothesis Testing

Definition 3.1. A hypothesis is a statement about a population parameter.


Definition 3.2. The two complementary hypotheses in a hypothesis testing problem are called the null hypothesis and the alternative hypothesis, and denoted
by H0 and H1 respectively.
2

We assume that observation is a random variable Y distributed with


probability set function Pi when true hypothesis is Hi for i {0, 1}.
Definition 3.3. A hypothesis test is a rule : {0, 1} that specifies for all
values of y , index of the accepted true hypothesis H(y) .
Definition 3.4. Region 1 = {y : (y) = 1} is called the rejection region,
and 0 = c1 is called the acceptance region.
Definition 3.5. When the true underlying hypothesis is Hj , the cost incurred on
accepting hypothesis Hi is denoted by Cij . Uniform cost is given by
Cij = 1{i6=j} , i, j {0, 1}.
Definition 3.6. For each hypothesis Hi , the conditional risk is denoted by Rj ()
and defined as the expected cost incurred by the decision rule , when it is the
underlying true hypothesis. That is,
"
#
"
#
X
X
Rj () = E
Cij 1{(y)=i} |Hj holds = Ej
Cij 1{(y)=i}
i

= C0j Pj (0 ) + C1j Pj (1 ).
Our objective is to design a decision rule that minimizes risk. Usually, costs
of correct identification of the true hypothesis is low, and cost of incorrect identification is higher. Hence, minimizing risk for any hypothesis would be to ensure
that probability Pj (i ) is low for i 6= j. One cant simultaneously decrease all
decision regions {i }, since they form a partition of observation space .

3.1

Bayesian Hypothesis Testing

In this approach, we assume a prior distribution on hypotheses to be true.


Specifically, let i denotes the prior probability of Hi being true.
Definition 3.7. We can defined unconditional risk as expected value of risk over
all possible hypotheses, denoted by r(). That is,
X
r() = ERj () =
Rj ()j .
j

Definition 3.8. For two distributions P1 (y) and P0 (y) we can define likelihood
dP1
(y). When two distributions admit density, it is ratio of their
ratio as L(y) = dP
0
densities at y. For discrete distributions, it is ratio of their probability mass
functions at y.
3

Theorem 3.9. For a Bayesian hypothesis testing problem optimal decision rule
that minimizes unconditional risk for a prior distribution and costs {Cij } is a
threshold based rule called likelihood ratio test . That is,
B (y) = 1{L(y) } ,
where likelihood ratio L(y) =

dP1
(y)
dP0

and threshold =

0 (C10 C00 )
.
1 (C01 C11 )

Proof. We can write unconditional risk as


Z
X
X
r() =
j C0j +
j (C1j C0j )dPj (y).
y1

Minimizing unconditional risk is equivalent to selecting rejection region 1 such


that the integrand is negative. That is,
X
1 = {y :
j (C1j C0j )dPj (y) 0}.
j

By the definition of likelihood ratio and the threshold as defined in the theorem
hypothesis, theorem follows.
Remark 1. For uniform cost, threshold =

0
1

and conditional risk

r() = 0 P0 (1 ) + 1 P1 (0 ),
is probability of error in detection.
Definition 3.10. We can define posterior probability of hypothesis Hj being
true conditioned on observation being y as
j (y) = Pr{Hj is true |Y = y} =

j dPj (y)
.
0 dP0 (y) + 1 dP1 (y)

Remark 2. Observe that rejection region can be written in terms of posterior


probabilities as
1 = {y : (C10 C00 )0 (y) (C01 C11 )1 (y) 0}.
This is again a likelihood ratio test in term of ratio of posterior probabilities
(y)
10 C00
L0 (y) = 01 (y)
and threshold 0 = C
. That is, Bayes decision rule is
C01 C11
B (y) = 1{L0 (y) 0 } .

Definition 3.11. The expected cost of choosing hypothesis Hi given observation


y is called posterior cost and denoted as Ri (y), where
X
X
Ri (y) = E[
Ckj 1{(y)=i} |Y = y] =
Cij j (y) = Ci0 0 (y) + Ci1 1 (y).
j

k,j

Remark 3. Alternatively, one can also write rejection region in terms of posterior
costs as
1 = {y : R0 (y) R1 (y)}.
Therefore, Bayes decision rule can be interpreted as the one that minimizes the
posterior cost of choosing a hypothesis when the observation is y. That is,
B (y) = 1{ R1 (y) 1} .
R0 (y)

Remark 4. For uniform cost, Bayes decision rule is likelihood ratio test for posterior probabilities when the threshold is unity, That is,
B (y) = 1{L0 (y)1} .
This is equivalent to maximizing a posterior probability of underlying hypothesis
based on the observation. This is also called a MAP decision rule for binary
hypothesis test.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy