Statistics 5620

Statistical Theory
Magdalena Bugaj

Fall 2014

Course Information

INSTRUCTOR: Magdalena Bugaj

OFFICE: 3305 Everett Tower, 387-4542,
m.bugaj@wmich.edu
OFFICE HOURS: T, H 1:00-2:00 pm, Mrs. Niu, 3367 Rood
TEXT: Probability and Statistical Inference by Bartoszynski
and Niewiadomska-Bugaj, Chapters 1-10. 2nd Ed., Wiley
Interscience.
ATTENDANCE: Role will not be taken. However, past
experience indicates poor attendance has an adverse
effect on grades.

2 / 30

Course Objectives

To introduce students to foundations of the theory of inferential

statistical techniques; to develop a working knowledge of
numerous probability distributions, their moments as well as
various transformations which are necessary for developing
sampling distributions of estimators used also as test statistics
in statistical inference.

3 / 30

HOMEWORK: Problems will be assigned at the end of

each class and they will be due one week later.
Assignments can be discussed with other students, but the
work you submit to be graded must be your own and must
not be copied. Any type of dishonest practices will not be
tolerated in any part of the course.
EXAMS: Two tests (October 19, November 20), and a
comprehensive final exam (December 9 - 5pm to 7 pm) will
be given. All exams will be closed book and closed notes.
One 8.5 x11 inch page of notes (formulas, theorems,
properties but NOT solved problems) is allowed for each
exam. The final determination of exam material will be
announced in class prior to the exam.
MAKE-UP EXAMS: Make-up exams will not be given
unless there is a legitimate reason for the absence and I
am notified before the exam. In the case of an emergency
situation, I must be notified within 24 hours of the exam

Grading
The final grade (10% scale) will be based on:
two tests 50%
final exam 30%
special project 5%
homework 15%
INCOMPLETES: Incompletes will only be given according to
University and Departmental policy. An incomplete is not a
substitute for a failing grade; they are given only after
completing a major portion of the coursework with a passing
grade, and circumstances beyond your control prevent you from
completing the coursework.

5 / 30

ACADEMIC INTEGRITY: You are responsible for making

yourself aware of and understanding the policies and
procedures in the Undergraduate and Graduate Catalogs that
pertain to Academic Honesty. These policies include cheating,
fabrication, falsification and forgery, multiple submission,
plagiarism, complicity and computer misuse. [The policies can
be found at www.wmich.edu/catalog under Academic Policies,
Student Rights and Responsibilities.] If there is reason to
believe you have been involved in academic dishonesty, you will
be referred to the Office of Student Conduct. You will be given
the opportunity to review the charge(s). If you believe you are
not responsible, you will have the opportunity for a hearing. You
should consult with me if you are uncertain about an issue of
academic honesty prior to the submission of an assignment or
test.

6 / 30

In addition, please check

http://osc.wmich.edu,

www.wmich.edu/registrar

and
www.wmich.edu/disabilityservices
to access the Code of Honor and general academic policies on
such issues as diversity, religious observance, student
disabilities, etc.
DISABILITY: If you are a person with a disability and anticipate
needing any type of accommodation in order to participate in
this class, please advise me so that we can make appropriate
arrangements.

7 / 30

Probabilistic Foundations of the Theory of Statistics

Probability theory calculus of randomnessa tool for statistics

(can also be studied on its own as an area of mathematics).
Historically, probability theory has developed for a very simple
reasonpeoples interest in gambling.
Games of chance can be traced to the earliest civilizations.
People tried to develop knowledge and skills to improve their
chances to win.
Mathematical framework that was developed to quantitatively
analyze these games, can be used to analyze practically any
random system.

8 / 30

Main components of the framework:

random experiment
elementary outcomes
events
random variables

9 / 30

Basic definitions
Experiment: any process not totally determined (controlled
or not) that generates data.
Elementary outcomes: possible results of an experiment.
Sample space: a set (S) of all possible outcomes of an
experiment. Can be finite, countably infinite, or
uncountable.
Event: any subset of a sample space, null event contains
no sample points.
Occurence of an event: For event A in a sample space S: if
an experiment is performed and an outcome s is observed
then if s A, we say that event A has occurred.

10 / 30

Example An experiment consists of a single toss of a regular

die. Two events are defined:
event Athe number of dots on the top face is an odd
number (e.g., 1, 3, 5)
event Bthe number of dots is larger than 2 (e.g., 3, 4, 5, 6)
The die was tossed and five dots appeared on the top face.
Event A occurred, and event B occurred (two events occurred).
If only one dot would appear on the top we would conclude that
only event A, but not B occurred (only one event occurred).
If two dots would appear on the top face of the die one would
conclude that neither A nor B occurred (zero events occurred).

11 / 30

More Complicated Situations

Outcomes are not always observed:
Students learn to drive a car. After some instruction and
practice some of them become good drivers while others
are worse. The level of their abilities cannot be measured
(observed) exactly. Instead students take a driving test.
The test scores are observed, but they do not exactly
represent the feature of interest.
A person likely has a disease. Diagnosis is based on
results of medical tests. Tests results are observed but
they do not represent the disease fully, although they are
strongly related (e.g., false positive and false negative
results).

12 / 30

Problem 1.2.1 (ii) S = {TT , HTT , THT , HHTT , HTHT , THHT ,

HHTH, HTHH, THHH, HHHH, HHHT }
Problem 1.2.8 (i) S = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6),
. . . , (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)}
S 0 = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}
A1 total odd S, S 0
A2 both faces even S but not S 0
A3 faces different S but not S 0
A4 first result larger than the second S but not S 0
A5 product greater than 10 S but not S 0
In pairs (1, 6), (2, 5), (3, 4) the total is the same (7)
but products are different
A6 product greater than 30 S, S 0
Only one outcome is possible: (6, 6).
HW 1.2.3 - for six chips (not five) in the urn
13 / 30

Algebra of Events

More complex events can be obtained by some algebra.

Venn diagram:

14 / 30

1. A B Event A is contained in B (implies B) if any

sample point in A is also in B.
S = {1, 2, 3, 4, 5, 6}
A number is larger than 5, B number is even.
In general A 6 B but it is true in this sample space S.
2. A = B Events A and B are equal if A B and B A
(they consist of the same sample points)
3. null event contains no sample points (needed
for technical reasons)

15 / 30

4. Ac a complement of event A contains sample

points that are not in A. If A occurs then Ac does not
occur and vice versa exactly one of these events HAS to
occur. If A = {1, 3, 5} then Ac = {2, 4, 6}.
5. A B a union of events A and B consists of
sample points that belong to A and/or B.
6. A B intersection of events A and B consists of
sample points that belong to A and to B.
Events A and B are disjoint if A B = .
7. A B a difference of events A and B consists of
sample points that are in A but not in B
8. A B symmetric difference (A B) (B A)
= (A B) (A B).
16 / 30

Example

S = {a, b, c, d, e}, A = {b, d}, B = {a, d, e}, C = {a, b, d}.

observed
sample point
a
b
c
d
e

events that
occurred
B, C
A, C
none
A, B, C
B

events that
did not occurr
A
B
A, B, C
none
A, C

Problem 1.3.7 p. 16

17 / 30

Operations on Events
Operations on events (union, intersection, complementation)
obey certain laws:
Idempotence A A = A, A A = A.
Double complementation (Ac )c = A.
Absorption A B = B A B = A A B.
For A S : A = A, A = , A S = S, A S = A.
Comutativity A B = B A, A B = B A.
Associativity A (B C) = (A B) C, A (B C)
= (A B) C.

18 / 30

Distributivity A (B C) = (A B) (A C),
A (B C) = (A B) (A C).
Generalization:
A (B1 Bn ) = (A B1 ) (A Bn )
A (B1 Bn ) = (A B1 ) (A Bn )
De Morgans Laws:
(A B)c = Ac B c , (A B)c = Ac B c .
Can be extended to k events:
(A1 Ak )c = Ac1 Ack ,
(A1 Ak )c = Ac1 Ack .

19 / 30

Infinite Operations on Events

Operations on events can be extended to infinitely many
events:
S
A1 A2 = T
i=1 Ai infinite union
A1 A2 =
i=1 Ai infinite intersection
Definition The class A of subsets of a sample space S is
closed under a given operation if sets resulting from
performing this operation on elements of A also belong to
A.
Example Let S = {0, 1, 2, . . .} and let A be a class of all
finite subsets of S.
A is closed under finite union and finite intersection but is
not closed under complementation (If B is a finite set then
B c is infinite and does not belong to A).
20 / 30

Definition A field is a nonempty class A of subsets of S that is

closed under complementation, finite union and finite
intersection (one is enough in view of De Morgan laws).
If A is a field and additionally is closed under infinite union nad
infinite intersection, then it is called a field
Closure under finite operations is weaker than closure under
infinite (countable) operations, therefore
A is a field (6)A is a field.
Example
A1 = {S, } a field and a field.
A2 = {S, , A, Ac } a field and a field (the smallest field
generated by event A

21 / 30

Example (cont.) S = {1, 2, 3, . . .}. A3 consists of subsets A of

S such that either A or Ac is finite.
A3 is a field but NOT a field: Ai = {i} A3 ,
S
A=
i=1 A2i = {2, 4, 6, . . .} 6 A3 since neither A = {2, 4, 6, . . .}
c
nor A = {1, 3, 5, . . .} is finite.
A3 is not closed under infinite union.
Verifying if a class A is a field is not difficult, but veryfying if it is
a field culd be difficult.

22 / 30

Definition A field is a monotone class (or a monotone field) if it

is closed under monotone passage to the limit, e.g., if
A1 , A2 , . . . are such subsets of S that if S
A1 A2 A3
then lim An = i Ai A
(increasing sequence)
(infinite union)
or
T
if A1 A2 A3
then lim An = i Ai A
(decreasing sequence)
(infinite intersection)
Theorem A field that is a monotone class is a field (that also
means that a field is a monotone class)

23 / 30

Probability

Probability is a number associated with an event that

represents its chance to occur. Three most common ways of
assigning that number are: frequencial (objective), classical
(logical) and subjective probability.
Frequential Probability a relative frequency of a certain
outcome in many repetitions of the same experiment (e.g.,
prob. of heads)
Subjective Probability expresses the degree of ones
belief that certain event (A) will occur.

24 / 30

Classical Probability obtained by the symmetry. For

example, the probability that in two tosses of a regular die
the first result is less than the second is 15/36.

1
2
first toss
3
4
5
6

second toss
1 2 3 4
x x x
x x
x

5
x
x
x
x

6
x
x
x
x
x

25 / 30

Axioms of Probability

Probability is a function on a field A (structure) of events in a

sample space S such that
1. Nonnegativity: P(A) 0 for any A A
2. Norming P(S) = 1
P
3. Countable additivity P(A1 A2 ) =
i=1 P(Ai ) if
A1 , A2 , . . . are pairwise disjoint events (any two events are
disjoint)

26 / 30

Consequences of the Axioms

The probability of the impossible (null) event is 0: P() = 0.

Finite additivity: if A1 , . . . , An are pairwise disjoint, then
P(A1 A2 An ) = P(A1 ) + P(A2 ) + + P(An )
Monotonicity: If A B then P(A) P(B).
Complementation: P(Ac ) = 1 P(A).
Union of two events: P(A B) = P(A) + P(B) P(A B)
Union of any n events:
P
P
P(A1 An ) = P(Ai ) i1 <i2 P(Ai1 Ai2 ) +
P
n+1 P(A A ).
n
1
i1 <i2 <i3 P(Ai1 Ai2 Ai3 ) + + (1)
Example:
P(A B C) = P(A) + P(B) + P(C)
P(A B) P(A C) P(B C) + P(A B C).

27 / 30

Problem 2.4.1 p. 35
P(A) = x, P(B) = P(A)/2 = x/2, P(C) = P(A)/3 =
x/3, P(Ac ) = 2P(A) = 2x. Since P(A) + P(Ac ) = 1, we have
x + 2x = 1, so that x = 1/3.
Consequently P(B) = 1/6, P(C) = 1/9, and
P(B C) = P(B) + P(C) = 1/6 + 1/9 = 5/18.

28 / 30

Problem 2.4.7 p. 35
Since x + 2x + 0.2 + 0.2 + 0.1 + 0.1 + 0.1 = 1, then
3x + 0.7 = 1 and x = 0.1.
(i) 0.1
(ii) 3x = 0.3
(iii) 1
(iv) 0.2+0.1+0.2+0.1=0.6

29 / 30

Problem 2.4.8 p. 35

P(Mary attends class) = 0.8.

P(only one of them is present) = 0.08 + 0.48 = 0.56.

P(both are absent) = 1 P(either of them is present) =
1 (0.08 + 0.32 + 0.48) = 1 0.88 = 0.12.

Problem 2.4.9 p. 35
R coin is returned, C connection is received
P(R) = 0.6, P(C) = 0.2, P(R c C c ) = 0.3, and P(R C) = x.
0.6 x + x + 0.2 x + 0.3 = 1, so that x = 1.1 1 = 0.1.

HW 2.4.6- for six events and k = 0.3, p = 0.05, 2.4.13What is

the probability that in two tosses of this die the result will the the
same?
30 / 30