Virtual University of Pakistan

Lecture No. 45
of the course on
Statistics and Probability
by
Miss Saleha Naghmi Habibullah
 IN THE LAST LECTURE,
YOU LEARNT

• Randomized Complete Block Design

• The Least Significant Difference
(LSD) Test
• Chi-Square Test of Goodness of Fit
 TOPICS FOR TODAY

•Chi-Square Test of Goodness of Fit

(in continuation of the last lecture)
• Chi-Square Test of Independence
• The Concept of Degrees of Freedom
• p-value
• Relationship Between Confidence
Interval and Tests of Hypothesis
• An Overview of the Science of Statistics
in Today’s World (including Latest
Definition of Statistics)
 The students will recall that,
towards the end of the last lecture,
we discussed the chi-square test of
goodness of fit.
We applied the test to the
example where we had fitted a
binomial distribution to real data,
and, since the computed value of
our test statistic turned out to be
insignificant, therefore we
concluded that the fit was good.
Let us consider another example:
 EXAMPLE

The platform manager of an airline’s

terminal ticket counter wants to determine
whether customer arrivals can be modelled by
using a Poisson distribution.
 The manager is especially interested in
late-night traffic.
Accordingly, data for the time period of
interest have been collected, as follows:
 Number of
Frequency
Arrivals Per Minute
0 84
1 114
2 70
3 60
4 32
5 16
6 15
7 4
8 5
400
Is the distribution Poisson?
 SOLUTION:

First of all, we fit a Poisson

distribution to the given data:
 Because a mean is not specified, it must
be estimated from the sample data.

The mean of the frequency distribution

can be found by using the formula

where n = f.x 
 fx
n
Thus we have the following
calculations:
Number of Frequency
Arrivals
x f fx
0 84 0
1 114 114
2 70 140
3 60 180
4 32 128
5 16 80
6 15 90
7 4 28
8 5 40
400 800
Hence:

Mean X 
 fX

800
2
n 400
 Replacing  by x , the formula for the
Poisson probabilities is

X 
x
2 
X 2 x
e e
f ( x)  
x! x!
Hence, we obtain:
Number
Poisson Expected
of Observed
Probabilities Frequencies
Customer Frequencies
f(x) 400 f(x)
Arrivals
0 84 0.1353 54.12
1 114 0.2707 108.28
2 70 0.2707 108.28
3 60 0.1804 72.16
4 32 0.0902 36.08
5 16 0.0361 14.44
6 15 0.0120 4.80
7 4 0.0034 1.36
8 5 0.0009 0.36
9 or more 0 0.0002 0.08
400 1 400
 Next, we apply the chi-square test of
goodness of fit according to the following
procedure:
 Hypothesis-Testing Procedure:

Step-1:
Null and Alternative Hypotheses:

H0 : Arrivals are Poisson-distributed.

H1 : The distribution is not Poisson.
Step-2:
Level of Significance:
 = 0.05
 Step-3: Test-Statistic:
2
 
oi  ei 
2

i ei
which, if H0 is true, follows the chi-square
distribution having
k - 1 - r degrees of freedom
(where k = No. of categories after having
carried out the necessary mergers, and r =
number of parameters that we estimate from
the sample data).
Step-4:
Computations:
The necessary calculations are shown
in the following table:
Number
Observed Expected
of 2 2
Frequency Frequency (0 – e) (0-e) (0-e) /e
Customer
oi ei
Arrivals
0 84 54.12 29.88 892.81 16.50
1 114 108.28 5.72 32.72 0.30
2 70 108.28 -38.28 1465.36 13.53
3 60 72.16 -12.16 147.87 2.05
4 32 36.08 -4.08 16.65 0.46
5 16 14.44 1.56 2.43 0.17
6 15 4.80
7 4 24
1.36 6.60 17.40 302.76 45.87
8 5 0.36
9 or more 0 0.08
400 400 2=78.88
 With reference to the above, it should
be noted that, since some of the expected
frequencies are less than the required
minimum of 5, it became necessary to
combine some of those classes.
 Combination is best accomplished
working from the bottom up.
In order that we obtain a number greater
than 5, the last four expected frequencies had
to be combined.
Hence, the effective number of
categories becomes 7.
 Step-5:
Determination of the
Critical Region:
Since the effective number of
categories becomes 7
therefore k = 7.
Also, since the one lone parameter of
the Poisson distribution has been
estimated from the sample data, hence r =
1.
 Hence:
Our statistic follows the chi-square
distribution having
k - 1 - r = 7 - 1 - 1 = 5
degrees of freedom.
The critical region is given by
2  20.05 (5) = 11.07
 CRITICAL REGION:

0.05

0 11.07 78.88
 Step-6:
Conclusion:
Since the computed value of our test
statistic i.e. 78.88 is much larger than the
critical value 11.07, therefore, we reject H0
and conclude that the distribution is probably
not a Poisson distribution with parameter 2.
(With only 5% risk of committing Type-1
error, we conclude that the fit is not good.)
 In fact, the computed value of our test
statistic i.e. 78.88 is so large that it is possible
that if we had set the level of significance at
1%, even then it would have exceeded the
critical value.
The students are encouraged to check
this up themselves.
If the computed value does fall in the
critical region corresponding to 1% level of
significance, then our result is highly
significant.
 RATIONALE OF THE CHI-SQUARE
TEST
OF GOODNESS OF FIT:

2 oi  ei 2
that 
It is clear 
i e i
will be a
small quantity when all the oi’s are close to
the corresponding ei’s.
(In fact, if the observed frequencies are
exactly equal to the expected ones, then 2
will be exactly equal to zero.)
 The 2 - statistic will become larger
when the differences between the oi’s and ei’s
become larger.
Thus, 2 measures the amount of
deviation (or discrepancy) between the
observed and the expected results.
 ASSUMPTIONS OF THE
CHI-SQUARE TEST OF GOODNESS OF
FIT:

While applying the chi-square test of

goodness of fit, certain requirements must be
satisfied, three of which are as follows:
1. The total number of observations (i.e.
the sample size) should be
at least 50.
2. The expected number ei in any of the
categories should not be less than 5.
(So, when the expected frequency ei in any
category is less than 5, we may combine
this category with one or more of the other
categories to get ei  5.)
3. The observations in the sample or the
frequencies of the categories should be
independent.
Next, we begin the discussion of the Chi-
Square Test of Independence:
 In this regard, it is interesting to note
that, (since the formula of chi-square in this
particular situation is very similar to the
formula that we have just discussed),
therefore, the chi-square test of independence
can also be regarded as a kind of chi-square
test of goodness of fit.
 We illustrate this concept with the help
of an example:
 EXAMPLE
A random sample of 250 men and 250
women were polled as to their desire
concerning the ownership of personal
computers.
The following data resulted:
 Men Women Total
Want PC 120 80 200
Don’t Want
130 170 300
PC
Total 250 250 500
 Test the hypothesis that desire to own
a personal computer is independent of sex
at the 0.05 level of significance.
 SOLUTION

i) H0 : The two variables of classification

(i.e. gender and desire for PC) are
independent, and
H1 : The two variables of classification
are not independent.
ii) The significance level is set at =
0.05.

iii) The test-statistic to be used is

  oij  eij 

2 2
eij
i j
This statistic, if H0 is true, has an
approximate chi-square distribution with
(r - 1) (c - 1) = (2 - 1) (2 - 1) = 1
degrees of freedom.
iv) Computations:
In order to determine the value of 2,
we carry out the following computations :
 The first step is to compute the
expected frequencies.
The expected frequency of any cell is
obtained by multiplying the marginal total to
the right of that cell by the marginal total
directly below that cell, and dividing this
product by the grand total.
 200 250 
In this example, e 11  100 ,
500
 
250 
200
e12  100 ,
500
300250 
e21  150 ,
500
and



300 250 

e22 150 .
500
Hence, we have:
 Expected Frequencies:

Men Women Total

Want PC 100 100 200
Don’t Want
150 150 300
PC
Total 250 250 500
Next, we construct the columns of

/
oij – eij, (oij - eij)2 and (oij - eij)2 eij , as shown
below:
 o  e  o  e  o  e  e
Observed Expected 2 2
Frequency Frequency
oij eij ij ij ij ij ij ij ij

120 100 20 400 4.00

130 150 - 20 400 2.67

80 100 - 20 400 4.00

170 150 20 400 2.67

2
 13.33
 Hence, the computed value of our test-
statistic comes out to be

2
 13.33
v) Critical Region:
2  20.05(1) = 3.84
vi) Conclusion:
Since 13.33 is bigger than 3.84, we
reject H0 and conclude that desire to own
a personal computer set and sex are
associated.
 Now that we have concluded that
gender and desire for PC are associated, the
natural question is, “Which gender is it
where the proportion of persons wanting a
PC is higher?”
We have:
 Men Women Total
Want PC 120 80 200
Don’t Want
130 170 300
PC
Total 250 250 500
 A close look at the given data indicates
clearly that the proportion of persons who
are desirous of owning a personal computer
is higher among men than among women.
 And, (since our test statistic has come
out to be significant), therefore we can say
that the proportion of men wanting a PC is
significantly higher than the proportion of
women wanting to own a PC.
Let us consider another example:
 EXAMPLE
A national survey was conducted in a
country to obtain information regarding the
smoking patterns of the adults males by
marital status.
A random sample of 1772 citizens, 18
years old and over, yielded the following
data :
 SMOKING PATTERN
Marital
Total Only Regular Total
Status
Abstinence at times Smoker
Single 67 213 74 354
Married 411 63 129 1173
Widowed 85 51 7 143
Divorced 27 60 15 102
Total 590 957 225 1772
 Use this data to decide whether there is
an association between marital status and
smoking patterns.
 The students are encouraged to work
on this problem on their own, and to decide
for themselves whether to accept or reject
the null hypothesis.
 (In this problem, the null and the
alternative hypotheses will be:
H0 : Marital status and smoking patterns
are statistically independent.
HA : Marital status and smoking patterns
are not statistically independent.)
 This brings us to the end of the series
of topics that were to be discussed in some
detail for this course on Statistics and
Probability.
For the remaining part of today’s
lecture, we will be discussing some
interesting and important concepts.
 First and foremost, let us consider the
concept of Degrees of Freedom:
As you will recall, when discussing the
t-distribution, the chi-square distribution, and
the F-distribution, it was conveyed to you
that the parameters that exists in the
equations of those distributions are known as
degrees of freedom.
 But the question is, ‘Why are these
parameters called degrees of freedom?’
Let us try to obtain an answer to this
question by considering the following :
 Consider the two-dimensional plane, and
consider a straight line segment in the plane.
If one edge of the line segment is fixed at
some point (x0, y0), the line segment can be
rotated in the plane such that the fixed edge
stays in its place. In other words, we can say
that the line segment is free to move in the
plane with one restriction. Hence, if we fix
one end-point of the line segment, then we are
left with one degree of freedom for its
movement.
 Next, consider the case when we fix both
end-points of the line segment in the plane. In
this case, both degrees of freedom are lost, and
therefore the line can no longer move in the
plane.
But, if we view the above situation with
reference to the three-dimensional space --- the
one that we live in --- we note that the whole
plane (containing the fixed line segment) can
move in three dimensions, and hence, we have
one degree of freedom for its movement.
 Let us try to understand this concept in
another way:
Suppose we have a sample of size n =
6, and suppose that the sum of the sample
values is 20. That is, we have the following
situation:
Our Sample :

Sr. No. Value

1
2
3
4
5
6
Total 20
 Now, the point is that, given this total
of 20, if we choose the first 5 values freely,
we are not free to choose the sixth value.
Hence, one degree of freedom is lost.
This point can also be explained in the
following alternative way:
 Given that the sum of the six values is 20,
if we have knowledge of the first five values,
but the sixth value is missing, then we can re-
generate the sixth value.
This can also be expressed as follows:
If there are six observations and you find
their sum; next, you throw away one of the six
observations; then, you can
re-generate that observation (because of the fact
that you have already computed the sum).
 Since, the number of values that can be
re-generated is one, hence, the degrees of
freedom are n minus one.
(The one which can be re-generated is not the
one that we can choose freely.)
 Going back to sampling distributions
such as the
t-distribution, the chi-square distribution and
the F-distribution, ‘degrees of freedom’ can
be defined as the number of observations in
the sample minus the number of population
parameters that are estimated from the sample
data (from those observations).
 For example, in lecture number 39, we
noted that the statistic x  0
t 
s
n

follows the t-distribution having n-1 degrees of

freedom.
Here n denotes the number of
observations in our sample, and since we are
estimating one population parameter i.e. 
from the sample data, hence the number of
degrees of freedom is n-1.
 Similarly, referring to lecture number 42,
the students will recall that it was stated that
the statistic s12
F 2
s2
follows the F-distribution having
(n1-1, n2-1) degrees of freedom.
Here n1 denotes the number of
observations in the first sample, and since we
are estimating one parameter of the first
population i.e. 12 from the sample data, hence
the number of degrees of freedom
for the numerator of our statistic is
 Similarly, n2 denotes the number of
observations in the second sample, and
since we are estimating one parameter of
the second population i.e. 22 from the
sample data, hence the number of degrees
of freedom for the denominator of our
statistic is n2 minus one.
 In addition, in today’s lecture, you
learnt that the statistic r c 0  e 2
2 ij ij
   ,
i 1 j1 eij
follows the chi-square distribution having (r-
1)(c-1) degrees of freedom.
Let us try to understand this point:
 Consider the 2  2 contingency table ---
similar to the one that we had in the example
regarding the desire for ownership of a
personal computer.
In this regard, suppose that we have two
variables of classification, A and B, and the
situation is as follows:
 A1 A2 Total
B1 200
B2 300
Total 250 250 500
 The point is that, given the marginal
totals and the grand total, if we choose the
frequencies of the first cell of the first row
freely, we are not free to choose the
frequency of the second cell of the first row.
Also, given the frequency of the above-
mentioned first cell, we are not even free to
choose the frequency of the second cell of
the first column.
 Not only this, it is interesting to note
that, given the above, we are not even free to
choose the frequency of the second cell of
the second row or the second column !
Hence, given the marginal and grand
totals, we have only degree of freedom (i.e.
1 = 1  1 = (2-1)
(2-1) degrees of freedom).
 A similar situation holds in the case of a
2 x 3 contingency table. The students are
encouraged to work on this point on their
own, and to realize for themselves that, in the
case of a 2 x 3 contingency table, there exist
(2 - 1) ( 3 - 1) = 2 degrees of freedom .
 Next, let us consider the concept of p-
value:
 You will recall that, with reference to
the concept of hypothesis-testing, we
compared the computed value of our test
statistic with a critical value.
For example, in case of a right-tailed
test, we rejected the null hypothesis if our
computed value exceeded the critical value,
and we accepted the null hypothesis if our
computed value turned out to be smaller than
the critical value.
 A hypothesis can also be tested by
means of what is known as the p-value:
 p-Value:

The probability of observing a sample

value as extreme as, or more extreme than,
the value observed, given that the null
hypothesis is true.
 We illustrate this concept with the help
of the example concerning the hourly wages
of computer analysts and registered nurses
that we discussed in an earlier lecture:
 The students will recall that the
example was as follows:
 EXAMPLE

A survey conducted by a market-

research organization five years ago showed
that the estimated hourly wage for temporary
computer analysts was essentially the same
as the hourly wage for registered nurses.
 This year, a random sample of 32
temporary computer analysts from across the
country is taken.
The analysts are contacted by telephone
and asked what rates they are currently able
to obtain in the market-place.
 A similar random sample of 34
registered nurses is taken.
The resulting wage figures are listed in
the following table:
 Computer Analysts Registered Nurses
$24.10 $25.00 $24.25 $20.75 $23.30 $22.75
23.75 22.70 21.75 23.80 24.00 23.00
24.25 21.30 22.00 22.00 21.75 21.25
22.00 22.55 18.00 21.85 21.50 20.00
23.50 23.25 23.50 24.16 20.40 21.75
22.80 22.10 22.70 21.10 23.25 20.50
24.00 24.25 21.50 23.75 19.50 22.60
23.85 23.50 23.80 22.50 21.75 21.70
24.20 22.75 25.60 25.00 20.80 20.75
22.90 23.80 24.10 22.70 20.25 22.50
23.20 23.25 22.45
23.55 21.90 19.10
 Conduct a hypothesis test at the 2%
level of significance to determine whether
the hourly wages of the computer analysts
are still the same as those of registered
nurses.
 In order to carry out this test, the Null
and Alternative Hypotheses were set up as
follows:
Null and Alternative Hypotheses:

H 0 : 1 – 2 = 0
H A : 1 – 2  0
(Two-tailed test)
 The computed value of our test statistic
came out to be 3.43, whereas, at the 5%
level of significance, the critical value was
2.33, hence, we rejected H0.
Z.01 = -2.33 Z=0 Z.01 = +2.33
Z
Calculated Z = 3.43

1   2 0 X1  X 2 1.15
X1  X 2
 Hence, we concluded that there was a
significant difference between the average
hourly wage of a temporary computer
analyst and the average hourly wage of a
temporary registered nurse.
 This conclusion could also have been
reached by using the
p-value method:
I. Looking up the probability of Z > 3.43 in
the area table of the standard normal
distribution yields an area of .5000 – .4996
= .0004.
II. To compute the p-value, we need to be
concerned with the region less than –3.43 as
well as the region greater than 3.43 (because
the rejection region is in both tails).
 p-value = 0.0004+0.0004 = 0.0008

0.0004 0.0004

 .05  .05
 .025  .025
2 2 2 2

-3.43
-1.96 -1.96 0 1.96 3.43 Scale of z

Rejection Region Rejection Region

The p-value is
0.0004 + 0.0004 = 0.0008 .
Since this value is very small, it means
that the result that we have obtained in this
example is highly improbable if, in fact, the
null hypothesis is true.
Hence, with such a small p-value, we
decide to reject the null hypothesis.
The above example shows that :
 The p-value is a property of the data,
and it indicates “how improbable” the
obtained result really is.
A simple rule is that
if our p-value is less than the level of
significance , then we should reject H0,
whereas
if our p-value is greater than the level of
significance , then we should accept H0.
 (In the above example,
 = 0.02 whereas the p-value is equal to
0.0008, hence we reject H0.)
 Next, I would like to convey to you the
Relationship Between Confidence Interval
and Tests of Hypothesis:
 Some of the students may already have
an idea that there exists some kind of a
relationship between the confidence interval
for a population parameter  and a test of
hypothesis about .
(After all:

When deriving the confidence interval

for , the area that was kept in the middle of
the sampling distribution of X was equal to
1- so that the area in each of the right and
left tails was equal to /2.

And, when testing the hypothesis

H0 :  = 0 versus HA :   0 at level of
significance , the area in each of the right
and left tails was again equal to /2.)
 Hence, consider the following
proposition:
 Let [L, U] be a 100(1 - )% confidence
interval for the parameter .

Then we will accept the null hypothesis

H0 :  = 0 against
H1 :   0 at a level of significance  if 0
falls inside the confidence interval, but if 0
falls outside the interval [L, U], we will reject
H0.
 In the language of hypothesis testing, the
(1 - ) 100% confidence interval is known as
the acceptance region and the region outside
the confidence interval is called the rejection
or critical region.
The critical values are the end points of
the confidence interval.
 The students are encouraged to work
on this point on their own.
 As we approach the end of this course,
we present an Overview of the Science of
Statistics in Today’s World:
 Statistics is a vast discipline!

In this course, we have discussed the

very basic and fundamental concepts of
statistics and probability.
 But, there are numerous other topics
that could have been discussed if we had the
time.
We could have talked about the Latin
Square Design, we could have considered
Inference Regarding Regression and
Correlation Coefficients, we could have
discussed Non-Parametric Statistics, and so
on, and so forth.
 The students are encouraged to study
some of these concepts on their own --- as and
when time permits --- in order to develop a
better understanding and appreciation of the
importance of the science of Statistics.
 In this course, numerous examples were
discussed and many numerical problems
were presented.
The solutions of these problems were
presented in detail, and the various steps
were worked out. In doing so, the purpose
was to develop in the students a better
understanding of the core concepts of the
various techniques that were applied.
 But, it is interesting and useful to note
that, a lot many of these numerical problems
can be solved within seconds by using the
wide variety of statistical packages that are
available.
 These include SPSS, SAS, Statistica,
Statgraph, Minitab, Stata, S-Plus, etc. etc.
(The students are welcome to try out
some of these packages on their own.)
 Towards the end of this course, we
present one of the latest definitions of
Statistics:
 LATEST STATISTICAL DEFINITION:
Statistics is a science of decision making for
governing the state affairs. It collects analyzes,
manages, monitors, interprets, evaluates and
validates information.
Statistics is Information Science and
Information Science is Statistics. It is an applicable
science as its tools are applied to all sciences
including humanities and social sciences.

Acknowledgement:

Munir Ahmad
(President, Islamic Society of Statistical Sciences)
 Also, at this point, the students are
reminded of what was conveyed to them in
the very first lecture of this course:
Statistical thinking will one day be as
necessary for efficient citizenship as the
ability to read and write.
 If this course has been able to inculcate
in the students the fundamentals of
statistical and probabilistic thinking, then
this course has served its purpose.
 Education is a wholesome process.
The essence of effective teaching is
effective and inspiring communication on
the part of the teacher, and concentrated
attention and effort on the part of the
student.
 May Allah be with you in your pursuit
of knowledge.
May He bless you with the wealth of
knowledge, and the capability to utilize this
knowledge for the betterment of humanity.