HOCHSCHULE RHEIN-WAAL

BE18
Chapter 9: Hypothesis Testing
Håkan Lane
Why are confidence intervals not enough

● Many situations require a decision

● Comes down to yes/no:
● Examples:
● Are average emissions higher than stated limit?
● Does a vaccine help more than 90 % of the population?
● Are left-handed people smarter than right-handed?

2
What is a hypothesis?

• Theory about something

• Dichotomous: can only be either true or untrue,
• Can be proven false or correct

Example: Do Dalmatians have

more or less than 35
spots on average?

3
Null and alternative

• Two opposing hypotheses

• H0: the conservative approach - null hypothesis
• HA: what you are trying to prove

• H0: Average number of spots <= 35

• HA: Average number of spots > 35

4
Example
• A medical trial is conducted to test whether or not a new medicine reduces
cholesterol by 25%. State the null and alternative hypotheses.

• H0: Average reduction <= 25 %

• HA: Average reduction > 25 %

5
Example 1

• We want to test whether the mean height of eighth

graders is not equal to 158 cm. State the null and
alternative hypotheses.
• H0: mean height=158
• HA: mean !=158

6
Three situations

• H0: <= , HA >

• H0: = , HA !=
• H0: >= , HA <

We want to test if it takes fewer than 45 minutes to teach a lesson plan. State
the null and alternative hypotheses.
Fill in the correct symbol ( =, ≠, ≥, <, ≤, >) for the null and alternative
hypotheses.
a. H0: μ >= 45
b. Ha: μ < 45

7
Four possibilities
• Remember: We make decision on population based on sample

• The conclusion can be right or wrong!!

Sample Population

25 Dalmatians
Mu=40, All Dalmatians > 35?
sigma=6

8
Four possibilities
• We support H0, and H0 is true: Correct
• We support HA, but H0 is true: Type I Error
• We support HA, and HA is true: Correct
• We support H0, and HA is true: Type II Error

Our guess Reality-> H0 HA

H0 OK Type II

HA Type I OK

9
Implications for dalmatians

• Average <=35, test says <=35: Correct

• Average < 35, test says > 35: Type I error
• Average >35, test says > 35: Correct
• Average > 35, test says <= 35: Type II error

10
Example
• Question: Is rock climbing equipment from X company unsafe?
• H0? Ha?
• H0: safe
• HA: unsafe

• Type 1? Test says unsafe, but is safe

• Type 2? Test says safe, but unsafe

11
Example

• A court uses a statistical test as the key piece of evaluating evidence

in a murder case. If proven guilty, the accused will get a life sentence.

• H0: innocent
• HA: guilty

• Type 1: Guilty but in fact innocent

• Type 2: Innocent but in fact guilty

12
Example

• Suppose the null hypothesis, H0, is: the blood

cultures contain no traces of pathogen X. State the
Type I and Type II errors.
• Which type of error has the greater consequence,
Type I or Type II?

• Type 1: has disease but in fact no disease

• Type 2: no disease but has disease

13
Distributions

• Calculation methods tie to presented distributions

• Only works for certain distributions
• Sample must be random

• Mean:
• Normal distribution

• Proportion:
• Independent trials
• Each has same probability p

14
Proof beyond reasonable doubt

• You have reason to believe that a coin flip is rigged in favour of heads,
but how to prove?

• Start flipping, how many times, after how many heads would you be
convinced?

• Write in chat

15
How to check

• If fair, consider chance of getting:

• 1: ½ , 2: 0.25, 3: 0.125, 4: 0.0625, 5: 0.03125

• These percentages are called p values

• Chance of getting the observed result if H0 true

16
Significance alpha

• Decided critical level

• P value under alpha => reject H0
• In other words, alpha=risk of type 1

• Standard = 0.05 = 5%
• If chance of results less than 5%, it was caused by effect and not by
chance
• Can never be 100 % certain
• How many heads would the statistician require?
• 5

17
Poll

• H0: mu >=60, HA: mu < 60

• Type 1: mu>1, in fact mu<=1

• Type 2: mu<=1, mu>1

18
Principle of proof

• Question: “If H0 was true, what is the chance that all this happened
just by chance”?

• Example:
• H0 : x <= 15 cm, HA : x > 15 cm
• Sample of 10 gives x* = 17 cm, known σ = 2 cm
• What is the probability to get this is real mean was 15 cm?

19
Consider the CLT

• For sample of 10 with x* = 17 and known σ=2

• Consider the mean of 10 as a random variable if mean = 15
• What is expected value? 15
• Standard Error? 2 / sqrt(10)
• X : N(15, 2/sqrt(10))
• So probability of results = P(X>17)
• Area to the right of 17 = 1 - Area to left of 17
• Area to left = cumulative norm

20
The p value

• p = probability of obtaining observed results if H0 is true

• For mean:
• calculate z score: (17-15)/(2/sqrt(10))
• Find cumulative norm for this value F(z)
• p-value = 1 - F(z)

21
Different types of tails I: Left tailed

• Where to look for the p?

• Depends on type of test:
• HA includes <: Left tailed

22
Tails 2: Double tailed

• HA includes ≠ : Double tailed

23
Tail type 3: Right tailed

• HA includes >: Right tailed

24
Example II

• H0: μ ≤ 1, Ha: μ > 1

• Assume the p-value is 0.1243. What type of test is this?
Right tail

• H0: p = 0.5, Ha: p ≠ 0.5

Assume the p-value is 0.2564. What type of test is this?
Double tailed

25
One sample for mean summary

• x* mean in sample with n people, x tested mean, s known sigma for

population
• R: Calculate z=abs((x*-x)/(s/sqrt(n)))
• P-value: 1-pnorm(z)

26
Example

• Suppose that 10 volunteers have done an intelligence test; here are

the results obtained. The mean obtained at the same test, from the
entire population is 75. You want to check if there is a statistically
significant difference (with a significance level of 95%) between the
means of the sample and the population, assuming that the sample
variance is known and equal to 18.
• 65, 78, 88, 55, 48, 95, 66, 57, 79, 81

• p=0.13 => Accept H0: mean for volunteers same as population

27
One sample test for proportions
• Remember that some questions are only yes/no.

• Calculate proportion of yes and test hypothesis in sample

• Example:
• Proportion of voters for party
• Percentage of people with cancer
• Proportion of crops of a certain type

28
One sample test for proportion in R

• We have a sample of 150 flowers and we want to test whether the

proportion of small flowers is the same than the proportion of big
flowers (measured by the variable size). 77 are big, 73 small
• R: prop.test(77,150,0.5)

29
Example

• A teacher believes that 85% of students in the class will want to go on

a field trip to the local zoo. She performs a hypothesis test to
determine if the percentage is the same or different from 85%. The
teacher samples 50 students and 39 reply that they would want to go
to the zoo. For the hypothesis test, use a 1% level of significance.
• First, determine what type of test this is, set up the hypothesis test,
find the p-value, sketch the graph, and state your conclusion.
• One sample test of proportions
• H0: prop = 85
• HA= prop!=85

30
Comparing two groups

• So far, we only compared a mean to a certain value

• Bigger than, not equals or less than

• One sample test

• What if we need to compare two groups?

Examples

• Are people who exercise healthier than others?

• Do people who take a course score higher on the exam?
• In general,compare mean in one group to mean in another

• Population I Population II

Mean Mean
μ1 μ2

32
T test two groups

• Must be independent, i.e. not same subjects in both groups!!

• Take random samples from each:

• Sample 1: average x*1 , standard deviation σ1 , sample size n1
• Sample 2: average x*2 , standard deviation σ2, sample size n2

Sample 1 Sample 2

Population 1 Population 2

33
Compare the samples

• Calculate difference: x1* - x2*

• Standard error:

• Degrees of freedom: n1 + n2 - 2

34
P value from t distribution

• Calculate se =

• Test quantity t = (x1* - x2*)/s

• Hypothesis depends on question:

• Are they equal? twosided
• Is μ1 bigger? One sided right tail
• Is μ2 bigger? One side left tail

35
In R

With summary characteristics:

install.packages(„BSDA“)
library(BSDA)

tsum.test

tsum.test( mean.x, s.x = NULL, n.x = NULL, mean.y = NULL, s.y = NULL, n.y = NULL, alternative = "two.sided", mu = 0,
var.equal = FALSE, conf.level = 0.95 )

36
Example

• Two samples are shown in Table 10.2. Both have normal distributions.
The means for the two populations are thought to be the same. Is
there a difference in the means? Test at the 5% level of significance.
• Sample Size Sample Mean Sample Standard Deviation
• Population A 25 5 1
• Population B 16 4.7 1.2

37
Example 2

• A study is done to determine if Company A retains its workers longer

than Company B. Company A samples 15 workers, and their average
time with the company is five years with a standard deviation of 1.2.
Company B samples 11 workers, and their average time with the
company is 4.5 years with a standard deviation of 0.8. The
populations are normally distributed.
• a. Are the population standard deviations known?
• b. Conduct an appropriate hypothesis test. At the 5% significance
level, what is your conclusion?

• HA: average for A > average for B

38
With actual data

• Store in vectors x and y

• t.test(x, y = NULL, alternative = c("two.sided", "less", "greater"), mu =

0, paired = FALSE, var.equal = FALSE, conf.level = 0.95, …)

• Paired = FALSE when different people in the groups

• Alternative:
• ”less” when HA is <
• ”two-sided” when HA !=
• ”greater” when HA is >
39
Example

40
Conclusion

• Determine significance level in advance

• If p value lower than alpha, reject H0

• Go with Ha, remember that conclusion refers to populations

Population 1 Population 2

41
T test for proportions
• Remember that some questions are only yes/no.

• Could we compare proportions between two groups?

• Examples:
• Smoking in different countries

42
Hypothesis test for proportions

• Divide in two groups (cohorts); 1 and 2

• Clearly separated by one and only one attribute, example:
• Men/women
• Nationality
• Activity
• Sample n1 in group 1 and n2 in group2

43
Procedure

• Calculate p*= (x1+x2)/(n1+n2)

• SE=

• Calculate p1=x1/n1, p2=x2/n2

• z-score : z*= (p1-p2)/SE

44
In R

• Extend prop.test and use vectors instead of just one number

45
Example 1

• Two types of valves are being tested to determine if there is a

difference in pressure tolerances. Fifteen out of a random sample of
100 of Valve A cracked under 4,500 psi. Six out of a random sample
of 100 of Valve B cracked under 4,500 psi. Test at a 5% level of
significance.

46
Repeated Measures Basics

• So far always different subjects in the groups

• What if we measure the same people in different situations?
• Example:
• Lung capacity before/after exercise
• Reaction time driving/driving and using cellphone
• Results before/after course

47
Repeated Measures test procedure

• Random sample from studied population n subjects

• Measure same thing for each person twice, x1 and x2
• Calculate difference for each person xd = x1 - x2

48
 In R

• Exam1
• Exam2

• t.test(Exam1,Exam2,paired=TRUE)

49
 Example:

TRY IT 10.11
A study was conducted to investigate how effective a new diet was in
lowering cholesterol.
Results for the randomly selected subjects are shown in the table.
The differences have a normal distribution.
Are the subjects’ cholesterol levels lower on average after the diet?
Test at the 5% level.

Subject A B C D E F G H I

Before 209 210 205 198 216 217 238 240 222
After 199 207 189 209 217 202 211 223 201

50
Poll

51