Statistical Techniques to

Compare Groups
June Rey S. Sulatra, Ph.D.
ASSUMPTIONS
There are some general assumptions that apply
to all of the parametric techniques (e.g. t-tests, analysis
of variance), and additional assumptions associated
with specific techniques in other statistical tests.
Level of Measurement
• Each of the parametric approaches assumes that the
dependent variable is measured at the interval or
ratio level; that is, using a continuous scale rather
than discrete categories.
• Wherever possible when designing your study, try to
make use of continuous, rather than categorical,
measures of your dependent variable. This gives you
a wider range of possible techniques to use when
analyzing your data.
Random Sampling
The parametric techniques assume that the
scores are obtained using a random sample from the
population. This is often not the case in real-life
research.
Independence of Observations
• The observations that make up your data must be
independent of one another; that is, each observation or
measurement must not be influenced by any other
observation or measurement. Violation of this assumption,
according to Stevens (1996, p. 238), is very serious.
• There are a number of research situations that may violate
this assumption of independence. Example: Studying the
performance of students working in pairs or small groups. The
behavior of each member of the group influences all other
group members, thereby violating the assumption of
independence.
Normal Distribution
For parametric techniques, it is assumed that the
populations from which the samples are taken are normally
distributed. In a lot of research (particularly in the social
sciences), scores on the dependent variable are not
normally distributed. Fortunately, most of the techniques
are reasonably ‘robust’ or tolerant of violations of this
assumption. With large enough sample sizes (e.g. 30+), the
violation of this assumption should not cause any major
problems. The distribution of scores for each of your groups
can be checked using histograms obtained as part of the
Graphs.
Homogeneity of Variance
• Parametric techniques in this section make the assumption that
samples are obtained from populations of equal variances. This
means that the variability of scores for each of the groups is
similar.
• To test this, SPSS performs Levene’s test for equality of
variances as part of the t-test and analysis of variance analyses.
Be careful in interpreting the results of this test; you are hoping
to find that the test is not significant (i.e. a significance level of
greater than .05).
• If you obtain a significance value of less than .05, this suggests
that variances for the two groups are not equal and you have
therefore violated the assumption of homogeneity of variance.
Don’t panic if you find this to be the case.
Homogeneity of Variance
• Analysis of variance is reasonably robust to violations
of this assumption, provided the size of your groups is
reasonably similar.
• For t-tests, you are provided with two sets of results,
for situations where the assumption is not violated
and for when it is violated. In this case, you just
consult whichever set of results is appropriate for your
data.
EFFECT SIZE
• One way that you can assess the importance of your finding is
to calculate the ‘effect size’ (also known as ‘strength of
association’). This is a set of statistics that indicates the relative
magnitude of the differences between means, or the amount of
the total variance in the dependent variable that is predictable
from knowledge of the levels of the independent variable.
• There are a number of different effect size statistics. The most
commonly used to compare groups are partial eta squared and
Cohen’s d. SPSS calculates partial eta squared for you as part of
the output from some techniques (e.g. analysis of variance).
• It does not provide effect size statistics for t-tests, but you can
use the information provided in the SPSS to calculate whichever
effect size statistic you need.
EFFECT SIZE
• Partial eta squared effect size statistics indicate the
proportion of variance of the dependent variable that
is explained by the independent variable.
• Values can range from 0 to 1. Cohen’s d, on the other
hand, presents difference between groups in terms of
standard deviation units.
• Be careful not to get the different effect size statistics
confused when interpreting the strength of the
association.
EFFECT SIZE
To interpret the strength of the different effect size
statistics, the following guidelines were proposed by
Cohen (1988, p. 22) when assessing research involving
the comparison of different groups.
Parametric and Non-parametric Techniques
There is a whole family of techniques that can be used to test
for significant differences between groups. These include
parametric and non-parametric techniques.
• Parametric techniques make a number of assumptions
about the population from which the sample has been
drawn (e.g. normally distributed scores) and the nature of
the data (interval level scaling).
• Non-parametric techniques do not have such stringent
assumptions, and are often the more suitable techniques
for smaller samples or when the data collected is
measured only at the ordinal (ranked) level.
Parametric and Non-parametric Techniques
T-tests
There are two types of t-tests that we commonly used.
• independent-samples t-test – used when you want to
compare the mean scores of two different groups of
people or conditions
• paired-samples t-test – used when you want to
compare the mean scores for the same group of
people on two different occasions, or when you have
matched pairs.
INDEPENDENT-SAMPLES T-TEST
An independent-samples t-test is used when you
want to compare the mean score, on some continuous
variable, for two different groups of participants.
To illustrate the use of this technique, the survey5ED.sav
data file will be used. This example explores sex
differences in self-esteem scores.
INDEPENDENT-SAMPLES T-TEST
INDEPENDENT-SAMPLES T-TEST
INDEPENDENT-SAMPLES T-TEST
INDEPENDENT-SAMPLES T-TEST
INDEPENDENT-SAMPLES T-TEST
INDEPENDENT-SAMPLES T-TEST
INDEPENDENT-SAMPLES T-TEST
Calculating the effect size for
independent-samples t-test
INDEPENDENT-SAMPLES T-TEST
MANN-WHITNEY U TEST
The Mann-Whitney U Test is used to test for differences
between two independent groups on a continuous measure.
• For example, do males and females differ in terms of their self-
esteem? This test is the non-parametric alternative to the t-test
for independent samples.
• Instead of comparing means of the two groups, as in the case of
the t-test, the Mann-Whitney U Test actually compares
medians. It converts the scores on the continuous variable to
ranks across the two groups.
• It then evaluates whether the ranks for the two groups differ
significantly. As the scores are converted to ranks, the actual
distribution of the scores does not matter.
MANN-WHITNEY U TEST
MANN-WHITNEY U TEST
MANN-WHITNEY U TEST
MANN-WHITNEY U TEST
MANN-WHITNEY U TEST
Exercises
Data file: staffsurvey5ED.sav
Follow the procedures of independent-samples t-tests
and Mann-Whitney U test to compare the mean staff
satisfaction scores (totsatis) for permanent and casual
staff (employstatus).
Is there a significant difference in mean satisfaction
scores?
PAIRED-SAMPLES T-TEST
A paired-samples t-test (also referred to as repeated measures)
is used when you have only one group of people (or companies, or
machines etc.) and you collect data from them on two different
occasions or under two different conditions.
• Pre-test/post-test experimental designs are an example of the type of
situation where this technique is appropriate. You assess each person
on some continuous measure at Time 1 and then again at Time 2,
after exposing them to some experimental manipulation or
intervention.
• This approach is also used when you have matched pairs of
participants (i.e. each person is matched with another on specific
criteria, such as age, sex). One of the pair is exposed to Intervention 1
and the other is exposed to Intervention 2. Scores on a continuous
measure are then compared for each pair.
PAIRED-SAMPLES T-TEST
For the example, I will explore the impact of an
intervention designed to increase students’ confidence in their
ability to survive a compulsory statistics course.
• Students were asked to complete a Fear of Statistics Test (FOST)
both before (Time 1) and after the intervention (Time 2).
• The two variables from the data file that I will be using are:
FOST1 (scores on the Fear of Statistics Test at Time 1) and FOST2
(scores on the Fear of Statistics Test at Time 2).
• If you wish to follow along with the following steps, you should
start SPSS and open the file labelled experim5ED.sav.
PAIRED-SAMPLES T-TEST
PAIRED-SAMPLES T-TEST
PAIRED-SAMPLES T-TEST
PAIRED-SAMPLES T-TEST
PAIRED-SAMPLES T-TEST
Calculating the effect size for
paired-samples t-test
PAIRED-SAMPLES T-TEST
WILCOXON SIGNED RANK TEST
The Wilcoxon Signed Rank Test (also referred to as the
Wilcoxon matched pairs signed ranks test) is designed for use
with repeated measures; that is, when your participants are
measured on two occasions, or under two different
conditions.
• It is the nonparametric alternative to the repeated
measures t-test, but instead of comparing means the
Wilcoxon converts scores to ranks and compares them at
Time 1 and at Time 2.
• The Wilcoxon can also be used in situations involving a
matched subject design, where participants are matched
on specific criteria.
WILCOXON SIGNED RANK TEST
WILCOXON SIGNED RANK TEST
WILCOXON SIGNED RANK TEST
WILCOXON SIGNED RANK TEST
WILCOXON SIGNED RANK TEST
ONE-WAY BETWEEN-GROUPS ANOVA
One-way between-groups ANOVA is used when
you have one independent (grouping) variable with three
or more levels (groups) and one dependent continuous
variable. The ‘one-way’ part of the title indicates there is
only one independent variable, and ‘between-groups’
means that you have different participants in each of the
groups.
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
Open the file survey5ED.sav
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
ONE-WAY BETWEEN-GROUPS ANOVA
KRUSKAL-WALLIS TEST
The Kruskal-Wallis Test (sometimes referred to as the
Kruskal-Wallis H Test) is the non-parametric alternative to a
one-way between-groups analysis of variance.
• It allows you to compare the scores on some continuous
variable for three or more groups.
• It is similar in nature to the Mann-Whitney U Test
presented earlier, but it allows you to compare more than
just two groups.
• Scores are converted to ranks and the mean rank for each
group is compared. This is a ‘between groups’ analysis, so
different people must be in each of the different groups.
KRUSKAL-WALLIS TEST
KRUSKAL-WALLIS TEST
KRUSKAL-WALLIS TEST
KRUSKAL-WALLIS TEST
KRUSKAL-WALLIS TEST
KRUSKAL-WALLIS TEST
Exercises
Data file: staffsurvey5ED.sav
Follow the procedures of independent-samples t-tests
and Mann-Whitney U test to compare the mean staff
satisfaction scores (totsatis) for permanent and casual
staff (employstatus).
Is there a significant difference in mean satisfaction
scores?
Exercises
Data file: sleep5ED.sav
Conduct a one-way ANOVA with post-hoc tests (if
appropriate) and Kruskal-Wallis test to compare the
mean sleepiness ratings (Sleepiness and Associated
Sensations Scale total score: totSAS) for the three age
groups defined by the variable agegp3 (<=37, 38–50,
51+)