Choosing the Right Statistical Test | Types & Examples

Statistical tests are used in hypothesis testing. They can be used to:

 Determine whether a predictor (independent) variable has a statistically significant

relationship with an outcome variable.
 Estimate the difference between two or more groups.

Statistical tests assume a null hypothesis of no relationship or no difference between groups.

Then they determine whether the observed data fall outside of the range of values predicted by
the null hypothesis.

When to perform a statistical test

You can perform statistical tests on data that have been collected in a statistically valid manner –
either through an experiment, or through observations made using probability sampling methods.

For a statistical test to be valid, your sample size needs to be large enough to approximate the
true distribution of the population being studied.

To determine which statistical test to use, you need to know:

 Whether your data meets certain assumptions.

 The types of variables that you’re dealing with.

Statistical assumptions
Statistical tests make some common assumptions about the data they are testing:
 1. Homogeneity of variance: the variance within each group being compared is similar among
all groups. If one group has much more variation than others, it will limit the test’s
effectiveness.
2. Normality of data: the data follows a normal distribution (a bell curve). This assumption
applies only to quantitative data.

If your data do not meet the assumptions of normality or homogeneity of variance, you may be
able to perform a nonparametric statistical test, which allows you to make comparisons without
any assumptions about the data distribution.

Types of variables
The types of variables you have usually determine what type of statistical test you can use.

Quantitative variables represent amounts of things (e.g. the number of trees in a forest). Types of
quantitative variables include:

 Continuous (ratio variables): represent measures and can usually be divided into units
smaller than one (e.g. 0.75 grams).
 Discrete (integer variables): represent counts and usually can’t be divided into units smaller
than one (e.g. 1 tree).

Categorical variables represent groupings of things (e.g. the different tree species in a forest).
Types of categorical variables include:

 Ordinal: represent data with an order (e.g. rankings).

 Nominal: represent group names (e.g. brands or species names).
 Binary: represent data with a yes/no or 1/0 outcome (e.g. win or lose).
Choose the test that fits the types of predictor and outcome variables you have collected (if you
are doing an experiment, these are the independent and dependent variables). Consult the tables
below to see which test best matches your variables.

Choosing a parametric test: regression, comparison, or correlation

Parametric tests usually have stricter requirements than nonparametric tests, and are able to
make stronger inferences from the data. They can only be conducted with data that adheres to
the common assumptions of statistical tests.

The most common types of parametric test include regression tests, comparison tests, and
correlation tests.

Regression tests
Regression is a statistical method used to examine the relationship between two or more
variables. In the simplest terms, it helps us understand how the typical value of the dependent
variable (what you are trying to predict) changes when any one of the independent variables (the
predictors) is varied, while the other variables are held constant, if not constant adjust those
confounding variables..

Predictor Outcome Example

variable variable
Simple  Continuous  Continuous  Let's say we want to predict the final
Linear  1 predictor  1 outcome exam score of a student based on
Regression their hours of study. In this case, the
hours of study is the independent
variable (predictor), and the exam
 score is the dependent variable
(outcome)

Multiple  Continuous  Continuous  Imagine we're looking to predict the

Regression  2 or more  1 outcome effectiveness of a new flu medication
predictors based on the dosage given and the
age of the patient. Here, dosage and
age are the independent variables,
and medication effectiveness
(measured by symptom reduction) is
the dependent variable.

Logistic  Continuous  Binary  What is the effect of drug dosage on

regression the survival of a test subject?

Comparison tests
Comparison tests look for differences among group means. They can be used to test the effect of
a categorical variable on the mean value of some other characteristic.

The Student's t-test\t-test is used to compare the means of two groups or to compare a
sample mean to a known value when the sample size is small and the population standard
deviation is unknown.

Example: Imagine you want to compare the effectiveness of two different brands of headache
medication. You randomly assign 10 patients to Brand A and another 10 to Brand B and measure
the time it takes for the headache to resolve. You would use a t-test to determine if there's a
significant difference in the mean resolution times between the two brands.

ANOVA and MANOVA tests are used when comparing the means of more than two groups (e.g.,
the average heights of children, teenagers, and adults).

Predictor variable Outcome variable example

Paired t-test  Categorical  Quantitative What is the effect of two different
 1 predictor  groups come test prep programs on
from the same the average exam scores for
population students from the same class?

Independent  Categorical  Quantitative What is the difference in average

t-test  1 predictor  groups come exam scores for students
from different from two different schools?
populations

ANOVA  Categorical  Quantitative What is the difference in average

 1 or more  1 outcome pain levels among post-surgical
predictor patients given three different
painkillers?
Correlation tests

Correlation tests check whether variables are related without hypothesizing a cause-and-effect
relationship.

The correlation coefficient is measured on a scale that varies from + 1 through 0 to – 1. Complete
correlation between two variables is expressed by either + 1 or -1. When one variable increases as
the other increases the correlation is positive; when one decreases as the other increases it is
negative.

Variables Research question example

Pearson’s r  2 continuous variables How are latitude and temperature related?

Choosing a nonparametric test

Non-parametric tests don’t make as many assumptions about the data, and are useful when one
or more of the common statistical assumptions are violated. However, the inferences they make
aren’t as strong as with parametric tests.

The Chi-square test is used to determine whether there is a significant association between two
categorical variables. It compares the observed frequencies in each category to the frequencies
you would expect if there were no association between the categories.

Chi-square is the standard method and is best when you have a large number of subjects in
categories. It provides an approximate P value and can be calculated by hand as well. This is also
known as the chi-square test of independence. Fisher's exact test is used to calculate P values
for small sample sizes.

The Kruskal-Wallis H test (sometimes also called the "one-way ANOVA on ranks") is a rank-based
nonparametric test that can be used to determine if there are statistically significant differences
between two or more groups of an independent variable on a continuous or ordinal dependent
variable. The null hypothesis of the Kruskal-Wallis test is that the mean ranks of the groups are
the same.

Predictor Outcome Use in place Example

variable variable of…
Spearman’s r  Quantitative  Quantitative Pearson’s r How medication practices
(Corelation) and medication adherence
related to?
Chi square  Categorical  Categorical Pearson’s r Patients are divided into
test of two groups based on their
independence smoking status (smoker vs.
non-smoker). They are then
observed for the
effectiveness of a lung
medication, categorized as
effective or not effective.

Kruskal–  Categorical  Quantitative ANOVA For example, you could use

Wallis H  3 or more a Kruskal-Wallis H test to
groups understand whether exam
performance, measured on
 a continuous scale from 0-
100, differed based on test
anxiety levels (i.e., your
dependent variable would
be "exam performance"
and your independent
variable would be "test
anxiety level", which has
three independent groups:
students with "low",
"medium" and "high" test
anxiety levels).
ANOSIM  Categorical  Quantitative MANOVA
 3 or more  2 or more
groups outcome
variables

Wilcoxon  Categorical  Quantitative Independent What is the difference

Rank-Sum  2 groups  groups t-test in average exam scores for
test come from students from two different
different schools?
populations

Wilcoxon  Categorical  Quantitative Paired t-test you could use a Wilcoxon

Signed-rank  2 groups  groups signed-rank test to
test come from understand whether there
was a difference in
 the same smokers' daily cigarette
population consumption before and
after a 6 week
hypnotherapy programme
(i.e., your dependent
variable would be "daily
cigarette consumption",
and your two related
groups would be the
cigarette consumption
values "before" and "after"
the hypnotherapy
programme).
Mcnemar  Categorical  dichotomous . Paired t- assume that each of
 1 predictor  groups test several test subjects is
come from affected with tinea
the same pedis(athlete's foot) on
population each foot, and each
subject is given a
treatment X on one foot
and Y on the other
foot. Frequency count from
matched pairs of foot is
measured for both cured
and not cured