ANOVA (Analysis of Variance)

ANOVA is a statistical technique that assesses potential differences in a scale-level dependent variable
by a nominal-level variable having 2 or more categories. For example, an ANOVA can examine potential
differences in IQ scores by Country (US vs. Canada vs. Italy vs. Spain). The ANOVA, developed by Ronald
Fisher in 1918, extends the t and the z test which have the problem of only allowing the nominal level
variable to have two categories. This test is also called the Fisher analysis of variance.

General Purpose of ANOVA

Researchers and students use ANOVA in many ways. The use of ANOVA depends on the research
design. Commonly, ANOVAs are used in three ways: one-way ANOVA, two-way ANOVA, and N-way
ANOVA.

One-Way ANOVA

A one-way ANOVA has just one independent variable. For example, difference in IQ can be assessed by
Country, and County can have 2, 20, or more different categories to compare.

Two-Way ANOVA

A two-way ANOVA refers to an ANOVA using two independent variables. Expanding the example above,
a 2-way ANOVA can examine differences in IQ scores (the dependent variable) by Country (independent
variable 1) and Gender (independent variable 2). Two-way ANOVA can be used to examine the
interaction between the two independent variables. Interactions indicate that differences are not
uniform across all categories of the independent variables. For example, females may have higher IQ
scores overall compared to males, but this difference could be greater (or less) in European countries
compared to North American countries. Two-way ANOVAs are also called factorial ANOVAs.

N-Way ANOVA
A researcher can also use more than two independent variables, and this is an n-way ANOVA (with n
being the number of independent variables you have). For example, potential differences in IQ scores
can be examined by Country, Gender, Age group, Ethnicity, etc, simultaneously.

General Purpose – Procedure

Omnibus ANOVA test:

The null hypothesis for an ANOVA is that there is no significant difference among the groups. The
alternative hypothesis assumes that there is at least one significant difference among the groups. After
cleaning the data, the researcher must test the assumptions of ANOVA. They must then calculate the F-
ratio and the associated probability value (p-value). In general, if the p-value associated with the F is
smaller than .05, then the null hypothesis is rejected and the alternative hypothesis is supported. If the
null hypothesis is rejected, one concludes that the means of all the groups are not equal. Post-hoc tests
tell the researcher which groups are different from each other.

So what if you find statistical significance? Multiple comparison tests

When you conduct an ANOVA, you are attempting to determine if there is a statistically significant
difference among the groups. If you find that there is a difference, you will then need to examine where
the group differences lay.

At this point you could run post-hoc tests which are t tests examining mean differences between the
groups. There are several multiple comparison tests that can be conducted that will control for Type I
error rate, including the Bonferroni, Scheffe, Dunnet, and Tukey tests.

Types of Research Questions the ANOVA Examines

One-way ANOVA: Are there differences in GPA by grade level (freshmen vs. sophomores vs. juniors)?

Two-way ANOVA: Are there differences in GPA by grade level (freshmen vs. sophomores vs. juniors) and
gender (male vs. female)?
Data Level and Assumptions

The level of measurement of the variables and assumptions of the test play an important role in ANOVA.
In ANOVA, the dependent variable must be a continuous (interval or ratio) level of measurement. The
independent variables in ANOVA must be categorical (nominal or ordinal) variables. Like the t-test,
ANOVA is also a parametric test and has some assumptions. ANOVA assumes that the data is normally
distributed. The ANOVA also assumes homogeneity of variance, which means that the variance among
the groups should be approximately equal. ANOVA also assumes that the observations are independent
of each other. Researchers should keep in mind when planning any study to look out for extraneous or
confounding variables. ANOVA has methods (i.e., ANCOVA) to control for confounding variables.

Testing of the Assumptions

The population from which samples are drawn should be normally distributed.

2. Independence of cases: the sample cases should be independent of each other.

3. Homogeneity of variance: Homogeneity means that the variance among the groups should be
approximately equal.

These assumptions can be tested using statistical software (like Intellectus Statistics!). The assumption
of homogeneity of variance can be tested using tests such as Levene’s test or the Brown-Forsythe Test.
Normality of the distribution of the scores can be tested using histograms, the values of skewness and
kurtosis, or using tests such as Shapiro-Wilk or Kolmogorov-Smirnov. The assumption of independence
can be determined from the design of the study.

It is important to note that ANOVA is not robust to violations to the assumption of independence. This
is to say, that even if you violate the assumptions of homogeneity or normality, you can conduct the test
and basically trust the findings. However, the results of the ANOVA are invalid if the independence
assumption is violated. In general, with violations of homogeneity the analysis is considered robust if
you have equal sized groups. With violations of normality, continuing with the ANOVA is generally ok if
you have a large sample size.

Related Statistical Tests: MANOVA and ANCOVA