STATISTICAL PARAMETER P-VALUE

P value
The p value is a number, calculated from a statistical test, that describes how likely to have
found a particular set of observations if the null hypothesis were true. P values are used
in hypothesis testing to help decide whether to reject the null hypothesis. The smaller the p value,
the more likely you are to reject the null hypothesis.
• The p value, or probability value, tells how likely data could have occurred under the null
hypothesis. It does this by calculating the likelihood of test statistic, which is the number
calculated by a statistical test using your data.
• A P-value less than 0.5 is statistically significant, while a value higher than 0.5 indicates
the null hypothesis is true; hence it is not statistically significant.
• The p value tells how often expect to see a test statistic as extreme or more extreme than
the one calculated by statistical test if the null hypothesis of that test was true.
• The p value gets smaller as the test statistic calculated from your data gets further away
from the range of test statistics predicted by the null hypothesis.
The p value is a proportion: if p value is 0.05, that means that 5% of the time you would see a
test statistic at least as extreme as the one can found if the null hypothesis was true. The p value
will never reach zero, because there’s always a possibility, even if extremely unlikely, that the
patterns in data occurred by chance.
Calculating the p value
P-Values are usually calculated using p-value tables or spreadsheets, or calculated
automatically using statistical software like R, SPSS, etc.
One can also find tables for estimating the p value of test statistic online. These tables
show, based on the test statistic and degrees of freedom (number of observations minus number
of independent variables) of test, how frequently expect to see that test statistic under the null
hypothesis.
The calculation of the p value depends on the statistical test to test hypothesis:
• Different statistical tests have different assumptions and generate different test statistics.
Choosing the statistical test that best fits the data and matches the effect or relationship to
test.
• The number of independent variables you include in your test changes how large or small
the test statistic needs to be to generate the same p value.
Choosing a statistical test
Comparing only two different diets, then a two-sample t test is a good way to compare the
groups. To compare three different diets, use an ANOVA instead – doing multiple pair-wise
comparisons will result in artificially low p values and over estimate the significance of the
difference between groups.
P-Values and Statistical Significance
The level of statistical significance is usually represented as a P-value between 0 and 1.
The smaller the p-value, the more likely it is that you would reject the null hypothesis. P-Values
are most often used by researchers to say whether a certain pattern they have measured is
statistically significant
• A P-Value < or = 0.05 is considered statistically significant. It denotes strong evidence against
the null hypothesis, since there is below 5% probability of the null being correct. So, we reject
the null hypothesis and accept the alternative hypothesis.
• But if P-Value is lower than your threshold of significance, though the null hypothesis can be
rejected, it does not mean that there is 95% probability of the alternative hypothesis being true.
• A P-Value >0.05 is not statistically significant. It denotes strong evidence for the null
hypothesis being true. Thus, we retain the null hypothesis and reject the alternative hypothesis.
We cannot accept null hypothesis; we can only reject or not reject it.
P-values are expressed as decimals and can be converted into percentage. For example, a p-
value of 0.0237 is 2.37%, which means there's a 2.37% chance of your results being random or
having happened by chance. The smaller the P-value, the more significant your results are.
Example: Statistical significance comparison of the two mouse diets results in a p value of less
than 0.01, below alpha value of 0.05; therefore determine that there is a statistically significant
difference between the two diets.
In our comparison of mouse diet A and mouse diet B, we found that the lifespan on diet A
(M = 2.1 years; SD = 0.12) was significantly shorter than the lifespan on diet B (M = 2.6
years; SD = 0.1), with an average difference of 6 months (t(80) = -12.75; p < 0.01).
P Value vs Alpha Level
P-value indicates the probability of getting an effect no less than that actually observed in
the sample data.
An alpha level will tell you the probability of wrongly rejecting a true null hypothesis. The
level is selected by the researcher and obtained by subtracting your confidence level from 100%.
For instance, if you are 95% confident in your research, the alpha level will be 5% (0.05).
When you run the hypothesis test, if you get:
• A small p value (<=0.05), you should reject the null hypothesis
• A large p value (>0.05), you should not reject the null hypothesis
Reporting P-Values
• You should report exact P-Values upto two or three decimal places.
• For P-values less than .001, report as p < .001.
• Do not use 0 before the decimal point as it cannot equal1. Write p = .001, and not p = 0.001
• Make sure p is always italicized and there is space on either side of the = sign.
• It is impossible to get P = .000, and should be written as p < .001
Caution when using p values
• P values are often interpreted as risk of rejecting the null hypothesis of test when the null
hypothesis is actually true.
• In reality, the risk of rejecting the null hypothesis is often higher than the p value, especially
when looking at a single study or when using small sample sizes. This is because the smaller
frame of reference, the greater the chance that you stumble across a statistically significant
pattern completely by accident.