Quick answer: A p-value is the probability of seeing data at least as extreme as yours if the null hypothesis were true. A small p-value means your result would be very surprising under that assumption, which is evidence against the null. It is not the probability that your hypothesis is true, and 0.05 is a convention, not a law.
What is a p-value in plain words?
Imagine a coin you suspect is unfair. You flip it 100 times and get 70 heads. The null hypothesis says the coin is fair. How likely is a fair coin to land heads 70 or more times out of 100? Very unlikely: that probability, calculated under the assumption the coin is fair, is the p-value. Here it is about 0.00004. So either you witnessed a rare coincidence, or the coin really is biased. That is the entire logic: the p-value measures how surprised the null hypothesis would be.
What p-values do NOT mean
- Not the probability the null hypothesis is true. The p-value is computed assuming it is true, so it cannot also tell you its truth probability.
- Not the probability your result is a fluke. “Fluke” language hides the same confusion; the p-value is about data extremes under one assumption, not about causes.
- Not a measure of effect size. A tiny p-value can accompany a trivially small difference if the sample is enormous. Always report effect size with it.
- Not proof. Statistical evidence is cumulative and contextual; one small p-value starts a conversation, it does not end one.
How to interpret a p-value: worked examples
As the PMC guidance in clinical research puts it, the p-value should be interpreted as a continuous measure of evidence, not a strict pass or fail. In practice:
| p-value | What it suggests | Typical language |
|---|---|---|
| 0.6 | Data are unremarkable under the null | No evidence of an effect |
| 0.12 | Somewhat unusual, but common by chance | Weak evidence, keep collecting |
| 0.04 | Surprising under the null | Evidence against the null at the 0.05 level |
| 0.004 | Very surprising under the null | Strong evidence against the null |
Notice the language: evidence, not proof. A p-value of 0.04 means the observed result would arise by chance about 4 times in 100 if the null were true. Whether that justifies action depends on the cost of being wrong, the study design, and whether the result replicates.
Why 0.05 is a convention, not magic
The 0.05 threshold traces to R. A. Fisher’s 1925 tables, chosen as a convenient cutoff for “worth a second look.” Physics often demands 0.0000003 (the five-sigma standard for particle discoveries). Genomics corrects for millions of simultaneous tests and uses thresholds near 0.00000005. Pick your significance level before looking at the data, based on how costly a false positive is in your field, and report the actual p-value rather than just the verdict. Our hypothesis testing guide shows where the p-value sits in the full testing procedure.
If you want to go beyond reading outputs to understanding exactly which test to run and why, Ampersand Academy teaches statistics one-to-one with your own data as the material.
Frequently asked questions
Is a p-value of 0.05 statistically significant?
By the common convention, yes: 0.05 is the most used significance cutoff, so a p-value below it counts as significant in most fields. It is a threshold the field agreed on, not a mathematical boundary, and many journals now prefer reporting exact p-values.
What does a p-value of 0.001 mean?
If the null hypothesis were true, data at least this extreme would occur about 1 time in 1000. That is strong evidence against the null, but it still says nothing about effect size or practical importance.
Can a p-value prove the null hypothesis?
No. A large p-value only means your data are unremarkable under the null. Absence of evidence is not evidence of absence; equivalence testing or Bayesian methods are the proper tools for supporting a null.
Does a small p-value mean a big effect?
No. With a large enough sample, even a trivially small difference produces a tiny p-value. Always report an effect size, such as a mean difference or correlation, alongside the p-value.
Why do researchers still use 0.05 if it is arbitrary?
Tradition and comparability. A shared threshold makes results readable across studies, but the modern recommendation is to treat the p-value as continuous evidence and state your chosen level and reasoning rather than hiding behind a fixed rule.