5.3 Take-Home Points

• Null hypothesis significance test results should be interpreted in relation to sample size and, if possible, test power.

• Statistically significant results do not have to be relevant or important. A small, negligible difference between the sample outcome and the hypothesized population value can be statistically significant in a very large sample with high test power.

• A practically relevant and important difference between the sample outcome and the hypothesized population value does not have to be statistically significant in a small sample because of low test power.

• Give priority to effect size over statistical significance in your interpretation of results.

• A confidence interval shows us how close to and distant from the hypothesized value the plausible population values are. It helps us to draw a more nuanced conclusion about the result than a null hypothesis significance test.

• Applying statistical inference to data other than random samples requires justification of either a theoretical population or a data generating process with a particular probability distribution.