Null Hypothesis Significance Testing

Author

Affiliation

Klinkenberg

University of Amsterdam

Published

September 11, 2023

Null Hypothesis
Significance Testing

Neyman-Pearson Paradigm

Neyman - Pearson

Two hypothesis

\(H_0\)

Skeptical point of view
No effect
No preference
No Correlation
No difference

\(H_A\)

Refute Skepticism
Effect
Preference
Correlation
Difference

Frequentist probability

Objective Probability
Relative frequency in the long run

Standard Error

95% confidence interval

\[SE = \frac{\text{Standard deviation}}{\text{Square root of sample size}} = \frac{s}{\sqrt{n}}\]

Lowerbound = \(\bar{x} - 1.96 \times SE\)
Upperbound = \(\bar{x} + 1.96 \times SE\)

Standard Error

Binomial \(H_0\) distribution

Binomial \(H_A\) distributions

Decision table

Alpha \(\alpha\)

Incorrectly reject \(H_0\)
Type I error
False Positive
Criteria often 5%
Distribution depends on sample size

Power

Correctly reject \(H_0\)
True positive
Power equal to: 1 - Beta
- Beta is Type II error
Criteria often 80%
Depends on sample size

Post-Hoc Power

Also known as: observed, retrospective, achieved, prospective and a priori power
Specificly meaning:

The power of a test assuming a population effect size equal to the observed effect size in the current sample.

Source: O’Keefe (2007)

Effect size

In statistics, an effect size is a quantitative measure of the strength of a phenomenon. Examples of effect sizes are the correlation between two variables, the regression coefficient in a regression, the mean difference and standardised differences.

For each type of effect size, a larger absolute value always indicates a stronger effect. Effect sizes complement statistical hypothesis testing, and play an important role in power analyses, sample size planning, and in meta-analyses.

Source: WIKIPEDIA