13 sep 2018

TOC

Null Hypothesis
Significance Testing

Neyman-Pearson Paradigm

\(H_0\) and \(H_A\)

\(H_0\)

  • Skeptical point of view
  • No effect
  • No preference
  • No Correlation
  • No difference

\(H_A\)

  • Refute Skepticism
  • Effect
  • Preference
  • Correlation
  • Difference

Frequentist probability

  • Objective Probability
  • Relative frequency in the long run

Standard Error

95% confidence interval

\[SE = \frac{\text{Standard deviation}}{\text{Square root of sample size}} = \frac{s}{\sqrt{n}}\]

  • Lowerbound = \(\bar{x} - 1.96 \times SE\)
  • Upperbound = \(\bar{x} + 1.96 \times SE\)

Binomial \(H_0\) distribution

n = 10   # Sample size
k = 0:n  # Discrete probability space
p = .5   # Probability of head

munt = 0:1

permutations = factorial(n) / ( factorial(k) * factorial(n-k) )
# permutations

p_k  = p^k * (1-p)^(n-k)  # Probability of single event
p_kp = p_k * permutations # Probability of event times 
                          # the occurrence of that event

title = "Binomial Null distribution"

# col=c(rep("red",2),rep("beige",7),rep("red",2))

barplot( p_kp, 
         main=title, 
         names.arg=0:10, 
         xlab="number of head", 
         ylab="P(%)", 
         col='beige',
         ylim=c(0,.3) )

# abline(v = c(2.5,10.9), lty=2, col='red')

text(.6:10.6*1.2,p_kp,round(p_kp,3),pos=3,cex=.5)

Binomial \(H_A\) distribution

Decision table

\(H_0\) = True \(H_0\) = False
Decide to
reject \(H_0\)
Type I error
Alpha \(\alpha\)
Correct
True positive = Power
Decide not
to reject \(H_0\)
Correct
True negative
Type II error
Beta \(\beta\)

Alpha \(\alpha\)

  • Incorrectly reject \(H_0\)
  • Type I error
  • False Positive
  • Criteria often 5%
  • Distribution depends on sample size

Power

  • Correctly reject \(H_0\)
  • True positive
  • Power equal to: 1 - Beta
    • Beta is Type II error
  • Criteria often 80%
  • Depends on sample size