The basic workflow

1. Define null hypothesis (H0) and alternative (H1)
2. Choose a significance level (\alpha) (commonly 0.05)
3. Compute a test statistic and p-value
4. Decide whether to reject H0

P-value intuition

A p-value is:

• Probability of seeing results at least as extreme as your observation if H0 were true.

Low p-value means:

• Data is unlikely under H0 → evidence against H0

Errors

• Type I error: reject H0 when H0 is true (false positive). Probability ≈ (\alpha).
• Type II error: fail to reject H0 when H0 is false (false negative). Probability = (\beta).
• Power: (1 - \beta)

Example: one-sample t-test

Is the average delivery time different from 30 minutes?

import numpy as np
from scipy import stats
 
x = np.array([28, 32, 31, 29, 26, 30, 33, 27])
 
t_stat, p = stats.ttest_1samp(x, popmean=30)
print("t:", t_stat)
print("p:", p)

import numpy as np
from scipy import stats
 
x = np.array([28, 32, 31, 29, 26, 30, 33, 27])
 
t_stat, p = stats.ttest_1samp(x, popmean=30)
print("t:", t_stat)
print("p:", p)

Good practice

• Report effect size + CI, not just p-value.
• Don’t equate “not significant” with “no difference”.
• Predefine your hypothesis; avoid testing many variants blindly.