Why non-parametric tests
When data is:
- Not normally distributed
- Heavy-tailed
- Contains outliers
…it can be safer to compare median/rank behavior.
Mann–Whitney U test (two independent groups)
Alternative to the independent t-test.
import numpy as np
from scipy import stats
A = np.array([1, 2, 2, 3, 100])
B = np.array([1, 1, 2, 2, 3])
u, p = stats.mannwhitneyu(A, B, alternative="two-sided")
print("U:", u)
print("p:", p)Wilcoxon signed-rank (paired)
Alternative to the paired t-test.
import numpy as np
from scipy import stats
before = np.array([10, 12, 11, 9, 13])
after = np.array([11, 12, 12, 10, 14])
w, p = stats.wilcoxon(after - before)
print("W:", w)
print("p:", p)Notes
- Non-parametric tests often have less power when assumptions for parametric tests are satisfied.
- Always combine tests with visualizations.
Choosing parametric vs. non-parametric
flowchart TD A["Comparing two groups"] --> B["Data roughly normal,
no big outliers?"] B -->|"Yes"| C["t-test
(parametric)"] B -->|"No"| D["Same subjects twice?"] D -->|"Yes"| E["Wilcoxon
signed-rank"] D -->|"No"| F["Mann-Whitney U"]
Check your understanding
Which situation favors a non-parametric test?
Rank and median-based tests are safer when data is non-normal, heavy-tailed or has outliers.
Show answer
B — Heavy-tailed data or data containing outliers — Rank and median-based tests are safer when data is non-normal, heavy-tailed or has outliers.
Which test is the alternative to the independent t-test?
Mann-Whitney U compares two independent groups. Wilcoxon signed-rank is the paired alternative.
Show answer
B — Mann-Whitney U — Mann-Whitney U compares two independent groups. Wilcoxon signed-rank is the paired alternative.
In the Mann-Whitney example, group A contains 100 among small numbers. Why prefer Mann-Whitney there?
Rank-based comparison treats 100 as just the largest rank rather than a huge value.
Show answer
A — Ranks limit the influence of the outlier 100 — Rank-based comparison treats 100 as just the largest rank rather than a huge value.
What trade-off do non-parametric tests make?
Rank tests often have less power if the parametric assumptions are satisfied.
Show answer
B — Slightly less power when parametric assumptions hold, for more robustness — Rank tests often have less power if the parametric assumptions are satisfied.
🧪 Try It Yourself
Exercise 1 – Mann-Whitney U test
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 2 – Wilcoxon signed-rank test
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 3 – Deciding which test fits
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Next
Continue to A/B Testing Basics to apply these comparison tests to a real product experiment.