Confidence Intervals (CI)

What a confidence interval means

A 95% confidence interval is not:

“There is a 95% probability the true mean is in this interval.”

It is:

“If we repeat this sampling process many times, ~95% of such intervals will contain the true mean.”

CI for a mean (using t-distribution)

Use when:

Population std is unknown (common)
Sample size is small-ish

CI for mean

import numpy as np
from scipy import stats
 
x = np.array([12, 13, 10, 9, 11, 12, 14, 8, 13, 11])
 
n = len(x)
mean = x.mean()
se = x.std(ddof=1) / np.sqrt(n)
 
alpha = 0.05
 
t_crit = stats.t.ppf(1 - alpha/2, df=n-1)
ci = (mean - t_crit * se, mean + t_crit * se)
 
print("mean:", mean)
print("95% CI:", ci)

CI for mean

import numpy as np
from scipy import stats
 
x = np.array([12, 13, 10, 9, 11, 12, 14, 8, 13, 11])
 
n = len(x)
mean = x.mean()
se = x.std(ddof=1) / np.sqrt(n)
 
alpha = 0.05
 
t_crit = stats.t.ppf(1 - alpha/2, df=n-1)
ci = (mean - t_crit * se, mean + t_crit * se)
 
print("mean:", mean)
print("95% CI:", ci)

CI for a proportion

Example: conversion rate.

CI for proportion (normal approx)

import numpy as np
 
# 60 conversions out of 1000
x = 60
n = 1000
p_hat = x / n
 
alpha = 0.05
z = 1.96
 
se = np.sqrt(p_hat * (1 - p_hat) / n)
ci = (p_hat - z * se, p_hat + z * se)
 
print("p_hat:", p_hat)
print("95% CI:", ci)

CI for proportion (normal approx)

import numpy as np
 
# 60 conversions out of 1000
x = 60
n = 1000
p_hat = x / n
 
alpha = 0.05
z = 1.96
 
se = np.sqrt(p_hat * (1 - p_hat) / n)
ci = (p_hat - z * se, p_hat + z * se)
 
print("p_hat:", p_hat)
print("95% CI:", ci)

Practical notes

CI width depends on sample size: larger n → tighter CI.
CI helps decision-making: overlap/non-overlap gives a quick signal (not a full test).

Building a CI, step by step

diagram How a confidence interval is built mermaid

  flowchart LR
  A["Sample mean"] --> B["Standard error
(s / sqrt n)"]
  B --> C["Critical value
(t or z)"]
  C --> D["Margin of error
= critical value x SE"]
  D --> E["CI = mean +/- margin"]

Starting from the sample mean and standard error, a critical value stretches the interval to a chosen confidence level.

Visualize it

Widening the sample size shrinks the confidence interval around the mean. Watch the amber band tighten around the estimate as nn grows — the true mean (dashed line) stays fixed, only your certainty about it improves:

sketch Confidence interval shrinks as n grows p5.js

A wider sample size produces a narrower band around the estimated mean.

🧪 Try It Yourself

Exercise 1 – 95% CI for a mean

Exercise 2 – CI for a conversion rate

Exercise 3 – Sample size shrinks the interval

Continue to Hypothesis Testing to turn “is the true value in this range” into a formal yes/no decision with p-values.

If this helped you, consider buying me a coffee ☕

Buy me a coffee