Comparing Python packages for A/B test analysis (with code examples) (opens in new tab)

(e10v.me)

9 pointse10v_me23d ago1 comments

1 comments

I published a practical comparison of Python packages for A/B test analysis: tea-tasting, Pingouin, statsmodels, and SciPy.

Instead of choosing a single "best" tool, I break down where each package fits and how much manual work is needed for production-style experiment reporting.

Includes code examples and a feature matrix across power analysis, ratio metrics, relative effect CIs, CUPED, multiple testing correction, and working aggregated statistics for efficiency.

Disclosure: I am also the author of tea-tasting.

j / k navigate · click thread line to collapse

1 comments

e10v_meOP23d ago

I published a practical comparison of Python packages for A/B test analysis: tea-tasting, Pingouin, statsmodels, and SciPy.

Instead of choosing a single "best" tool, I break down where each package fits and how much manual work is needed for production-style experiment reporting.

Includes code examples and a feature matrix across power analysis, ratio metrics, relative effect CIs, CUPED, multiple testing correction, and working aggregated statistics for efficiency.

Disclosure: I am also the author of tea-tasting.

j / k navigate · click thread line to collapse