WorkStackAboutContact
← All Projects
Independent Project In Progress Experimentation · Causal Inference · Product Analytics

SplitLab —
A/B Testing
Intelligence Platform

A full experimentation platform that decides whether a product change should ship — using power analysis, CUPED variance reduction, sequential testing, Bayesian analysis, uplift modeling, and contextual bandits that shift traffic to the winner automatically.

↗ Live Demo API Docs GitHub

The Problem

A/B testing sounds simple - split users into two groups and compare but naive tests fail in predictable ways. Broken splits go unnoticed. Daily peeking inflates false positives. Small effects require enormous sample sizes that most teams never reach. And a single average hides who a change actually helped or hurt.

SplitLab builds a full testing platform that fixes each of these failure modes, then goes further: a bandit mode that shifts traffic to the winning variant automatically instead of running a fixed 50/50 split until the experiment ends.

What It Does

Power Analysis & Sample Sizing
Computes required sample sizes before an experiment runs. Prevents underpowered tests from shipping false conclusions.
CUPED Variance Reduction
Uses pre-experiment covariates to reduce outcome variance — achieves the same statistical power with significantly fewer users.
Sequential Testing
Allows valid early stopping without inflating false positives. Fixes the peeking problem that breaks most frequentist A/B tests.
Bayesian Analysis
Posterior distributions over treatment effects. Gives probability that variant B beats control — more interpretable than p-values alone.
Uplift Modeling
LightGBM-based heterogeneous treatment effect estimation. Identifies which user segments actually benefit from a change — not just the average effect.
Contextual Bandits
Replaces fixed 50/50 splits with adaptive traffic allocation. Routes users to the better variant automatically as evidence accumulates.

Tech Stack

PythonNumPySciPy statsmodelsLightGBMFastAPI React 18DockerHuggingFace Spaces VercelGitHub Actions
Back to
All Projects
→