Conversion

A/B Testing

Also: split testing

A/B testing is a method of comparing two versions of a page or element by splitting live traffic between them at random, then measuring which version converts better so that the winner is chosen from real behaviour rather than opinion.

The discipline of A/B testing lives in two rules. Change one variable at a time, so that a lift can be attributed to a specific cause rather than a tangle of changes you cannot untangle later. And let the test reach a real sample size before reading it, because a few dozen orders can swing wildly by chance and tempt you into the wrong conclusion.

That second rule is where most stores go wrong. A version that looks like a clear winner after two days often regresses once enough visitors have seen it, so calling a test early is the most common way to ship a change that does nothing or quietly hurts. Decide the duration and the minimum number of conversions in advance, and resist the urge to peek and declare victory.

A/B testing rewards patience more than cleverness. Most tests come back flat or inconclusive, which is itself a useful result: it tells you the element you were arguing about does not move the number, and you can stop spending attention on it.