What is A/B Testing?
A/B testing refers to randomized experiments consisting of two variants, A and B. It involves the use of statistical hypothesis testing or “two-sample hypothesis testing” as used in the field of statistics. Through this test, one has the opportunity to compare two versions of a single variable by comparing a subject’s response to Variant A with Variant B. The two versions are compared. It is determined which of the two variants is more effective.
A short video explanation can be found here:
Definition of version A and B
The versions are identical except for one variation, so this can affect a user’s behavior:
- Version A may be the version currently in use (control),
- Version B will be changed in some way (treatment).
On an e-commerce website, the purchase funnel is usually a good candidate for A/B testing, as even minor improvements in drop-off rates can mean a significant increase in sales. Sometimes significant improvements can be seen when elements such as text, layouts, images, and colors are tested.
Multivariate testing or multinomial testing is similar to A/B testing, but can test more than two versions at once or use more controls. Simple A/B testing does not apply to observational, quasi-experimental, or other non-experimental situations, as is the case with survey data, offline data, and other, more complex phenomena.
For more information on the topic of A/B testing, click here:
Do you have any more questions?