A/B Testing or Split Testing

Compare two versions of an webpage to identify the better performer. based on real data.

In A/B testing
A — refers to ‘control’ or the original part of webpage.
B — refers to ‘variation’ or a new version.

Idea- A/B testing is one of main component into process of Conversion Rate Optimization (CRO), using which we can gather user insights. we can use this also for collecting data to understand user behavior, engagement rate, pain points.

Different types of A/B tests

Split URL testing — When you run a Split URL test, your website traffic is split between two url’s first one as original url, second url with variations.
this is recommended testing method for dynamic content.

Multipage testing — Test changes to particular elements across multiple pages. based on this you can analyse impact across an entire flow. It enables to implement the same change across pages to ensure that web page visitors get consistent experiences.

Multivariate testing- variations of multiple page variables are simultaneously tested to analyze which combination is performing better. it helps in avoiding the need to conduct several sequential A/B tests and saves time.

Approaches to run an A/B test-

Frequentist Approach — Requires test to run for a set period to get correct data from it but can’t figure out how close or far A and B actually are.
data used from current experiment to draw conclusions.

Bayesian Approach- You can now plan better, have a more accurate reason to end tests. use existing data to draw conclusions. the Bayesian approach provides actionable results almost 50% faster then Frequentist Approach