Let’s lay the foundations of A/B testing with its definition and brief history.
A/B testing, at its simplest, is a way to compare two versions of something, in our digital product case it is a web page or an app, to figure out which performs better.
In A/B tests, users are randomly split between the two groups where one group sees the original version (Control) and the other sees the new version (Variation) as shown in the figure below:
During the tests, we collect the necessary metrics and in the end we prove or disprove our hypothesis by analyzing the data we collected.
Scurvy was the leading cause of death among seamen in the 18th century. James Lind, a physician in the British Royal Navy, ran a test to prove his theory that citrus fruits could cure scurvy. For this purpose, he divided the 12 sick sailors into six pairs, and provided each of them with a different supplement in their diet: cider, vitriolic elixir, vinegar, sea water, two oranges and a lemon, or a purgative mixture.
As a result of what some have considered the first clinical trial in history, only the two sailors who took the citrus fruits improved. Of course it was a bit far away from today’s A/B testing standards and statistics since it was based on a small number of observations, but in the end the result was convincing and decisive enough since the impact was so dramatic.
Ronald Fisher designed experiments to determine what effect various external factors had on the growth of plants and crops. He was the first to figure out the basic principles, statistics and mathematics behind A/B testing and make them a science. His work is considered to be the starting point of modern hypothesis testing.
Google engineers ran their first A/B test to determine the optimum number of results to display on its search results page. The first test was unsuccessful due to glitches that were caused by long loading times, but that in itself yielded a critical insight — tenths of a second could make or break user satisfaction in a precisely quantifiable way.
A/B testing is like a standard in today’s world since it is especially useful in combination with agile software development. Microsoft and several other leading companies including Amazon, Booking.com, Facebook, and Google each conduct more than 10.000 A/B tests annually, with many tests engaging millions of users. Today, A/B testing is utilized by businesses of all sizes across various industries.
While the things we test change and evolve everyday, it is quite stunning to see that A/B testing has been helping us for ages to establish causalities and understand more about the world whether online or offline.
We’ll be discussing why we are doing A/B tests next time. Stay tuned!
References:
https://cxl.com/blog/ab-testing-guide/
https://hbr.org/2017/06/a-refresher-on-ab-testing
Article writer: Aybala Coşkun Karadayılar
For the A/B testing inquiries of your online assets, please do not hesitate to contact us!
Choosing the right pricing model for a product or service in the B2B space can be a real challenge for many companies. For starters, the product or service needs to be something that makes sense and matches the expectations of…
What is Social Selling? Social selling relies on a business’s network of contacts to generate leads, qualify them, and close deals. It has become particularly popular in B2B sales because of the ability to generate, qualify and nurture leads at…
At some point in your company, it might make sense to add a marketing person to your senior team. good leadership. Things like buy-ins, sell-outs, and driven marketing leadership you don't get from junior or mid-level team members, freelancers, or…
Estonia
Harju maakond, Tallinn, Kesklinna linnaosa, Valge tn 13, 11415
Email: info@frandity.com