Even though A/B testing has already been around for several years it has only become part of mainstream online marketing in the last year or so in South Africa. So I thought it would be a great idea to share the story of how I got involved in A/B testing and what my initial experience was.

Back in 2014, A/B testing was the new “buzz word” in my vocabulary. After having heard multiple success stories from my colleagues, I researched everything I needed to know about A/B testing by reading up on all the blogs and articles that I could find online and found several stories of improvements ranging from 5% to 300% across various types of online stores.

I immediately began scrutinizing each of the online stores in my portfolio for elements and areas that I would like to test. Playing devil’s advocate by going through each and every webpage that a user would browse to and analyze each element in order to determine the value, the benefit and the necessity of each element. Lots of research has already been done by similar websites, and so I was able to make calculated and educated decisions. The aesthetics and user experience varied, but essentially the user journeys were identical; (1) browse products, (2) add products to cart, (3) go to shopping cart, (4) checkout.

I mapped out all the goals with their respective tests that would be implemented. For an online store, each area has a tailored primary goal, with five strategies and three tactics (A/B Tests) per strategy, structuring each so that overlapping tests did not coincide with each other as that would have skewed the tests and the subsequent data.Multiple tests were launched simultaneously across the different areas of the websites. Anticipation and expectations were high from both the clients and myself.

In contrast to A/B testing done in the USA, the traffic for the tests were slower in gathering momentum as traffic volumes to the websites in SA were not at the same level as those in the USA. Depending on the tests being used, website visitors would not necessarily interact with your test changes. The traffic to the website was there, the transactions were there, but the volumes were not substantial enough to make a clear-cut decision as quickly as expected.

Online shopping in the US compared to SA is very different, so I needed to adjust both my and the clients’ expectations in line with the market in SA. The answer was to increase the lifespan to a period of 3 - 6 months for several tests that were live and for queued tests, as this would be sufficient time for the sample to be substantial enough to make decisions based on factual results.

In that time a new article had come across my desk, which was about how A/B testing was helpful even if the final test results were not good. It looked at a very interesting angle of how A/B test results highlight what the consumers are already comfortable with, and that they may be happy to interact with elements of the website that you may have previously thought needed workl. This would not be known if there was nothing to test against, so we are able to take the positives out of the negative test results. This was something I had not considered, previously, thinking that the tests I created would all need to perform better than the originals.

The tests yielded improvements, and while they may not have been as astronomical as 300%, each and every improvement counts in the bigger picture. Even if results show 5% - 20% improvements per test, these are still improvements to the online store holistically, which when combined with other tests will help to give website visitors a better user experience and improve the performance of the website.

The tests ran their course over the next few months and there were many positives to take from them. Below are the results from three tests run for one campaign.

• Removal of the Navigation Menu in the shopping page had an overall improvement of 6.4%. This is the percentage of users the proceeded to the checkout page, 6.4% more users engaged with the Call-To-Action button in the variation than on the original website shopping cart
• Removal of the header, footer, and other distractions in the checkout page resulted in an improvement of 18.7%, meaning that 18.7% of the users to reach this page continue the checkout process compared to those that viewed the original website checkout page
• In addition to this, the engagement level of the page was better than the original by 8.9%, keeping in mind that there were less clickable options available to the user as the unnecessary elements were removed, resulting in a better user experience.

In another campaign the results were contrasting as highlighted below.

• Compared the to above campaign, the removal of the Navigation Menu in the shopping cart resulted in a decrease of -1.7%. This means that users preferred the original website page compared to the variation that I had created.
• The result that stands out more is an increase of 686% of users clicking on the brand logo to leave the shopping cart to go back into the website. This could indicate that users still wish to continue shopping and are not necessarily ready to checkout.
• The Enclosed Checkout test yielded more positive results with an improvement of 7.2% over the original website checkout page. This means that a checkout page without a header, footer, navigation menu or any other distractions led more users to convert.

Overall the results from the A/B testing proved to be beneficial to all my clients online stores. Even though the testing time was longer than expected the results were incredibly informative and change the client’s website strategies. So for all you new to A/B testing my final thoughts are that A/B Testing is here to stay indefinitely. Each test helps us learn more about our users, what they like and what will send them to your competitor websites.The smallest changes can result in a massive impact on your sales and ultimate success.