By Kyle Reddy

In my previous blog post I discussed A/B testing for online stores and the positive impact it can have on the overall performance of the store. My second installment in the A/B testing series is related to email marketing and why A/B testing emails is essential. In the 45 years since the first email was sent by Ray Tomlinson, emails have evolved into one of the best forms of direct marketing, as it is a fast method of communication that will not break the bank, it is still regarded by many as one of the most effective tools to market your online store and if you are not utilizing its full potential then now is the time to change your thinking and get on board.

Some history on A/B testing in general, Google ran its first A/B test back in the year 2000, and it was unsuccessful due to technical glitches. The glitches were fixed and massive improvements were made and 11 years later, Google have run over 7000 different A/B tests to improve their search engine.

Before you start A/B testing your email campaigns, you need to decide what you are trying to achieve from the tests. Some of the key metrics that can and should be measured are: Open Rates, Click through rates and conversion rates. I will cover the following elements of an email campaign that email marketers should A/B test if they want to improve any of the above mentioned performance metrics.

• Subject Line
• Design
• Design
• Verbiage
• Call-To-Action
• Tags
• Delivery Date & Time

Subject lines are usually a good place to start your testing. The content of the email is important, but if you were giving away a free car to the first person to open the mailer, and nobody bothered to open your email because of a bad subject line, then the content is worthless. So focusing on increasing the open rate should be your first priority. A simple test like the 1 below, were the “You Need to Know” was removed from the subject line had a significant decrease in the open rate. Having a high open rate should be your first prize, as it means recipients are opening your email so you have their attention. It is also worth playing around with the length of the subject line, users may prefer more detail or less, the only way to know what works it to A/B test is and find a pattern in the user behaviour.

Now that the awesome subject line has grabbed your recipient’s attention, you need to make sure that the content of the email is in accordance with the subject line. Depending on what the subject line is you will need to ensure that the design, verbiage and imagery portrays the same message. It is important to include your strategy into the layout, so be sure to discuss this with your design agency. It is also important to remember that images and visuals are a critical part of marketing your products or services. but ensure that imagery does not exceed 30% of the mailer as this will send your email straight to  the spam folder.

Factor in the placement of both imagery and verbiage to ensure that the mailer is eye-catching. For many readers imagery immediately stands out while others may prefer to have a blurb or the main reason for the email like “Buy Now”. This is ideal for creating multiple variations for testing to help understand your user’s preferences. Play around with different variations to help you understand what appeals to your specific database.

To make the mailer financially effective you first need a click-through and then a conversion. It is important to clearly indicate the action expected of the reader. The call-to-action must be direct and tell the user where you want them to click next. It is a good idea to try different words, colours, button styles,

When trying to find the right call-to-action there are several options that you can investigate, some of which include the wording on the call-to-action, the colour of the button, the placement and the size of the actual button. From an A/B testing perspective, the below example is an easy test that you can run which involves changing the wording of the Call-To-Action button. There was a 213.16% increase in the Click Through Rate for just changing the wording.  

This test for the call-to-action button kept the same wording but changed the colour to red which makes the button more outstandings and eye-catching. It clearly points the user in the right direction for what they need to do next. A simple test like this had a 28.7% increase in conversions.

This may not always have a positive impact on conversions. Some changes may go unnoticed to the user, or they may not be enticing the user enough to click on the call-to-action, as you can see in the example below that the change had a negative effect on conversions. Even though the result of the test are negative, the test itself is worth it because this gives you insight into what makes your users tick.

When dealing with promotions, discounts and savings on products in your mailers it is worthwhile to highlight the details with the product image. You can also use it for bringing other elements to your customers attention, with some examples of limited stock on offer, or limited duration sale time, essentially it should be your unique selling point. There are different examples of how to use tags with your products, and these can be tested in many variations. It would also be worthwhile to test the same product with a tag and without to determine effectiveness.

Lastly, make sure you try distributing mailer campaigns to your database on different days of the week and at different times of each day. Depending on what you are marketing to your users, it may work better in the morning or it could have a better open rate and click through rate towards the late afternoon when people are looking for distractions and may be tempted to shop online as the day slows down. Also take into consideration how many people view their emails on the mobile devices in the evenings. Sending marketing campaigns in the evenings have at times had better open rates and click through rates than during working hours. So there are many tests that can be run to determine the best time for your clients and what works for your database may not work for another database as user behaviour is unique so test to your hearts content.

A/B testing the various elements on your mailer can help to make each campaign more successful than the previous campaign. Each campaign will bring learnings that can be used to improve the next campaign. Keep in mind that negative results are still beneficial and that some changes made may have no impact at all. Google has already run 7000+ A/B tests and they won’t stop there, that number will continue to increase and they will continue to A/B test to be better. You cannot afford to be left behind in the ecommerce space with emails that are generic and not inline with your users behaviour, if you have not started A/B testing your email campaigns as yet, then now is the time to start.