VWO Logo
Like this post?
Read our in-depth guide to A/B Testing
Share this post
9 Min Read

Why Email A/B Testing Should Be Your First Step As A Marketer

placeholder
Nida Zehra
Content Marketer at VWO

When you step foot in the enthralling world of CRO (hey, we do it for a living, we better have fun along the way!), we are inundated with a gracious sprinkling of bylines – “Landing Page Optimization”, “A/A”, “A/B/N”, “Above The Fold”, “Below The Fold” – phew!

What we seldom talk about though, is email marketing done right by leveraging A/B Testing. Year after year, email marketing delivers highest ROI across all acquisition channels.[1]

It’s only natural that we distribute some of our content hours to address the many ways email marketing can deliver even higher ROI by inculcating A/B testing best practices, methodologies, and mental models.

bar graph highlighting the Return on Investment from email campaigns

Image: Source [1]

What is Email Marketing?

Email marketing is a direct form of marketing, utilizing emails to promote a business. It’s a form of relationship marketing that most brands do to directly engage with their potential customers and inform and update them.

Why Email Marketing?

Email beats social when it comes to customer acquisition by 40x, according to a McKinsey report. Your prospect takes your brand more seriously over email communication as compared to when they engage on social. Yes, email is popular.

Now, let’s understand how the game of email marketing is going strong in an era of social media. Since smartphones are ubiquitous, people tend to check their emails on a 24/7 accessible device, which is invariably their phones. A study by Exacttarget found that 91% of people use smartphones to check their email as compared to 75% of people who use their smartphones for social media.[2] Not leveraging such a group for email marketing means missing out on a potential channel, which is extensively used across the globe.

As compared to other forms of marketing, email marketing is a very personalized form of marketing to reach out and engage with your target customers.It is cost-effective, and the associated ROIs are higher vis-a-vis other forms of marketing, such as display ads and other traditional ways. 

Email marketing enables businesses to not only reach out to targeted masses at a rate of nearly nothing per customer but also to track the metrics easily. 

The typical metrics for a successful email marketing campaign are open and click-through rates. To maximizing your open, click, and response rate, you need to design your campaigns carefully. 

You can do this by testing your designed campaigns and understand what works best for you.

Why Should You A/B Test Emails?

A/B testing is a great tool to help improve your email marketing performance – but only if you know what you’re doing. 

Experimenting with your email campaigns would allow you to compare and contrast the elements that would gauge their impact on your subscribers’ reactions. 

an illustration of A/B testing

A/B testing can be done by forming a hypothesis and then formulating a test to see the results.

It is a great tool to:

  • Increase the open and click-through rates:

The tool allows you to identify common factors and trends that can help increase open and click-through rates. For example, make your emails mobile-friendly, show relevant information in images, use a responsive email design that will allow it to all screen sizes. 

  • Increase conversion rate and generate revenue: 

With high click-through rates, there are chances that your subscribers are clicking on your website and engaging with the call-to-action (CTA) buttons such as filling forms. You may identify how many open emails led to sales or leads. 

  • Get clarity about your target group’s expectations:

Your communication should be relevant to your target group’s interest. You need to know them well to understand their behavior, and hence, their expectations. Craft personalized emails by addressing them with their names. You can do so by creating targeted content for the segmented list and address their pain points with a solution by privately arriving in their mailbox. 

  • Save time and money:

A lot of money, energy, and time is spent on creating landing pages, forms, and sending emails to a broad audience.Through A/B testing, you can formulate the right recipe for a successful campaign by implementing incremental changes that, leading to statistically significant results. 

What Elements Should You A/B Test in Your Emails?

A/B testing or split testing is the practice of showing two variants of the same web page to different segments of visitors at the same time and comparing which variant drives more conversions. Typically, the one that gives higher conversions is the winning variant, applying, which can help you optimize your site for better results.

You can A/B test the email campaigns by sending a portion of your list one version of an email and the other a different version while you watch for changes in your open rate, click-through rate, or other KPI. 

The elements that you can A/B test are:

  1. Email subject line:

Test the subject line for its length, topic, personalization, and the kind of promotion you’re planning to run through the emails.

how to set up a/b testing of subject line for an email campaign

2. Preheader:

Try and include a preheader, and test it against preheader subject lines you may hypothesize would help increase open rate. You can also try personalizing the preheaders by adding the first name of the email recipient.

3. Different time and days:

You must test different time zones as you may be catering to a global audience, which means that send time of an email will affect your open rate. 

Send emails on weekdays as people are most likely to check their emails on weekdays as compared to weekends. For example, if you always send your newsletter on Tuesdays, try sending it on Wednesday at the same time to see what happens.

Many studies have been done on the best days and times to send emails. Here’s a great display of open and click-through rates from GetResponse, analyzing 21 million US-based emails.

comparison of email campaign open rate across different days of the week
Comparing open rate across different days of the week
comparison of click rate for email campaigns across different days of the week
Comparing click rate across different days of the week

Also, testing periods of time to send a message on users abandoning the site would help you optimize the open and click-through rates, which eventually affect your sales.

4. Visuals:

Since our brain reacts better to visuals than text, you make your copy super engaging by incorporating compelling visuals.

an example of an email from House of Fraser containing Visuals

5. Personalization:

Besides preheaders, there is nothing like personalizing your email design as per the expectations of your prospects. You can only accomplish this through effective communication. Your audience actually listens to you when you address them by their names, greet them through the right channel, at the right moment, with the right message. They are most likely to engage with your content if they entrust you with the solutions to their pain points. 

an example of a personalized email from LinkedIn Learning inside the inbox
Source: Biz Journals

6. Content:

Content is the king! Your content is the copy you write, images, videos, gifs, and their placement in a particular layout to make your email look engaging. 

Below are a few interesting parameters that you should test in your content:

  • Length: You must A/B test your copy for its length to understand and conclude if your audience prefers short-form copy with links or a long-form copy to delve deeper. 
  • Specific or generic: You don’t know what would work best for your target group, so it is recommended that you test your copies as of specific and generic and see which one gets more traction. 
  • Positive, neutral, or provocative: By incorporating positivity in your copy, you are engaging with your reader’s brain in a powerful way that would result in a better understanding of your message. Your readers are more likely to convert if you engage them and motivate them through a robust and well-written copy.

7. CTA and buttons:

Your call to actions can be tested against copy and color.

  • Copy: By testing generic CTA’s such as “Buy more” against specific CTAs like “Get your custom denims here,” you might be able to understand which works wonders for your brand. 
  • Color: Your CTA button must complement the rest of your email body and stands out. For example, a CTA button in green on a white background is very impactful.

A/B Testing – Best Practices

Your email marketing solution should offer you A/B testing functionality. Still, even if it doesn’t, you can create your testing campaigns by manually segmenting lists and creating separate campaigns for each. 

Review the following best practices to improve your email A/B testing campaigns:

Tip #1: Start by Formulating  a Hypothesis 

If you make changes to an email, run an A/B test, and find out that one variation performs better than another, that’s a start. But if you don’t know which elements to test, why do you feel the need to run a test, and what’s the goal you want to achieve, you can’t go ahead. 

So, start by formulating a hypothesis and define what you want to improve and why you think the changes will contribute positively to your desired outcome.

Tip #2: Test High-Impact Elements

While A/B testing, make sure you do not get overwhelmed and start testing every element. You must try and test elements that you feel can pose a high impact on your email open rate. Work primarily on your CTAs, headlines, visual hierarchy, etc.

You can, later on, shift to testing the timing of your email automation flows– the actions you use as triggers, or the way you segment your recipients as well to gauge their impact on the open rates.

Tip#3: Focus on Frequently Sent Emails First

Focus on the emails that you send frequently. For instance, your weekly newsletter emails. Pay much attention to the kind of templates you choose, the headlines you write, the type of content you wish to input, the placement of CTAs, etc. Testing and perfecting them is help you get more traction and leads.  

Tip #4: Test Broadcast, Segmented, Automated, and Transactional Messages 

Test your broadcast and segmented messages and ensure you’re extending both the practice of A/B testing and noting any learnings you’ve discovered, to the other types of emails you send to the sample size.

Tip #5: Consider the Potential Impact of Timing on Email Performance

No matter how excited you are to kick off a new email A/B testing program, be cautious if that means starting around a period of irregular seasonal or industry-specific activity. Reaching incorrect conclusions from abnormal spikes of activity won’t do your future testing any good.

Conclusion

From creating hypotheses to improve your email campaigns, writing engaging crafty email messages, A/B testing them to attain perfection, and to finally sending them across to your list.

You can start it off today!

Having said that, do not forget to weed out the unengaged subscribers from your list. Also at the same time, don’t end up being a spammer!

More from VWO

Content Marketer’s Guide to Conversion Rate Optimization

Your website is your number one tool for lead generation and likely your number one…

Read More
Peter Dulay

Peter Dulay

8 Min Read

Proof of how important website titles are for SEO

On popular forum Hacker News, there were recent discussions on how important website titles are…

Read More
Paras Chopra

Paras Chopra

2 Min Read

Six Ecommerce Pricing Strategies For Brave Hearts

Pricing is tricky. You can ask consumers how much they would pay for a product…

Read More
Smriti Chawla

Smriti Chawla

9 Min Read
>
Shanaz Khan from VWO

Hi, I am Shanaz from the VWO Research Desk.

Join our community of 10,000+ Marketing, Product & UX Folks today & never miss the latest from the world of experience optimization.

A value for this field is required.

Thank you!

Check your inbox for the confirmation mail