A/B split testing works. You’d be hard pushed to find any email marketer that says it doesn’t. But to what degree it works and the best method to use are hot topics of debate.
What we do know is that split testing delivers a lift. A 13.2% lift on average according to our latest survey of email and brand marketers on their subject line split testing.
Here are the key learnings:
1) Although most marketers regard subject lines as the most important element affecting response, 74% of them spend less than an hour on their subject lines
How often do you A/B split test your subject lines?
2) Despite seeing an average lift in response of 13.2% from A/B split testing, 61% of marketers do not test subject lines on every campaign they send out
On average what sort of improvement do you see from your subject line A/B test?
How often do you A/B split test your subject lines?
3) The number 1 reason for not testing with every send is “it takes too long”
4) Only 35% of marketers test more than 3 subject lines when they do test
5) 48.2% of marketers regard open rate as the most accurate performance measure when A/B split testing but 85.7% think click rate most directly correlates to revenue
In truth, there is nothing particularly new in these survey findings. By and large email marketing is under-resourced so execution tends to take priority over optimization.
But therein lies the dilemma: how do you measure the cost of something you’re not doing so you can make the case for doing it?
So we’ve put together some simple cost calculation benchmarks to give you an idea of how much you might be losing by not split testing all the time. How much is not split testing costing you?
This is something each marketer or brand can (and should) calculate for their own email program and clients using their own numbers for accuracy. But we’ve produced a ballpark lookup table (below) so you can get an idea of what you may be leaving on the table.
It’s based on a few assumptions: that you send an email to your whole list once a week and the average open lift you get from split testing is 13.2% (taken from our survey).
Although your own numbers will vary, it’s clear that split testing has an impact on the bottom line. Even with a very modest 10 cents per open, your program could be losing thousands of dollars in additional revenue if you aren’t split testing all the time. How much are losing subject lines costing you?
Using the same assumptions we can also have a look at how much leakage there is to losing subject lines if you live split test all the time.
Again, your own numbers will vary but leakage to losing subject lines (or the “testing tax” as we call it) is potentially eating into your success even if you do test all the time! Why not look into your own email program to see how much you might be losing to split testing.So is there a solution to these challenges? Yes. We call it Virtual Testing.
Virtual Testing, Real Results
As an email marketing agency that’s been around for 15 years, we’ve come across these challenges with split testing many times. Our answer was to develop a virtual testing platform for subject lines.
It works by creating a replica database that mimics the behavior of your customers. This allows you to send virtual tests in seconds and instantly compare your subject lines to see which will perform best to 100% of your list.
As it only takes seconds to send a test campaign it removes the “it takes too long” problem of split testing for every send. In fact, you could test 100s of different subject lines in the time it would take you to run a traditional split test!It also means you never have to send a losing subject line to your real customers. And as you are making a judgement based on how your entire list would behave rather than a small sample, the results are more accurate than traditional split testing with small samples.
Better still, you can upload click and revenue data and use these instead of open rate to determine which subject lines you should send. The tool even offers recommendations on which words to change in your subject lines to improve your performance.
So if you are interested in how you can effectively and efficiently split test your subject lines all the time then try our Virtual Testing Tool for free. *these calculations are based on the assumption that the current program has no split testing and sends 1 email a week to the entire list. The lift is an average taken from the survey of 13.2%† these calculations are based on the assumption that the current program split tests every campaign and sends 1 email a week to the entire list. The lift is an average taken from the survey of 13.2%.
Last updated: Feb 24, 2016 Alchemy Worx
