Last updated: Jul 26, 2017 admin The key to A/B split testing in email is patience. Wait as long as you can to analyze results and you’re more likely to get a true picture of what will happen when you roll out your eventual email campaign.Does your ESP provide split testing?Marketers and ESP sales teams often sell email deployment tools with built-in A/B split testing software on the basis that they will make your programs run with the minimum amount of resource.The A/B split testing functionality varies by platform, from running the entire split test and deploying the “winning” campaign automatically, to simply splitting the data into 2 random sets.As with most tools, A/B split testing software tends to be generic. If you have a specific need or want to test something that is unique to your organization, then the manual approach is often the best method, and it’s not half as difficult as you might think…How do I go about manual A/B split testing?To ensure that your findings are robust and easy to implement, start with your test plan. Once you’ve decided what you’re going to test, there are 3 main areas you need to consider: data, set up and analysis.How do I split my data?To test the impact of one variable on another, a random selection is the best choice for most marketers – essentially splitting your list in half.Only have a small amount of data? Plan on breaking the analysis down into smaller groups, such as demographic groups. Make sure that you have equal and sufficient representation in each group.Alternatively, if you believe that one approach is likely to deliver significantly higher returns, you may choose to only keep a small sample for one variable, and send the majority of addresses the alternate option.How big does a representative sample need to be?The answer depends on a number of factors:primarily, how statistically viable you need the results to beyour objectiveshow many factors you are testingThere are differing opinions on sample size, but it is smaller than you would think. The sample size used to predict the general election is based on just 2000 people, for instance, although over 27 million people actually vote.You can choose your sample size scientifically by using a sample size calculator. You can find one easily – just type “sample size calculators” into a search engine.How to use a sample size calculator:Say you wanted to test for how many people would open your email if you sent it to 10,000 people and you wanted the accuracy of your answer to be within 5% either way. That means your final answer is going to be an interval, like 25-35%.So how many people do you need to sample? Here’s how to find out:Find a sample size calculator online, like this from creative research systems or this from RaosoftChoose the size of the interval you want as your answer – in this example it’s 5%Type in the population – in this case it’s 10,000Choose how certain you want to be. You can never have total certainty, but most calculators let you choose 95% (pretty sure) or 99% (very sure, which means you’ll need to use a larger sample)Press calculate and the calculator will give you your sample size. For this example you can be pretty sure by sampling 624 people.So if 10% of your sample open your email, you can be pretty sure that you’ll get a 5-15% open rate when you send it out to 10,000 people.How do I set up my mailings?Once you’ve split your data, all you need to do is set up your mailings.If you plan on delving deeper into your results to profile different segments, the best way may be to set the messages up as one campaign and analyze the data offline after the campaign has finished.Though this approach requires some additional data work initially, it will keep your campaigns organized within your platform and ensure your future regular campaign reporting remains consistent.You could opt to:test different subject lines, content or creative by using dynamic contenttest a variety of send days by setting up a triggered campaign, based on a field you add to your datadetermine the most effective deployment time, if your list is large enough, by implementing your campaign very slowlyExtra tip: track how you split your campaign and make sure that you can identify which subscribers received which test.How do I analyze the results?It can’t be stressed enough: you must let your campaign run its course. We’d advise any results gathered in less than 2 days will probably turn out to be inaccurate as the testing continues. Depending on your purchasing cycle, you may need to wait as long as a month. Then, gather as much information to include as you can – this can be anything that you can attribute to an email address, such as revenue, or even web stats.To determine the impact of each of your tests, you can then layer this information over your sent, delivered, opened and clicked activities for each email address.Based on your hypothesis, you will then be able to easily see which campaign met your objectives best. Look at all possibilities. By splitting the data in different ways (which is easier to do offline), you may uncover some surprising findings that could guide future testing or campaign development.