How to A/B Test Your Direct Mail Campaigns

Yes, A/B testing is just as important in direct mail as it is online! Here’s how to A/B test your direct mail campaigns to make sure you know what works and what doesn’t. 

Testing is critical. Whether you’re creating a new product or cooking a meal, you need to test it out to understand whether or not it’s working! 

If you aren’t testing, you may be missing out on valuable information that could supercharge your campaigns and bring in more business. 

An A/B test is a direct comparison between two things. In your case, two pieces of mail. Its purpose? To compare two designs, CTAs, tones of voice, etc. to find out which one gets a better response from your customers. 

Here’s how to A/B test your direct mail campaigns easily. 

How to A/B Test Your Direct Mail 

Planning your first A/B test? Take these steps to make it successful! 

Set A Measurable Goal 

What’s the point of your A/B test? What are you measuring? Knowing this is your first step to running a successful test! Maybe you want to measure: 

• Number of sales 
• Website visits 
• Responses to a prompt 

Having a specific goal helps you to know what metrics to focus on when you’re analysing results later. Which leads to our next point… 

Test One Metric At a Time 

You can test all 3 of those metrics, but each one should have its own campaign. One A/B test, one variable. This will prevent confusion and help you to really nail down the specifics of what you’re testing for. 

Choose Your Batch Size 

You’ll get less information out of a test group of 1,000 than you will out of a group of 10,000. Of course, if you’re running your normal campaign and simply using it for testing purposes, you’re likely to just be sending out to the general area and not a smaller portion of the population. 

However, if you specifically want to (and have the capacity to) do a smaller campaign purely for testing purposes, it may be worthwhile to go for a slightly larger testing group. 

Track Your Results 

Consider how you’re going to track your desired results. If you’re tracking clicks or sales, all of that information is likely to be found on your analytics software. Make sure you’re tracking the right thing as per your goal – if you’re aiming to track conversion rate, for example, don’t waste time staring at your website visits. 

Direct Mail Testing Ideas 

Of course, the idea is to compare two sets of results, so a full A/B test contains two campaigns. There are two easy ways to do this: 

Single Variable Split-Test 

Let’s say you’re sending your campaign to 10,000 people. Create 5,000 pieces of mail with one element—eg. “Buy Now” CTA—and 5,000 with another—eg. “Give Me The Good Stuff”. Only change one small thing, not the entire leaflet! 

Send the first 5k to one split of your group and the second 5k to the second split. From there, keep an eye on your results to see which one is the favourite. 

This is a full A/B test within a single direct mailing campaign. The next type uses 2 separate mailing campaigns to test. 

Old/New Campaign A/B Test 

In this test, you don’t need to send out two lots of mail with a small change. You just need to grab the data from an old campaign and create a new campaign to challenge it. 

Change one small item from the original campaign and send it out to your full audience. Compare the results to those from the original. Easy peasy!