A/B Testing Campaigns allow you to test different versions of a single campaign to see how small changes can have a big impact on your results. Choose what you want to test, like the subject line or content, and compare results to find out what works and what doesn’t.
How A/B Testing Campaigns Work
Set Up the A/B Testing Campaign
You’ll choose a single variable type—Subject Line, From Name, Content, or Send Time—and create up to three variations. We’ll combine each variation with your other content to create combinations that we’ll send to different sets of subscribers, so no subscriber receives more than one combination of your campaign.
Choose Winner Criteria
Send the combinations to all your recipients at once if you have a small list, or if you’re testing Send Time. With other variables or a larger list, you can choose to send to a percentage of your recipients to test combination results, and then automatically or manually send the winning combination to your remaining recipients.
To choose the winner, use one of these options.
- Automatic: Open or Click Rate
Use these options if you want MailChimp to send the winning campaign to your remaining recipients after a set amount of time, based on the highest open or click rate.
- Manual: Report Statistics
Use this option if you want to choose the winner based on reporting data or other factors that you find to be the most valuable. For example, if your campaign combinations sell different products, you may want to send the combination that has the highest number of sales even if it has a lower open or click rate.
When we talk about A/B Testing Campaigns, we use some terminology that’s a little different from how we talk about other tools and tasks in MailChimp.
The element of your campaign that you want to test. With an A/B Testing Campaign, you can test one of four variables: subject line, from name, content, and send time. Each instance of the variable is called a variation.
A campaign created with a testing variable and your other content. To test three different From names, we'll create three different combinations of your campaign.
The period of time after the combinations are sent, during which we compare the results. The test phase provides data that will be used to determine the campaign’s winning combination.
The campaign that performs the best. This can be chosen automatically based on open or click rate, or you can manually choose the winner based on the reporting data you find the most valuable.
Variables You Can Test
Try different phrasing or sales offers to see what gets the most attention.
See if your subscribers are more responsive to emails coming from a person’s name or from the name of your company or organization. You’ll provide the From name and From email address you want to use for each combination.
Create different versions of your content to see what gets a better response. Use this variable to test small content changes or completely different templates.
After your A/B Testing Campaign is sent, use the campaign report to compare the performance of specific links. For example, if you have the same link in two different campaign combinations, the campaign report will tell you in which combination that link performed the best.
Learn when your subscribers are most likely to open your campaigns. This option tests specific days and times, so you won't be able to send a winning campaign to the remainder of your subscribers. Use this data to inform when to send or schedule future campaigns.
A/B Testing Campaign Ideas
Here are some common ways MailChimp users learn from A/B Testing Campaigns.
- What day of the week gets better open rates?
- Does a subject line with an incentive or a teaser work best?
- Does including your company name in your subject line increase engagement?
- Is it better to use your name as the From name, or your company’s name?
- Does the time of day a campaign is sent affect the click rate?
- Are subscribers more likely to click a linked image or linked text?
- Do subscribers prefer a campaign that contains a GIF or one with static images?