Create an A/B Test
See which email performs best with Mailchimp's A/B testing.
Get the job done with a pro
From training to full-service marketing, our community of partners can help you make things happen.
QuickBooks makes managing your finances easy and integrates with Mailchimp, allowing you to use your paid invoices to create smarter, more personalized marketing campaigns.
A/B tests compare different versions of an email or SMS to see how small changes can have an impact on your results. Choose what you want to test, like the subject line or content, and compare results to find out what works and what doesn't work for your audience.
In this article, you'll learn about A/B tests.
Here are some terms you’ll come across when you use A/B tests in Mailchimp.
Variable
The element of your test that you want to compare. With an email A/B test, you can test 1 of 4 variables: subject line, From name, content, or send time. Content is available to test with an SMS. Each version of the variable is called a variation.
Combination
Each variation of your test that is created from the variables you choose. If you want to test 3 different From names, we'll create 3 different combinations of your email. Combinations sent in the test phase are called test combinations.
Test phase
The period of time after the combinations are sent out and we compare the results. Data collected during the test phase can be used to determine the winning combination automatically or manually.
Winner or winning combination
The combination that performs the best. This may be automatically determined by click rate, open rate, or total revenue, or manually chosen based on the reporting data you find the most valuable.
When setting up an email A/B test, you'll choose a single variable type—subject line, From name, content, or send time—and create up to 3 variations. With an SMS, content is the variable that’ll be tested. We'll generate all possible combinations and send them to different sets of subscribed contacts, so no one receives more than 1 combination of your test.
The combination that your subscribed contacts receive are chosen at random and tracked solely for the purpose of choosing a winner. This means you won't be able to see which combination a subscribed contact receives.
Send the combinations to all your subscribed contacts at once if you have a small audience or segment, or if you're testing the send time of your email. With other variables or a large audience segment, or tag, send your test combinations to a percentage of your subscribed contacts. Then send the winning combination to your remaining subscribed contacts.
To choose the winner, use 1 of these options.
Here’s some more information on the variables you can test along with some ideas to help you get started.
Content
Create different versions of your content to see what gets a better response. Use this variable to test content changes or different templates.
When testing content in your email, you may want to learn how successful your links, or buttons are. Use our link comparison tool in the email report to see how your links performed in each combination.
Here are some common ways to use A/B tests to improve your marketing.
Technical Support
Have a question?
Paid users can log in to access email and chat support.
See which email performs best with Mailchimp's A/B testing.
Need help getting started with Mailchimp campaigns? This article walks you through the basics.