The Subject Line Researcher Project

The nerds in MailChimp Labs recently unveiled a new experiment called the Subject Line Researcher. It's kind of like Google's Keyword Tool, but for email marketing. And it's free for all MailChimp users.

  1. To test your subject line go to the Setup step of the Campaign Builder.
  2. Click the How do I write a good subject line? link under the Email subject field.

    Subject line researcher link

  3. In the How do I write a good subject line? pop-up modal, click the option for Subject line researcher.

    Subject line researcher

  4. Type some terms into the subject line terms field on the Subject Line Researcher popup window and press the Search button to populate results. For the best results, enter one term or phrase per line and use no more than 3 terms per phrase.

    Subject line researcher terms

  5. When you click the search button MailChimp will compare those terms to all subject lines ever used in our system, displaying a 5 star rating system for how the subject lines containing those terms performed.
    Subject line researcher results
    It's better to enter just a couple of keywords, to keep the search broad, and see subject lines that may have incorporated those terms in different ways than expected.

MailChimp will tell you how those words have performed in the past. As you can see from the screenshot, you might want to check out “Holiday Gift Guide” instead of say Holiday Gift Ideas, because it performed better. Or, maybe you should consider using our patent-pending A/B split testing tool to experiment with two subject lines on the same campaign.

Notice we don’t give you exact numbers on open rates. We give you nice, pretty little stars. We’re not going to open a can of worms by promising that the word “Free” will give you +40% open rates. The idea here is to simply give you more — well, ideas.

Now keep in mind, this is only our first stab at predictive reporting. It took months to generate this data, using our in-house supercomputer, some genetic pairing science, and Amazon EC2. Actually, we were doing something else entirely with our experiment, but this happened to be a cool byproduct, so we added it to our app.

Point is, we’re far from complete, and it’s far from perfect, but this’ll get better as more people use it and send us feedback.

Email Marketing Subject Line Comparison

Study of Best and Worst Open Rates on MailChimp

People who are new to email marketing often ask us, "How should I write my subject lines so that more recipients will open my emails?" In order to answer that question, we recently analyzed over 40 million emails sent from customers through MailChimp, and found the ones with the highest open rates and the ones with the lowest open rates. Then we pulled 20 from each pile and put their subject lines in a side-by-side comparison. The "highest" open rates were in the range of 60%-87%, while the "lowest" performers fell in the dismal 1%-14% range. Do you see a pattern below?

Best Open Rates (60%-87%) Worst Open Rates (1%-14%)
  1. *|LIST:COMPANY|* Sales & Marketing Newsletter
  2. Eye on the *|LIST:COMPANY|* Update (Oct 31 - Nov 4)
  3. *|LIST:COMPANY|* Staff Shirts & Photos
  4. *|LIST:COMPANY|* May 2005 News Bulletin!
  5. *|LIST:COMPANY|* Newsletter - February 2006
  6. *|LIST:COMPANY|* Newsletter - January 2006 [ *|FNAME|* *|LNAME|* ]
  7. *|LIST:COMPANY|* and Other Company Name Invites You!
  8. Happy Holidays from *|LIST:COMPANY|*
  9. ATTENTION *|LIST:COMPANY|* Staff!
  10. ATTENTION *|LIST:COMPANY|* West Staff!!
  11. Invitation from *|LIST:COMPANY|*
  12. *|LIST:COMPANY|* Jan/Feb 2006 Newsletter
  13. Website news - Issue 3
  14. Upcoming Events at *|LIST:COMPANY|*
  15. *|LIST:COMPANY|* Councils: Letter of Interest
  16. *|LIST:COMPANY|* Coffee Exchange - Post-Katrina Update
  17. We're Throwing a Party
  18. October 2005 Newsletter
  19. *|LIST:COMPANY|*: 02.10.06
  20. *|LIST:COMPANY|* Racing Newsletter
  1. Last Minute Gift - We Have The Answer
  2. Valentines - Shop Early & Save 10%
  3. Give a Gift Certificate this Holiday
  4. Valentine's Day Salon and Spa Specials!
  5. Gift Certificates - Easy & Elegant Giving - Let Them Choose
  6. Need More Advertising Value From Your Marketing Partner?
  7. *|LIST:COMPANY|* Pioneers in Banana Technology
  8. *|LIST:COMPANY|* Moves You Home for the Holidays
  9. Renewal
  10. Technology Company Works with *|LIST:COMPANY|* on Bananas Efforts
  11. *|LIST:COMPANY|* Update - A Summary of Security and Emergency Preparedness News
  12. Now Offering Banana Services!
  13. It's still summer in Tahoe!
  14. *|LIST:COMPANY|* endorses Successor Company Name as successor
  15. *|LIST:COMPANY|* Holiday Sales Event
  16. The Future of International Trade
  17. *|LIST:COMPANY|* for your next dream home.
  18. True automation of your Banana Research
  19. *|LIST:COMPANY|* Resort - Spring into May Savings
  20. You Asked For More...
* Study only included campaigns sent to at least 100 recipients.

Observations

On the "best" side, you'll notice the subject lines are pretty straightforward. They're not very "salesy" or "pushy" at all. Heck, some people might even say they're "boring." On the "worst" side however, notice how the subject lines read like headlines from advertisements you'd see in the Sunday paper. They might look more "creative," but their open rates are horrible. It's as if those email marketers assumed that subject lines have to jump off the screen and "GRAB THE READER'S ATTENTION!" or something. Unfortunately, most people get so much junk mail in their inbox, anything that even hints of spam gets thrown away immediately.

Setting Expectations

So does that mean your subject lines should be really stale and un-creative to get high open rates? We don't think so. In our study, we actually saw some campaigns that used more "creative" subject lines (like the ones on the "bad" side of our table) but they had pretty decent open rates. The difference seemed to be in the expectations that were set for the emails.

For example, email "newsletters" are for "soft-selling." They build relationships with your customers, and they're great if your products have a very long sales cycle. Use them to slowly soften your customers for the sale, or to make them feel really good about your brand. If your recipients signed up for these kinds of emails, don't expect them to be very enthusiastic when, out of the blue, you send an email with a subject line like, "10% Discount! Open Now!" For newsletters, keep your subject lines simple, straightforward, and consistent.

On the other hand, if your subscribers specifically opted in to receive "special offers and promotions" from your company, there's nothing wrong with saying there's a "10% off e-coupon inside." They'll be expecting a "hard sell" from you. It's when marketers send promotional emails to their entire "newsletter" list when things go wrong. The idea is to create a totally separate opt-in list for those who want to receive promotional emails. Furthermore, segment your promotions list into smaller, more focused groups before you send your campaign (don't send an offer for purses and high-heel shoes to the men on your list).

The Secret Formula for Subject-Lines

So what's our advice for email subject lines? This is going to sound "stupid simple" to a lot of people, but here goes: Your subject line should (drum roll please): Describe the subject of your email. Yep, that's it.

Always set your subscribers' expectations during the opt-in process about what kinds of emails they'll be receiving. Don't confuse newsletters with promotions. If your email is a newsletter, put the name and issue of the newsletter in your subject line. Because that's what's inside. If your email is a special promotion, tell them what's inside. Either way, just don't write your subject lines like advertisements.

When it comes to email marketing, the best subject lines tell what's inside, and the worst subject lines sell what's inside.


Related Articles