Extreme Email Experiment: How Much Is Too Much?

Email is like a marketing battery for search marketers: it stores marketing potential that can be tapped later to generate sales and awareness. The most effective way to tap this resource is through email.

If you’re investing in search and have not considered growing an email list from the traffic, you are probably missing a huge opportunity. When entrusted with an email address, and permission to continue the conversation, we have one, two, three or more chances to persuade a prospect to consider our offers.

When entrusted with an email address, and permission to continue the conversation, I have one, two, three or more chances to persuade a prospect to reconsider.

In a business-to-business situation — the considered purchase — in which a decision will be made over a period of weeks or months, email is a true friend. And if it is executed with respect, it is a friend to those struggling with a purchase decision.

The question is, how many second chances am I going to take?

Five Emails An Hour?

The question is: can you send a daily email to a business-to-business list?

I tell companies that they can send email as often as their content allows.

I once got five emails from American Airlines within the space of an hour. Did I unsubscribe? Did I feel spammed? No, the emails were telling me the status of a flight I was booked on as its departure time and gate changed. The emails were completely relevant to my situation, and were welcome.

If I were to stand by my statement that businesses can send as often as their emails’ relevance allows, I needed to understand the dynamics of a high-frequency email campaign.

An Email A Day Experiment

In September of 2010, we undertook an experiment to determine what would happen if we suddenly turned up the frequency of our emails in a business-to-business situation.

The goal of this experiment was to examine four hypotheses:

1. Sending email would outperform social media marketing.
2. Sending frequent email would significantly increase my conversion rate.
3. Sending frequently would cause an unacceptable number of my subscribers to unsubscribe.
4. Sending frequent email would reduce my ability to deliver email due to spam reports.

The first hypothesis is there to test on of my frequent recommendation: “Get your email channel rocking before you invest in social media.”

The List

We chose a selection of 2000 names from our house list. This list consists of contacts made through personal interactions, meetings and consultations. It is a business-to-business list.

I would call the list a “semi-warm” list having received email from me only quarterly. This list had received emails on January 11 and April 30. The experiment began September 7.

Your list could easily be generated from social media traffic or search engine traffic.

The Content

Because of the frequent nature of these emails, it was important that the email content provide some value and be entertaining. This proved to be a significant challenge, as Conversion Scientists are not known for their wit.

Each email followed the following formula:

• A non-promotional subject line
• “Light” educational copy
• A link to relevant online content
• At least one offer

Subject lines were non-promotional.

“Are you the victim of the Email Invisibility Ray?”

“Social Media: Marketing from my La-Z-Boy”

“Why eight-year-olds beat me at Chess“

Offers varied, including an invitation to subscribe to my mailing list, registration for a live workshop and an invitation to a Webinar on writing for landing pages.

We linked to content on our blog, at Search Engine Land, and at the Content Marketing Institute.

The complete contents of the emails sent can be downloaded at The Conversion Scientist™.

The Frequency

Emails were sent daily, Tuesday through Friday for two consecutive weeks. Eight emails were sent over twelve days.

Test Results

Email Performance vs. Social Media

We’ve had relatively good luck using social media to drive traffic to The Conversion Scientist blog. However, in Figure 1, you can see that the email resulted in significant increases in traffic, even outperforming our summer social media experiment.

Figure 1: Emails’ Effect on Site Traffic

Notice the rise in search engine traffic at the time of the email. This underscores that click-through rate is only a partial measurement of email effectiveness.

Hypothesis: “Sending email would outperform social media marketing.” Outcome: True.

Increased Conversion Rate

It is probably not surprising that sending email to a targeted list is going to result in more conversions. However, keep in mind that my social media networks are also quite well-targeted.

As expected, both conversions and conversion rates for new subscribers increased.

Figure 2: Emails’ Effect on Conversion Rate

Just looking at new email subscribers, the conversion rate for our social media experiment were 2.5%. For the period of this email, new subscriber conversion rates were 7.6%, a 145% increase.

We can also attribute thirteen (13) workshop registrations to this email series, generating almost $1300 in sales.

Hypothesis: “Sending frequent email would significantly increase my conversion rate.” Outcome: True.

Opt-out Rates

This was the metric I was most interested in examining. How would unsubscribe rates change over the course of the experiment?

Figure 3: Unsubscribe Rates for the Email Series

I consider an unsubscribe rate of 1% or less acceptable and expected in any email that asks the reader to take action. So, I got pretty nervous as unsubscribe rates rose to 3.2%, and stayed well above 1%. Over the course of the experiment, 15% of the list unsubscribed.

There are two ways to look at this:

1. We lost 15% of our prospects.
2. We identified the 85% of list members that are interested and qualified.

If our goal with this list was primarily to sell, we would consider the 15% loss to be acceptable and even desirable. The process of weeding out unlikely buyers is called shaping a list.

Our goal, however, is to evangelize conversion and to educate. So, the opt-outs represent a pretty significant loss of reach.

From a brand perspective, there were very few negative comments about the emails, and many positive ones.

In comparing our opt-out rates against the increase in conversions, would we do this again? The answer is a resounding yes.

Hypothesis: “Sending frequently would cause an unacceptable number of my subscribers to unsubscribe.” Outcome: False.

The Effect On Deliverability

A negative effect that frequent emails can create is an increase in spam reports. Changes in deliverability of email is the primary measure of increases in spam reports.

For most email service providers, deliverability is the inverse of the bounce rate. If my emails are reported as spam, we would see an increase in bounces. Intuitively, when shaping a list, you expect bounce rates to drop quickly as bouncing addresses are removed from the list.

We saw this in our test, as you can see from the blue line in Figure 3. The bounce rate began at 2.5% but quickly dropped, leveling at an imperceptible 0.06%.

One reader was kind enough to let me know that they had “spammed” my email.

If we had generated a large number of spam reports, our Email Service Provider (ESP) would have let us know, probably by cutting us off. It is a big value of an ESP in that they work with ISPs keep themselves – and you – off of spammer lists.

Figure 4: Open rate, Click-through rate and Bounce Rate for each drop.

Another measure of reader rejection is open rate.

Email service providers place a special image in each email. They can tell when that image is downloaded by a reader to establish open rates. However, many people have images turned off in their email client, and the image doesn’t get downloaded. Thus, the open rate is not an accurate measure of actual opens.

I would interpret a steady drop in open rates as a sign that the list is becoming fatigued with my communications. Open rate can also be a good indicator of the quality of your subject lines.

For our test, open rates were relatively flat, dropping on Fridays.

Overall, I believe that few of my readers reported these emails as spam. I attribute this positive outcome to the non-promotional nature of the copy, even though the emails were clearly promoting our email list, workshop and webinar.

Hypothesis: “Sending frequent email would reduce my ability to deliver email due to spam reports.” Outcome: False.

Conclusions

With some simple analytics in place, you can pretty easily establish the ideal frequency of your email campaigns.

Based on the results from our experiment, we confirmed that we should be sending email more frequently. You will probably come to a similar conclusion.

However, we tested a certain kind of email with this experiment; an email that is informational and entertaining as well as promotional. This style of email requires a bit more work and creativity on the part of the publishing company.

The payoff is quite clear.

Today, email is a more effective channel in a B2B sale than is social media. It is also a great way to get more out of your search engine and advertising traffic. When you get an email address, you get a second chance at the sale. And a third, fourth and fifth chance.

If you’re interested, you can see the complete content of the emails sent during this experiment.