ActiveCampaign: A Guide to Split-Testing Your Email Automations (with 10 Practical Examples)

Are you using ActiveCampaign and looking to take your email marketing to the next level in 2019?

(The answer should be a resounding YES!)

Look, if there’s one thing most online businesses still overlook when it comes to email, it’s leveraging and optimizing automation for massive growth.

Whether you’re new to email automation or just trying to squeeze more value from your existing workflows, this guide is for you.

We’re going to go in-depth into split testing your market automations with ActiveCampaign and here’s what we will be covering in this article:

Click any link below to jump to the relevant section:


Email Automation And Why You Should Care (Hint: It Makes You More $$$)

Email automation is a form of email marketing that uses a trigger-based system to send an email (or series of emails) on a case-by-case basis.

So, instead of the traditional blanket approach where you send everyone the same email at the same time, email automation allows you to automatically send different emails at different times.

In 2019, email automations are everywhere. And most of the best email services include some form of automation capability now.

You get an automated email when you sign up to a newsletter, or create an account, or buy a product, or contact support, or just about anything else you can do online.

The reason? I’ll give you three…

  1. It saves time: Automations trigger the sending of emails without manual input, saving you a silly number of hours.
  2. It’s targeted: You can send just the right kind of email to the right person at just the right time, increasing user engagement.
  3. It’s scalable: Simply drop leads into your automation funnels and the rest will be taken care of by your email marketing software.

The result is a highly efficient, fully automated system that is capable of increasing email open rates by 14.32%, and click-through rates by a whopping 100.95%. (Yep, that’s double!)

With a reported average return of $44 for every $1 spent, you simply cannot afford to overlook email automation as a way to drive up traffic, engagement and sales in your business.

But the REAL Magic Happens When You Start Split-Testing Email Automations

Split-testing has long been a popular optimization tool for email marketers. (Even back in the day when everyone used AWeber*shiver*)

Whether it’s for different design elements, time-frames, or offers, running creative split-tests can help you discover winning formulas that lead to a significant boost in engagement, sales and profit in your business.

This is true in almost every area of digital marketing, but these benefits can be huge when it comes to email marketing — and email automation in particular.

Even if you already have successful automations doing the heavy lifting, there’s ALWAYS an opportunity to squeeze more opens, more clicks, and most importantly, more sales from the same traffic.

Here’s the good news:

ActiveCampaign allows you to create a testing environment for even the most complex email automation workflows— a task that was once considered a technical nightmare in the industry.

In fact, this is a feature that almost no other email marketing platform has managed to pull off to date, including the likes of MailChimp, ConvertKit, and GetResponse.

So how do you actually pull off this digital miracle?

How to Split-Test Automations with ActiveCampaign (Step-by-Step)

activecampaign - home

ActiveCampaign is easily one of the most powerful email automation platforms on the market right now, and you better believe they offer some awesome split-testing features to get the most out of it.

Here’s how the process breaks down:

  1. Come up with a hypothesis
  2. Specify your test conditions
  3. Create the competing workflows
  4. Measure the results

Let’s go through each phase of this process using ActiveCampaign to set up a split-test campaign.

1. Come Up with a Hypothesis

Before you can run any kind of test, you need to know what it is you’re trying to achieve, and you do that by coming up with a split-test hypothesis.

First, you’ll need to go beyond the idea of getting “more leads” or “more sales” and start thinking about a specific and measurable objective.

For example, saying that you want a “higher open rate” on your emails is fine and dandy, but it’s hard to test and measure such a broad objective.

Instead, a good objective relies on a specific metric:

  • Get a higher open rate on your welcome email
  • Get a higher click-through rate on your sales email
  • Get a lower unsubscribe rate on your newsletter

Once you have a clear objective, you can begin to formulate a hypothesis that will aim to improve that metric.

So, taking the previous examples, I might say:

  • Objective: Get a higher open rate on your welcome email
  • Hypothesis: A question will perform better than a statement in the against in the subject line
  • Objective: Get a higher click-through rate on your sales email
  • Hypothesis: Multiple links will generate more clicks than a single link to my sales page
  • Objective: Get a lower unsubscribe rate on your newsletter
  • Hypothesis: A bi-weekly newsletter will decrease the unsubscribe rate for each newsletter

A hypothesis is not about being right, it’s about making an educated prediction on which factors will help you get to your objective. You can only know what really works by testing.

Of course, in order to run an effective split-test, you absolutely need to have both of these elements clear in your head before you begin creating your automations.

2. Specify Your Test Conditions

Now that you have an objective in mind, you’ll need to define your test conditions.

To do this, you’ll first want to create a new Automation Workflow within ActiveCampaign:

activecampaign - new automation

ActiveCampaign will then give you a bunch of “recipes” to choose from, which are essentially pre-built automations.

I prefer to build from scratch:

activecampaign - from scratch

From here, you’ll be prompted to select a trigger for your automation, and your trigger will come before your split-test conditions.

This will of course depend on the nature of your test, so you’ll need to figure out which trigger makes sense depending on your hypothesis.

activecampaign - actions

Finally, with a trigger selected, you’ll now enter ActiveCampaign’s famous Workflow builder where you can start to add conditions and actions.

Following the trigger, you’ll need to add a “Split” element in order to run a split-test.

activecampaign - split

ActiveCampaign will ask if you want to do an even split, or a conditional split, and this is an important step.

Here’s what they mean:

  • Even split: Alternates subscribers through each possible automation, keeping them equally balanced and ensuring there is an even divide of incoming contacts at all times.
  • Conditional split: Sends a specified number of subscribers through the first automation until a condition is met, before sending the same number of subscribers through the second automation and so on.

So… which one should you use?

Well, conditional splits tend to be used for things like special offers, and more important when you want to limit that offer to first X number of people who sign up to your list.

Since that’s not what we’re doing here, and also considering that this kind of behaviour can skew results by creating a significant time-gap between individual tests, you will almost always want an even split for running split-tests.

When selecting an “even split”, you’ll be asked how determine how long the test should run, whether that’s by a number of subscribers, or a point in time, or… never.

activecampaign - split indefinitely

It’s no secret that running a split-test using a small sample size can often produce misleading results, meaning you should aim for a larger sample size to produce much better results.

So the next logical question is, how many combined subscribers should you run through your automations before selecting a winner?

The answer to this question is fairly complex and requires a thorough explanation of statistical significance that is outside the scope of this article. (Fortunately, there are plenty of resources available, such as this one.)

3. Create the Competing Workflows

By now, you’ll have your objective and hypothesis all figured out, as well as your automation trigger and your split condition in place.

The result is something like this:

activecampaign - automation workflow

As you can see, the trigger which in this case is “on subscribe”, will cause both the A and B automation sequences to run while splitting those subscribers evenly until the automation is stopped.

From here, based on your hypothesis, you’ll need to build out two separate automation workflows for each branch so that they can compete against one another.

Sticking with a previous example, let’s build on the following hypothesis for increasing open-rates on the welcome email: “A question will perform better than a statement in the subject line

So, starting with branch A, the plus icon opens up the automation actions where you can find the “Send Email” action:

activecampaign - send email

ActiveCampaign will then take you through the usual email setup process, much like if you were creating a single email campaign.

It’s also important to enable the correct tracking options on the final screen when creating emails for split-tests, else you won’t be able to measure the results of the test later on.

In this case, we want to make sure open-tracking is enabled since that’s the focus of our test:

activecampaign - sending options

Going through this process twice, I created two identical emails with only the subject line being changed.

Here’s what each branch looked like afterwards:

activecampaign - two paths

This automation will now equally distribute my welcome email to new subscribers, with half of them getting a question-based subject line, and the other half getting a statement-based subject line.

Now, simply enable the automation and that’s it, you have a running split-test using ActiveCampaign.

4. Measure the Results

Assuming you’ve allowed enough people to go through your automation in order to properly test your hypothesis, it’s time to measure the results.

ActiveCampaign has a ton of reporting options, but what we’re interested in here is are the ‘Automation Reports’.

Here’s what my split-test looks like under this report:

activecampaign - reporting

Now, obviously this is an example for this review and you would (hopefully) have a lot more than 3 people go through your test.

That aside, you can see that my hypothesis was correct and the question-based subject line is winning, as it has a higher open-rate than the statement-based subject line.

Since this is an indefinite split-test, however, it will run until you decide a clear winner has emerged from testing both variations. Ideally, you would want to see a significantly higher open rate from one variation.

Once you’re happy with your test, you can eliminate the losing branch or simply replace it based on another hypothesis.

activecampaign - keep path

See, that wasn’t so hard was it?

10 Email Automation Split-Test Ideas You Should Try

Now that you know how to identify, create and measure an effective split-test using ActiveCampaign, let’s dive into some practical angle you can use to start experimenting with right away.

Ok, deep breath…. 3…2…1…. GO!

#1: Short Vs Long Emails

This test involves pitching a long-form email against a short-form email to see which one generates more engagement.

The key here is to keep the content as similar as possible while only reducing the length. That means writing on the same topic with same style and tone as well as delivering the same overall value.

#2: Email Structure

Most people tend to write their marketing emails off-the-cuff, meaning there isn’t any method to their madness.

While this approach can work, some have seen better success by leveraging a proven structure, otherwise known as an email copywriting formula. A common formula used is P.A.S, but this is one of many that are well worth testing.

#3: Plain Vs HTML

Plain text emails are exactly that, plain. Unlike emails written in HTML, they don’t include any fancy formatting, images or colors that might compromise deliverability.

ActiveCampaign isn’t able to provide direct deliverability metrics as this is determined on the receiving end, but you could use other, closely related metrics along with a tool like GlockApps to get more insight.

#4: Short Vs Long Sequences

While most test ideas tend to be on the email-level, there’s a lot to be said about running tests on an entire email sequence as well.

Some would argue that you can have too many touchpoints in your funnel, which means, for example, testing a 3-day sales sequence against a 7-day sales sequence could produce surprising results.

#5: Sequence Order

Following on from the last split-test idea, the order of emails in a sequence can play a huge part on the overall performance of it.

For example, many people claim to have figured out the perfect order for a sales sequence, though you’ll always find some discrepancy when comparing these articles from one blog to the next.

Testing this could be as simple as switching the timing of your ‘FAQ’ email with your ‘Testimonials’ email.

#6: Coupon Percentage Vs Dollar Amount

Nathan Decker of Evo.com found that offering a $50 coupon generated 170% more revenue than the 15% off coupon of the same value.

This one will require some configuring in your shopping cart as well as your automation, but with numbers like that it could be well worth the hassle of setting it up.

The interesting part about this test is that, in essence, nothing changes apart from the perception of the amount itself — an excellent demonstration of the quirks of human psychology.

#7: Subject Lines

We looked at a subject line test earlier in this guide, but there are various different tests you can run here aside from ‘question vs statement’.

For example, you can try:

  • Short vs long
  • Serious vs funny
  • Uppercase vs lowercase
  • Clear vs vague
  • Numbers vs no numbers
  • Brackets vs no brackets
  • Emoji vs no emoji

#8: Text vs Video

According to Animoto, “4 times as many consumers would rather watch a video about a product than read about it.”

So, instead of sending everyone a 500-word email about your product or service, try creating a split-test variation that sends half of your subscribers to a video with the same information.

#9: Call to Action

Your call to action (or CTA) is perhaps the most important part of any email, since that’s what drives people to visit your site, share your content, buy your product, or countless other actions.

Again, we mentioned an example earlier in regards to testing the number of call to actions throughout the email, but there are a number of other tests you can run.

For example, you can try:

  • Putting your CTA at the start vs at the end
  • Formatting your CTA (bold, italic, underline)
  • Uppercase vs lowercase CTA
  • CTA destination (Example: sales page vs checkout page)

#10: Email Personalization

Personalization within automated emails has been used almost since the beginning of email marketing, but since everyone knows it’s automated, how effective is it in 2018?

Using custom fields in on your opt-in forms allows you to capture all kinds of information that can be used later on in both your subject lines and email copy, and this opens the doors for some interesting tests.

For example, you can try:

  • Using a generic opener (no name)
  • Using their first name
  • Using their full name
  • Mentioning their company
  • Mentioning their interests
  • Mentioning their age

Final Thoughts

Split testing your automations is a great technique to squeeze more ROI out of your email marketing workflows and I hope this tutorial gives you a better idea on how to do it.

To start though, you need to make sure you’re using an email service that supports it.

Which is either ActiveCampaign or ONTRAPORT.

I recommend ActiveCampaign myself in most cases because it’s the service I use, and I’ve found it to be the best fit for most businesses.

They have a full 14 day free trial to test the service out.  And you don’t even need to pull out your credit card to do so.

Try ActiveCampaign Today

Affiliate Disclaimer: I hope you liked this Split testing email automation tutorial. Please note that some of the links inside this article may be affiliate links to ActiveCampaign. That means that if you click on one of the links and sign up, I may be compensated for it. If you do happen to click, I really appreciate it! Any money we make keeps this site running smoothly and allows us to keep writing these high-quality reviews.
About Lewis Parrott

Lewis is a writer and SEO nerd based out of Southeast Asia. He spends most of his day churning out internet marketing related content from his laptop. Believe it or not, he also has a girlfriend.

>