Split testing (A/B testing) is a professional email marketing tool used for comparing bulk email effectiveness.
Split tests enable you to compare the conversion rates for several different emails of the same recipient segment to define the most productive communication method with the target audience.
During the split testing, you compare at least two variants of the same email. Split testing can be applied to the following items of an email:
-
The email template, for example, the image location or the color of the call-to-action button (CTA). As a result of the test, you can define in what way the style and the form of the CTA button influence the number of the clicks.
-
The subject of the email. The subject is the first thing that the potential customer will see upon receiving the email. Define the wording of the email subject so that it stimulates the users to open the email.
-
The sender's name. You can test which sender's name has more credibility with the recipient.
Create emails for split testing
Prepare the email content that you want to be tested and add it to the new split test.
-
In the Email section, create emails for the test. You can prepare two or more variants of emails for A/B testing. Do not add the audience to the email as the recipients will be added and distributed upon testing. Also, do not start created marketing emails.
-
Open the list of split tests: Email → Actions → Go to split tests (Fig. 1).
-
Add a new split test.
-
At the first Bulk email wizard step, select which emails you want to be compared during the test. Make sure that each email:
-
has a “Planned” status;
-
is not included in another split test;
-
does not contain any list of recipients on the Audience detail.
-
Add audience
Go to the second wizard Audience step, and add the list of email recipients. There are several general recommendations for audiences used in split tests. Read more in “”.
-
Specify the audience percentage that will comprise the test group of the email recipients during the split test. For example, if you test two versions, you can specify 10% and add 1000 contacts on the Split test audience detail. As a result, the test audience of each version will contain 50 contacts. The email recipients are randomly selected from the audience. Also, you can test several emails on the entirety (100%) of a selected segment.
You cannot change the test group percentage after you added the audience. -
Add the required contact folders on the Split test audience detail. For example, to test several email templates for a seminar which your company is going to hold, select the “Participated in events”.
Start sending emails as part of the split test
Go to the third wizard Start step and specify the email sending time. You can start sending manually or specify the time when the email will be started automatically. All split test emails will be sent simultaneously.
Analyzing the results
When your split test is performed, analyze the email totals at the Results step. The dashboard functionality of this step is similar to the functionality of the email delivery statistics diagram. Read more in “Email delivery analysis”.
The diagram displays summary information about the split test, for example, the total number of all email opens by different test groups. The totals for each email are displayed in a table above the diagram (Fig. 2).
When evaluating the split test results, adhere to the recommendations specified in the “Split testing guidelines” article
Sending the better variant of the email to the remaining audience
If at the Audience step of the wizard you select a test group audience percentage that is different from 100%, you can send the winning email to the rest of the target audience that was not involved in the test group. To do this:
-
Select the needed email in the Email section and click Copy in the Email section.
-
Select the dynamic folder in the Contacts section, which will be used for sending the winner email. This folder includes contacts from the segment that were tested but were not included in the split test audience. An example of setting the folder is shown in Fig. 3.
