How to conduct split tests correctly
Split testing (A/B testing) is a professional email marketing tool used for comparing bulk email effectiveness.
As the test progresses, two or several emails containing a modified item, such as email subject, a template or a sender name, are compared. The emails are sent to the test group of the target audience in equal amounts.
To perform split testing of several emails:
1.Create email variants that you plan to test in the [Email] section. Define the test object: a variable in the template, email caption or the sender's name.
Afterward, create a split test and add the prepared emails into it.
2.Select the target audience of the split test. You can define the test group of the contacts who will receive the email variants during the split test or conduct testing on a selected segment.
3.Run split tests Specify the start time and Creatio will send all the variations of the email simultaneously. This helps to exclude time as a factor that influences the result.
4.After some time, analyze the totals of the split test and define which variant of the email had the best conversion.
5.Apply the winning variant of the email to the remaining target audience not included in the test group. Optimize the email templates based on the results of the testing.
Read more about split testing in the “Starting a split test analysis” article.
-
Test email versions that differ by one variable in the template, subject line or sender name.
-
Form email variants for split tests that differ only slightly. This will simplify the measurement of user responses and test result evaluations.
-
Create an email variant that will be used as the primary. A “Primary” bulk email has no changes made to it. Test alternative variants together with the primary one and estimate the results.
Recommendations on the audience
-
The audience to whom the templates variants are sent must be large enough to receive a statistically significant result. It is recommended to include at least 1000 contacts in a split test.
-
Perform split testing using an audience that belongs to one segment, to minimize the effect of the time factor on the result.
-
Before starting the A/B testing, perform A/A testing to make sure that the audience segment is homogeneous. Send the same email variant to two groups of the target audience that are equal by the number of contacts. If the conversion indicators are the same for the two groups, it means that the segment is homogeneous.
See also
