Create an A/B Test Experience

An A/B test experience is a randomized experiment that lets you present two or more content variants of a web page or an app and display them to different sets of users at the same time to verify the effectiveness of these variants.

In Contentstack Personalize, A/B Testing empowers you to make data-driven decisions and optimize your content strategy by experimenting with different content variations. By creating and comparing these variations, you can:

• Identifying Effective Content
• Optimizing for Conversions
• Personalizing Experiences
• Reducing Bounce Rates
• Improving Customer Satisfaction

When to use A/B Test experiences in Contentstack Personalize?

• Comparing Content Variations
• Data-Driven Decision Making
• Optimizing for Conversions
• Experimentation and Iteration
• Technical Expertise and Resources
• Contentstack's A/B Testing Capabilities

By carefully considering these factors, you can determine whether A/B testing is the right fit for your optimization strategy and leverage it to drive better results.

• Contentstack account
• Access to the Contentstack Organization that has Personalize enabled
• Access to a project in Personalize
• Audience created in your Personalize project
• Event created in your Personalize project

Note: Users with Owner and Member access to a Personalize project can create A/B Test experiences.

Note: For this guide, we have assumed that you have already created a Personalize project. If not, please follow this guide and create a project in Personalize.

To create an A/B Test Experience, log in to your Contentstack account and perform the following steps:

1. In the top navigation bar, click the App Switcher icon and then click Personalize.
2. You will be redirected to the Personalize Projects landing page. Click the project for which you want to create a A/B Test Experience.
3. On the Experiences page, click the + New Experience button.
4. In the Select Experience Type modal, click the A/B Test experience type.
5. On the experience draft page, in the Overview tab, provide a suitable Name and an optional Description for the experience and then click the Save General Details button.
6. Click the Configuration tab.
7. Under the Variants section,
   1. Variant Distribution: Variant Distribution in A/B testing is the process of determining how traffic is to be divided between the different variants of your content. You can choose to distribute traffic evenly (Equally split) between all variants, or you can weight the distribution (Custom) in favor of one variant over another. By default, this is set to Equally split.
      Here, we are using the Equally split option for the Variant Distribution.
   2. After selecting the variant distribution, you can create variants for the A/B Test. Variants are alternative versions (CMS Entry Variants) of content or experiences created for testing against each other. To create variants for your experience, click the + Add Variant button.
      1. Short UID: This unique ID is automatically assigned to each variant and used in Personalize Edge SDK and Personalize API requests.
      2. Variant Name: Provide a meaningful name for the variant. This name will be reflected in the Entry Editor for the Entry Variants in the CMS.

         Note: Each A/B Test experience must contain at least 1 variant. Each experience is reflected in the CMS as Variant Group and you can create Entry Variants for each of these variants via the Entry Editor.

      3. Traffic Distribution in %: For Equally split the distribution percentage is automatically calculated. For Custom, you can specify the percentage as per your preference.Similarly, you can add multiple variants by clicking the + Add Variant button.
8. Under the Metrics section,
   1. Click + Add Event and select the preferred event from the drop-down list.You can add multiple events to an A/B Test experience as Metrics. Use the Personalize Edge SDK to trigger the events for your experiences using the triggerEvent method.

      Note: The first event that you add becomes the ‘primary’ metric. The 'primary' metric determines the winning variants, while secondary metrics offer additional insights. When any of the listed events occur, metrics calculate an increase in unique conversions per visitor, which you can view in the experience Analytics tab.

9. Under the Target Group section,
   1. Target Group: By default, Everyone is part of the A/B test, you can target the A/B test to a specific set of audiences (Selective). Here, we are using the Selective split option for the target group.
   2. Condition: You can set the rule to satisfy either all audiences (Match All) or at least one of them (Match Any) by selecting from the dropdown.
   3. Audiences: Click the Audiences field. From the Select Audience(s) modal, select one or multiple audiences, then click the Apply Selected Audiences button to set the audience criteria for the target group.
   4. Once you have defined your variant, click the Save Draft button.
10. Now click the Activate Draft button.
11. Click Activate in the Activate Draft Modal to activate the experience on your site for your visitors.

    Note: Ensure you create Entry Variants in the CMS before activating an experience for a seamless personalized campaign for your visitors.

This creates a new A/B Test Experience in your Personalize project. For managing experience versions (activated/draft/paused), please refer to this guide.

Additional Information: To achieve optimal personalization for your users, prioritize experiences and variants independently. Use experience priority when multiple experiences customize the same element on a digital property, and apply variant priority when audience conditions match multiple variants. For more details, please refer to this guide.

After activating your A/B Test experience, the Analytics are updated within a few minutes when your visitors start accessing and interacting with your content.

Note: To design, preview, and analyze A/B Test experiences effectively, make sure Live Preview and Visual Builder are enabled and set up on your stack. Live Preview provides real-time visibility into how content appears across variants. Visual Builder allows editors to manage variant-specific content and preview multiple variants simultaneously based on different audience selections without developer involvement.

Additional Resource: You can use the Personalize Management API to create, edit, delete, and retrieve all existing experiences.