Experiments

Storyblok Experiments enables content variants at the story level. Use experiments for A/B testing or to manage personalized content.

Note

Experiments is a premium feature. Learn more on the pricing page.

Caution

Experiments is part of Storyblok Labs and is still in the beta stage. Enable it as shown in Labs.

The feature is built around three core elements:

• Experiments: define which variants get tested, their weight distribution, and optionally for how long.
• Variants: create variations of stories to test against each other.
• Results: feed performance data from your analytics platform into Storyblok via an API endpoint, giving teams visibility into experiment results without leaving the CMS.

Create an experiment

Navigate to Labs → Experiments and select + Create new.

Create a new experiment

In the modal, enter an experiment name and a technical name (used as the identifier in the Content Delivery API). Optionally, add a description that summarizes the scope and goal of the experiment.

By default, an experiment includes two variants: a control and a test. The control is the variant designated as the baseline against which other variants are measured—any variant can serve this role. Each variant requires a name, a technical name, and a weight distribution in percent. The sum of all distribution percentages must equal 100 percent. Select + Add variant to add additional variants.

Select Create. The experiment is created in draft status. Associate at least one story with the experiment before starting it.

Link a story to an experiment

An experiment can include multiple stories, but each story can be linked to only one experiment at a time. Stories can be linked only while the experiment is in draft status.

Under Content, open a story in the Visual Editor. In the right sidebar, open the Experiment section and link the story to the experiment. Once linked, the experiment title, status, and variants appear.

Link an experiment to a story

Access the story’s variants using the variants toggle in the top toolbar. Select a variant to edit its content.

Select a variant in a story

In the Edit section, make changes to the variant’s content by editing its fields. Save and publish changes to each variant independently from the main story.

Repeat the process for any additional stories to include in the experiment.

Tip

To create a new experiment directly from within the Visual Editor, select + Create new in the Experiment section.

Unlink a story from an experiment

A story can only be linked to one experiment at a time. To switch experiments, select Unlink experiment next to the experiment title, then select a different experiment.

The following applies when unlinking a story from an experiment:

• From a finished experiment: variants other than the winner are deleted
• From a draft experiment: all variants are maintained
• From a running experiment: unlinking is not possible

Start, pause, and finish an experiment

After an experiment has been created and linked to at least one story, it moves through three states: running, paused, and finished. Each state has its own action in the experiment view.

Start

Open the experiment under Experiments. Alternatively, open it from the Experiment section within the Visual Editor.

All stories and their variants must be published before the experiment can be started.

Select Start experiment.

Pause

Select Pause experiment to pause a running experiment. A paused experiment can be resumed at any time. While paused, variant content no longer appears in production.

Finish

Select Finish experiment to end the experiment. This action cannot be undone.

Choose a winning variant immediately, or decide later. Once you select a winning variant other than the control, it supersedes the control and becomes the only published story.

Results

In the experiment details, select Results to review metrics for ongoing and finished experiments.

Analytics data needs to be pushed to Storyblok in order to show experiment results. Refer to the Experiments concept to learn more.

Compatibility

The following sections describe how Experiments works alongside existing Storyblok features. As Experiments is part of Storyblok Labs, functionality is subject to change.

Workflows

Storyblok tracks workflow stages independently for the main story and each variant. Refer to the Workflows manual to learn more about workflows and workflow stages.

Search

Search and indexing apply only to the main story. Variants are excluded from the search functionality of Storyblok’s Content Delivery API and from the Algolia app.

Internationalization

Experiments support both field-level and folder-level translation.

With field-level translation, fields marked as translatable can hold a different translation per variant, independent of the main story.

With folder-level translation, add individual stories to an experiment and manage its variants per story. The Dimensions app is currently not supported in combination with Experiments.

Refer to the Internationalization concept to learn more.

Releases

When working with the Releases app, variants are disabled while editing a story in an upcoming release. Only the main version of the story can be edited within the release. On publish, Storyblok merges the changes from the release version into the current main story, and variants carry over unchanged.

Pipelines

Content staging for Experiments is currently not supported. When working with the Pipelines app, variants are available only in the preview branch.

Backups

Experiments and variants are currently excluded from backups.

Space duplication

Experiments and variants are currently excluded from space duplication.