Experiments (A/B Testing)

Experiments let you run A/B tests on your quizzes and paywalls — compare different variants against each other and make data-driven decisions based on real user analytics.

Two modes of experiments

1. Variant routing (classic A/B test)

The classic mode routes different users to entirely different quiz or paywall variants. For example, you can compare two paywall structures, two different payment method setups, or two quiz flows. Each user sees exactly one variant.

2. Content A/B testing (intersection experiments)

Intersection experiments let you test specific pieces of content within a quiz — a headline, an image, a button label — without creating separate full quiz variants. Multiple intersection experiments can run simultaneously for the same user. See Conditional Logic for Fields and Screens for how to use exp_* properties to vary content based on experiment group.

---

Creating an experiment

Go to Experiments in the left sidebar and click New Experiment.

Basic settings

Variants

Add two or more variants. Each variant points to a specific quiz or paywall and has a percentage share that controls how traffic is split.

Example — 50/50 paywall test:

• Variant 1: Control → Default Paywall → 50%
• Variant 2: Variant A → Test Paywall → 50%

Allow experiments intersection

Enable Allow experiments intersection to run this experiment simultaneously alongside other intersection experiments. When enabled, users can be assigned to this experiment in addition to any other intersection experiments — each stored as exp_<id> in their user properties.

Use this for content-level A/B tests where you want to vary specific elements (text, images, styles) using Conditional Logic, rather than routing users to different quiz variants.

Note: Standard (non-intersection) experiments still follow the classic model — only one can be active per user per quiz/paywall.

Advanced filtering

By default, the experiment runs for all users. Use filters to target a specific audience.

Click Add Filter and choose a filter type:

Example: Filter by Country = United States to run the experiment only for US users.

---

Experiment statuses

Status transitions

Draft ──► Active ──► Stop      ──► Archive
                ──► Pause      ──► Archive
                ──► Success    ──► Archive
                ──► Fail       ──► Archive

• A Draft experiment is started by clicking the ▶ play icon in the experiment list, which sets it to Active.
• From any non-Draft, non-Archive status you can manually change status to Active, Stop, Pause, Success, or Fail via the Status field in the experiment settings.
• The ⏹ Stop and archive button (visible on Active, Stop, Pause, etc.) moves the experiment directly to Archive.
• You cannot move an experiment back to Draft once it has left that state.

What gets locked after activation

Once an experiment leaves Draft status, the following fields can no longer be changed:

• Project
• Type (Quiz or Paywall)
• Run on (target slug)
• Variant quiz/paywall assignments

You can still edit: name, start/end dates, filters, and the Allow experiments intersection flag.

---

Viewing analytics

After the experiment has been running for at least a day or two, open the Analytics tab on the experiment page.

The analytics dashboard shows:

• Number of users per variant
• Conversion rates per variant (purchases, quiz completions, etc.)
• Revenue metrics per variant

Use this data to determine which variant performs better, then either end the losing variant or make the winner permanent.

---

How variant assignment works

1. A user opens the quiz or paywall that the experiment is running on.
2. The backend checks all active experiments for that slug and project.
3. Filters are evaluated (country, language, device, UTMs, etc.). If the user doesn't pass all filters for an experiment, they are skipped for that experiment.
4. For standard experiments: the first matching experiment assigns the user to a variant (consistent across visits — the same user always gets the same variant).
5. For intersection experiments: the user is assigned to all matching intersection experiments simultaneously.
6. The assigned variant's quiz or paywall is loaded. For intersection experiments, the exp_<id> property is set in window.user_properties and can be used in Conditional Logic.

---

Content A/B testing with intersection experiments

Step-by-step example: test a headline

Goal: Test two versions of a quiz headline — "Lose weight in 4 weeks" vs "Get fit in just 21 days" — and measure which drives more quiz completions.

Step 1 — Create the experiment

1. Go to Experiments → New Experiment
2. Set Type to Quiz, Run on to your quiz slug
3. Add two variants:
   • Control — points to your quiz — 50%
   • Variant A — points to your quiz — 50%
4. Enable Allow experiments intersection
5. Activate the experiment

Both variants point to the same quiz here. The content difference is handled by conditional logic inside the quiz, not by routing to a different quiz.

Step 2 — Add conditional logic to the headline

1. Open your quiz in the editor
2. Select the screen with the headline block
3. Select the headline block and click the slider icon next to the Text property
4. Add an override rule:
   • Condition: exp_42 == Variant A
   • Value: Get fit in just 21 days
5. The default text (Lose weight in 4 weeks) remains as the base value shown to Control users

Step 3 — Check results

After a few days, open the experiment's Analytics tab and compare completion rates between Control and Variant A.

---

Tips

• Name your variants clearly — Control, Variant A, Variant B works well. Avoid names like Test 1 / Test 2 that become meaningless in analytics later.
• Don't change variants mid-experiment — once users are assigned, changing variants invalidates the results.
• Use filters to reduce noise — if your experiment is only relevant for US mobile users, filter to that audience so you're not diluting results with irrelevant traffic.
• Run one standard experiment at a time per quiz/paywall — but you can stack multiple intersection experiments freely.
• Give it enough time — aim for statistical significance before drawing conclusions. A few hundred conversions per variant is a reasonable minimum.

---

API reference

You can manage and query experiments programmatically via the web2wave API: