Paywall experiments
Run A/B tests on the products and order your paywall offers, with sticky per-install assignment, exposure counting, weight epochs and readable results.
A paywall experiment splits an entitlement's offer set into weighted variants. Each install is assigned one variant and keeps it. Results compare paid conversion per exposure across variants. Your app needs no change beyond the server-driven paywall integration.
Lifecycle
draft > running > (paused and back) > completed. An experiment past its end date shows as ended: it serves defaults but can still be concluded or extended.
| Action | Allowed from | Notes |
|---|---|---|
| Create | Created as a draft; reaches nobody. | |
| Start | draft | Needs a duration and entitlement defaults. The control's SKUs are replaced by a frozen snapshot of the defaults. |
| Pause | running | Stops new assignments. Assigned installs keep their variant. Serves defaults on a 900 second TTL. |
| Resume | paused | Fails if it ended while paused; extend the end date first. |
| Update | draft, running, paused | While live, only weights and the end date can change. |
| Conclude | Live or ended | Requires a winning variant; can promote the winner to the defaults. |
| Delete | draft only | Started experiments are history. |
Validation rules
| Rule | Error sentence |
|---|---|
| 2 to 6 variants | An experiment needs at least two variants / At most 6 variants |
Variant keys match ^[a-z0-9][a-z0-9_-]{0,23}$ | Variant keys must be short lowercase identifiers |
| Unique keys | Duplicate variant key "…" |
| Whole-number weights 0 to 100 | Weights must be whole percentages between 0 and 100 |
| Exactly one control | Exactly one variant must be the control |
| Weights sum to 100 | Weights must sum to 100 (currently 110) |
| Name at most 80 characters | Name the experiment (at most 80 characters) |
| Duration 1 to 180 days | Duration must be 1 to 180 days |
| At start, every non-control variant has SKUs | Variant "…" has no SKUs |
| At start, defaults exist | Set the entitlement's default SKUs before starting; the control variant snapshots them |
| One live experiment per app and entitlement, checked at start | An experiment is already live for this entitlement |
| Live edits | Only weights and the end date are editable while a test is live; changing what a variant shows mid-test corrupts its cohort |
Leave the control's skus empty when creating. Editing defaults mid-test does not change a running experiment. targetExposures is an optional, advisory count. Pausing does not release the live slot; only concluding does.
Reweighting
Changing weights while running opens a new epoch. Existing assignments stick; only new installs follow the new weights. The sample-ratio check runs per epoch, so a reweight does not false-flag the test.
Exposures
Fetching assigns; the impression beacon exposes. Results use beacon-confirmed exposures (mode: exposed) when the app calls paywallShown, and fall back to assignment counts (mode: assigned) otherwise, which over-counts apps that prefetch. Each beacon also counts a render issue when a SKU the variant specified did not render.
Reading results
Per variant: exposures, startInstalls, paidInstalls, paidRate (with a Wilson interval), trials, trialConverted, trialConversionPct, per-SKU starts and paid, realizedUsd, normalizedMonthlyUsd, and per-exposure versions of both. The headline is paid installs per exposure with its interval.
Check health before believing a result:
srm.flagged: the split is not landing at the configured weights. Do not conclude.renderIssuePct: a high value means a broken variant, not a losing offer.attribution.coveragePct: low coverage means revenue undercounts; checkattributecalls.environment: denominators count Production only; Sandbox and Xcode traffic is shown, not counted.maturingInstallsandtrialMaturing: outcomes not decided yet.truncated: the scan hit its cap; numbers are a floor.
Do not call a winner on overlapping intervals.
QA a variant
Send forceVariant: "<key>" from a non-Production environment (Sandbox or Xcode). The server returns that variant with forced: true and writes no assignment, so QA never pollutes results. The SDKs never cache forced answers.
let preview = await RevenueHog.paywall(for: "pro", fallback: ids, forceVariant: "monthly_first")Where to run experiments
From the dashboard, the RevenueHog mobile app, the paywall management API, or the revenuehog-experiments skill in Claude Code (see Skills).
Related
Last updated on
Server-driven paywall
Let the RevenueHog dashboard decide which products your paywall offers and in what order, with a compiled-in fallback, a TTL cache and the impression beacon.
In-app feedback
Send bug reports, ideas, questions and praise from your app to RevenueHog with the keyless SDK feedback endpoint, or from your servers with an API key.