Experiments and A/B testing

Experiments let you test content changes against a control so you can see what actually moves your metrics before rolling a change out to every product. This guide covers working with experiments through the API. You can also create and monitor them in the dashboard under Experiments.

Create an experiment

Experiments belong to a project, so every request needs a projectId. Send a POST /api/v1/experiments with at least the project and a name. ACE fills in sensible defaults for the rest: a control variant and a test variant split evenly, a 30 day duration, and conversion_rate as the primary metric.

curl -X POST "https://ace.authoritas.com/api/v1/experiments" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "projectId": "<project-id>",
    "name": "Buying criteria in descriptions",
    "hypothesis": "Adding buying criteria lifts conversion",
    "primaryMetric": "conversion_rate",
    "duration": 30
  }'

See the Experiments API reference for every field you can set, including custom variants, secondary metrics, and the confidence level.

Run and monitor

• List the experiments in a project: GET /api/v1/experiments?projectId=<project-id>.
• Read one experiment, including its current results: GET /api/v1/experiments/{id}?projectId=<project-id>.
• Start, pause, or complete an experiment, or edit its configuration: PUT /api/v1/experiments/{id}?projectId=<project-id>.
• Remove an experiment: DELETE /api/v1/experiments/{id}?projectId=<project-id>.

To follow results without polling, subscribe to the experiment.started, experiment.paused, and experiment.completed events with a webhook.

Tips

• Change one thing at a time. If a variant differs in several ways, a win does not tell you which change caused it.
• Let an experiment gather enough traffic before you read too much into early numbers. The confidence level you set controls when a result is called significant.
• Keep the control truly untouched so the comparison stays honest.