From a flag to an experiment

This guide takes one flag from a simple on/off switch to a measured experiment. By the end you will have split traffic between two experiences, recorded who saw which, tied a goal to it, and read whether the new experience actually won.

It builds directly on the Quickstart: you already have a boolean flag new-checkout, a client key, and an app that evaluates the flag. If not, do that first. It takes about ten minutes.

The whole idea in one line: an experiment is a flag plus a metric. The flag already decides who sees what; the metric measures what happened next.

1. Split the traffic

An experiment needs at least two groups to compare. Open new-checkout in the console, choose an environment, and set it to serve a 50/50 split between its two variants:

• off (false) is the control: the current checkout.
• on (true) is the treatment: the new checkout you want to test.

Now half of your users get each experience, assigned deterministically by their targeting key, so a given user always lands in the same group.

2. Evaluate, and let exposures record themselves

Your app already evaluates the flag with a stable targetingKey. Because the flag now splits, each user resolves to on or off, and that resolution is their experiment assignment:

const client = OpenFeature.getClient();

// The targetingKey is the unit of the experiment: same user, same arm, every time.
const showNewCheckout = await client.getBooleanValue("new-checkout", false, {
targetingKey: user.id,
});

renderCheckout({ variant: showNewCheckout ? "new" : "current" });

That evaluation records an exposure automatically: a remote evaluation carrying a targeting key logs that this user saw this variant, deduped per user per hour. You do not have to send anything extra. (Cached-mode SDKs, which evaluate locally, send exposures explicitly instead. See Record exposures.)

The exposure is what lets Flagon attribute an outcome back to the arm the user was in.

3. Define what "winning" means

Pick the outcome the experiment should move. Open Experiments → Metrics → New metric:

• Name: Checkout completed
• Event name: checkout_completed (what your app will send)
• Type: Conversion (each unit either did it or did not)
• Direction: Increase (higher is the win)

Prefer the API? It is one call:

POST /v1/orgs/{org}/experiment-metrics
Authorization: Bearer flagon_...
Content-Type: application/json

{
  "key": "checkout-completed",
  "name": "Checkout completed",
  "type": "conversion",
  "eventName": "checkout_completed",
  "direction": "increase"
}

4. Create the experiment

Open Experiments → New experiment and build it on the flag:

1. Flag: new-checkout. The experiment reuses the flag's split and assignment.
2. Control: off. Every other variant is compared against it.
3. Primary metric: attach Checkout completed. Add secondary or guardrail metrics later if you want (guardrails are watched for regressions, not wins).
4. Start the experiment.

From this point, exposures and goal events are collected against the experiment. You can start it after traffic is already flowing: Flagon analyzes the history within your plan's retention window, so nothing is lost by attaching the metric a little late.

5. Send the goal event

When a user actually completes checkout, tell Flagon, with the same targeting key you evaluated the flag with:

await fetch("https://api.flagon.io/ofrep/v1/track", {
method: "POST",
headers: {
  Authorization: `Bearer ${process.env.FLAGON_CLIENT_KEY}`,
  "Content-Type": "application/json",
  "Idempotency-Key": crypto.randomUUID(),
},
body: JSON.stringify({
  events: [{ metric: "checkout_completed", targetingKey: user.id }],
}),
});

Flagon joins the goal event to the arm the user was assigned to, as long as it happened after their exposure. The targeting key is stored only as a salted hash; the raw value is never persisted. See Record goal events for values, batching, and idempotency.

6. Read the results

Open the experiment and go to Results. As data arrives you will see, per arm:

• Lift: how much the treatment moved the metric versus control, with a confidence interval.
• Chance to beat control: the Bayesian probability the treatment is genuinely better.
• Significance: a p-value from the frequentist test, and a separate safe to call signal from the always-valid (sequential) test, so you can peek at the numbers as they come in without inflating your false-positive rate.
• Sample-ratio check (SRM): a health flag that the split arrived as intended. If it is unhealthy, fix the assignment before trusting the readout.

When the primary metric is significant and pointing the right way, you have your answer. Roll the winner out by setting the flag's default to that variant, or stop the experiment and keep iterating.

How each number is computed is spelled out in How results work.

What you built

• A flag that splits traffic into control and treatment.
• Exposures that record who saw which arm, automatically.
• A metric and an experiment that measure the outcome.
• A result you can act on, then ship with the same flag.

Next steps

• Targeting and segments: run the experiment on just one segment, or exclude internal users.
• Holdouts: hold a global control group out of every experiment to measure your combined impact over time.
• How results work: the stats behind lift, significance, and safe-to-call.