Run an Experiment

Beta

Experimentation is in beta.

Once an experiment is running, Flagsmith serves the variations automatically through the flag. Your application has two jobs:

1. Record exposures: tell Flagsmith when an identity actually experienced a variation.
2. Record conversion events: send the events your metrics aggregate.

Instrument your application

Event collection is available in these SDKs, with more to follow:

SDK	Package	Minimum version
JavaScript (browser, React, server-side rendering)	@flagsmith/flagsmith	12.4.0
Python	flagsmith	6.3.0

Node.js

The Node.js SDK (flagsmith-nodejs) does not support events yet. For server-side rendering, use the JavaScript SDK's @flagsmith/flagsmith/isomorphic build and call flushEvents() before the response ends.

Enable event collection by passing the enableEvents / enable_events option when you initialise the client. Keep the environment key you already use for flags: events accept the client-side key and server-side keys.

When self-hosting, also point the SDK at your Flagsmith API and your event ingestion server, as in the commented lines below. Without them, the SDK sends events to Flagsmith Cloud. To deploy the ingestion server, see Experimentation in the Helm chart.

Users must be identified. Exposures and conversion events are joined per identity, so use the same identifier for flags and events.

JavaScript

import flagsmith from '@flagsmith/flagsmith';

await flagsmith.init({
    environmentID: 'YOUR_CLIENT_SIDE_ENVIRONMENT_KEY',
    enableEvents: true,
    // Self-hosted only:
    // api: 'https://flagsmith.example.com/api/v1/',
    // eventProcessorConfig: { eventsApiUrl: 'https://events.flagsmith.example.com/' },
});
await flagsmith.identify('user_42');

// Evaluate the flag and record an exposure in one call
const flag = flagsmith.getExperimentFlag('checkout_button');
// ...render based on flag.value

// Record a conversion event; the name must match your metric's event name
flagsmith.trackEvent('purchase', { value: 99.5 });

eventProcessorConfig requires enableEvents: true; init throws without it.

In React, pass the same options to FlagsmithProvider. The useExperiment hook resolves the flag and records the exposure:

import flagsmith from '@flagsmith/flagsmith';
import { FlagsmithProvider, useExperiment, useFlagsmith } from '@flagsmith/flagsmith/react';

export function App() {
    return (
        <FlagsmithProvider
            flagsmith={flagsmith}
            options={{
                environmentID: 'YOUR_CLIENT_SIDE_ENVIRONMENT_KEY',
                identity: 'user_42',
                enableEvents: true,
                // Self-hosted only: add api and eventProcessorConfig as above
            }}
        >
            <CheckoutButton />
        </FlagsmithProvider>
    );
}

function CheckoutButton() {
    const flagsmith = useFlagsmith();
    const flag = useExperiment('checkout_button'); // null until flags load

    return (
        <button onClick={() => flagsmith.trackEvent('purchase', { value: 99.5 })}>
            {flag?.value === 'buy_now' ? 'Buy now' : 'Checkout'}
        </button>
    );
}

Python

from flagsmith import EventProcessorConfig, Flagsmith

flagsmith = Flagsmith(
    environment_key="YOUR_ENVIRONMENT_KEY",
    enable_events=True,
    # Self-hosted only:
    # api_url="https://flagsmith.example.com/api/v1/",
    # event_processor_config=EventProcessorConfig(
    #     events_api_url="https://events.flagsmith.example.com/",
    # ),
)

# Evaluate the flag and record an exposure in one call
flag = flagsmith.get_experiment_flag("checkout_button", identifier="user_42")

# Record a conversion event; the name must match your metric's event name
flagsmith.track_event("purchase", identifier="user_42", value=99.5)

Local evaluation

get_experiment_flag does not record exposures with local evaluation (enable_local_evaluation=True) yet. Use remote evaluation for experiment flags.

Exposures

An exposure is the moment an identity is served a variation; it is what enrols them in the results. getExperimentFlag / get_experiment_flag evaluates the flag and records the exposure in one call.

Record the exposure where the user actually experiences the change; if you evaluate in one place and render in another, call trackExposureEvent / track_exposure_event at the point of display instead.

Exposures are deduplicated per identity: each identity is counted once, in the variation it saw first, so recording an exposure more than once is safe.

Conversion events

trackEvent / track_event sends a named event, optionally with a numeric value, traits and metadata.

The event name is case-sensitive and must exactly match the metric's configured event name. Unmatched events are silently ignored. Conversions are attributed post-exposure: an identity's events only count after its first exposure.

Troubleshooting

The SDKs skip events silently rather than throw. If the Exposures panel stays empty, check that:

• The environment has a connected warehouse. The ingestion server rejects events for environments without one.
• The user is identified. Exposures without an identity are skipped.
• Every variation of the flag has a label. Exposures are attributed by label, so a variation without one, for example one created through the API without a key, is not counted.
• The identity is inside the experiment's rollout. Identities outside it are not enrolled and record no exposure.
• In Python, the client uses remote evaluation.
• Event names match your metrics exactly, including case.

To see why an event was skipped, turn on SDK logging: enableLogs: true in JavaScript, or the flagsmith logger at DEBUG level in Python.

While it runs

• The flag is locked while the experiment is running.
• Monitor enrolment on the experiment page's Exposures panel to confirm data is arriving.
• You can edit the rollout from the experiment page, but changing it mid-experiment can affect the statistical validity of your results.