A/B Testing

While Feature Flags is not explictily an A/B testing add-on, it could be used as such. If a feature flag has the "Percent Variant" option set to 50%, and you update the template tag to use an else conditional you can run an A/B test. The only caveat is that you can't run multi-variant tests since a feature flag only has 2 states, on and off.

{if feature:your_ab_test_name}
    Show A
    Show B

When using a "Percent Variant", the user is cookied to that flag, so they will continue to see the original or variant upon subsequent page requests if you use the same conditional in mulitple places.

The "Percent Variant" field only sets the percentage chance that a user will see a particular feature flag, and in the context of an A/B test refers to the probability for an individual user to see the content gated by the flag. Feature Flags doesn't guarantee that the overall distribution of users will be exactly in line with those probabilities. For example, if you set the "Percent Variant" value to 80%, Feature Flags does not track and guarantee evenly distributed users across the variants.

Last updated