Why might the same user id get different treatments?

Question

Why are impressions for the same user id showing different treatments? Shouldn't the same user id should always receive the same treatment throughout the same version of the experiment?

Answer

The most likely explanation for this is that the definition of the feature flag has a targeting rule that checks an attribute value when assigning the treatment where if the value of the attribute state passed to getTreatment is CO, then the off treatment is assigned, otherwise the default rule applies and the on treatment is assigned.

To see if this is indeed the case, make sure that on the Live tail tab you have elected to show the Targeting rule field of the impression (this is the case by default). Then it will be clear why a different treatment was assigned.

In this case, the attribute checked in the targeting rule is the deciding factor as to which treatment a user receives. If the value of that attribute is different between the two calls to getTreatment, a different treatment could be assigned.

Of course if the version showed for the impressions is different, that is another possibility: the definition of the feature flag changed from one version to another in a way that caused the user to be assigned a different treatment.