Does changing a targeting rule's percentages alter existing treatment allocation?
I have a basic on/off treatment split that is off by default. I've enabled specific segments of users to get the on treatment and would like to control the mass rollout of the on treatment beyond those segments incrementally via % using a new rule of "if user not in segments then serve 20:80 on" then 40:60 on and so on until 100:0 on.
I'm a bit confused though about whether incrementing that percentage number will result in users who were previously served the on treatment at say 20:80 then not getting the on treatment when incremented to 40:60 (assuming no other split or rule configuration changes, only incrementing the % allocations of the same targeting rule)
-
A bit more reading put me on to this article which sounds like I should actually be performing the larger rollout via the default rule: https://help.split.io/hc/en-us/articles/360030117011-Why-setting-the-order-of-treatments-matters
Reading that suggests the % bucketing would be consistent as the on/off ratios are adjusted incrementally.
-
Two things: 1) you don't need to exclude the users who are in the segment as long as that rule comes before the percentage rollout. So, if you use the Default Rule for "everyone else', the segment will be set in a Whitelist or in a Targeting Rule. That will get evaluated first, so you don't explicitly need to exclude those in the segment. If you are going to put percentages on a Targeting Rule then make sure your segment is the first Targeting Rule.
2) Users always stay in the same bucket every time they are evaluated. So any user in bucket 15 will always be in bucket 15. Thus, if they were getting on as being part of the first 20% they will continue to get on as you increase the percentage. The only reason they would change to off is if you set the percentage down to 10:90, for example.
Here's a brief article on how we bucket: https://help.split.io/hc/en-us/articles/360030024391-How-does-Split-ensure-a-consistent-user-experience-
dave
Please sign in to leave a comment.
Comments
4 comments