There are three primary use cases for determining "who is seeing which treatment" that can be answered in Split:
Use Case 1 - What will a user see whenever they log back in?
This question is often asked to validate the rollout plan when a Split is changed. You can answer this question by going to the "Key Page". Here's an example:
In this image, you can see that the user "bilal" will see the "off" treatment for the "aa_test" split every time they log in, based on the current rollout plan. Similarly, they will see the "show_text" treatment for "front_end_show_more_info".
- This view will only show you treatments for Splits that are not using attributes for targeting. This is because for data privacy reasons, Split does not copy attribute values to the Split cloud. So, we do not know the value of a specific attribute for user "bilal".
- The key "bilal" is just for demonstration purposes. It will typically be a database id or a cookie id.
How to get to the key page:
- You can get to the key page either by searching for that key (say "bilal") in the search box near the top left corner of the app, above ("My Work").
- You can also click on the key id in Live Tail, once paused, or the Datahub to get to the key page.
Use Case 2 - What treatments is a user seeing right now for one or more Splits?
This question is often asked when debugging the live experience of a customer or when making changes to a Split to see which users are seeing a particular treatment.
You can answer this question by going to Datahub. See this example:
Here, you are searching for a specific key (say user id). Split fetches the live treatments that are being assigned right now. For instance, the key "b5181890-686d-11e9-8af6-0a1c02136776" is seeing treatment "on" for "es_disable_impressions" and "data_hub_live_tail" while seeing treatment "performance" for "group_by_event_type
- Since this is a live tail of logs, this query will only show you results from "after" the time you issued the query. It will not show you events from the past.
- Also, given the size of the data we are searching for, there is an initial lag in the query response of about 10 seconds.
How to get to Datahub?
- You can click on the "Data hub" in the side navigation to get to this page.
- If you are on a split page, you can also click on "Live Tail" tab to get a filtered view of impressions for that specific split.
As noted above, you can get to the key page from the Data Hub or the Live Tail tab by pausing the query and clicking on the key.
Use Case 3 - What treatments was a user shown in the past?
This is a common question for support teams. You can answer this question by going to the "Data Export" tab of Data Hub as shown in the picture above. However, this functionality is currently behind a split and will be released during the month of August 2020.