After you create your metric, you can further define it by doing the following:
-
In the Select desired impact field, select whether you want to see this metric increase or decrease.
-
In the Select traffic type field, select what traffic type to measure this metric for.
-
In the Measured as field, define functions to perform or calculate specific analyses on events. These functions can be performed per your traffic type. For more information, refer to the section Statistical comparison possible. Measured per user section later in this guide.
-
In the Value field, select which property to use as the value field. For more information, refer to the Event property capture guide.
-
In the Property filter field, select the property to filter the event by. For more information, refer to the Event property capture guide.
Advanced section
You can apply the following additional filters.
-
In the Filter by (optional) field, optionally apply one of the following filters to your analyses:
- No filter criteria. No filters are applied.
- Has done the following event prior to the metric event. For example, if you want to measure the percent of users that added to cart after viewing the product page, select Added to cart for the metric event and Viewed special offers page for this filter (recommended).
- Has done the following event (not ordered). For example, if you want to measure the percent of users that opened both Application A and Application B, select Application A login success for the metric event and Application B login success for this filter.
Note about the Has done the following event (not ordered) filter. This filter counts customers who have triggered an event at least once during the version of a feature flag. As long as a customer triggered an event within the same feature flag version, the filter is applied even if it happened after the metric event.
-
In the Properties field, apply a property filter to the Has done event you're filtering your metric by. When you select an event for the Has done filter, you can optionally select a property to filter an event by.
-
In the Capped at field, apply a cap to your metric value. With metric capping, any outlier values in your metrics are capped and replaced with a fixed threshold value. This reduces the variance and increases the sensitivity of your metric. When a metric cap is set to a per user per day, this is 24 hours from a user’s first impression within a particular version of a feature flag.
When you create a metric that is measured on a per traffic type basis, you can create an alert policy for this metric. For more information, refer to the Create alert policies guide.
Statistical comparison possible. Measured per user
When metrics are calculated per user, each user's individual contribution is calculated. Every user adds a single data point to the distribution of the user metric results, which means they all have the same weight in the result. We use the resulting distribution for statistical testing of the metrics described below.
Note: All the examples assume that users are used for the traffic type in the experiment and the users are exposed to a particular treatment of a particular feature flag version.
Function | Description | Example |
---|---|---|
Count of events per user | For each user, the number of times they perform an event is calculated. | Measure the average number of times a user visits your website. Measure the average number of support tickets a user files with your support team. |
Sum of event values per user | For each user, the sum of their event values is calculated. | Measure the total value of items purchased by a user. Measure the total number of minutes a user listened to media on your website. |
Average of the event values per user | For each user, the average value of the user's events is calculated. | Measure the average value of items a user purchases. Measure the average page load time a user experiences. |
Ratio of two events per user | For each user, the ratio of the number of times two different events are performed is calculated. | Measure the number of hotel searches that occur for a user to make a hotel booking. Measure the number of invites sent for a user to accept the invite. |
Percent of unique users | The number of distinct users who performed the event as a percent of those in the sample is calculated. | Measure what percent of your sample size clicked the checkout button. Measure the percent of users who filed a support ticket. |
Comments
0 comments
Please sign in to leave a comment.