Step 4: Create a metric, monitor and measure the impact

You can create a metric, monitor, and measure impact in six steps:

1. Define your metric name and meta information
2. Define your metric definition
3. Create an alert policy
4. Manage alerts
5. Review metrics impact during a review period
6. Analyze metric impact details and trends over time

Step 1: Set your metric name and information

Once you are sending us event data, to create your first metric, click on the “metrics” icon in the left navigation and click the button labeled “create metric”. From there, you can enter your metric name. We recommend describing what you intend to measure with a unique and recognizable name. Similar to splits, you can easily specify meta information for your new metric such as name, description, owners, and tags. This information can help you and your team manage your metrics. Learn more about creating a metric.

Step 2: Define your metric definition

When defining the metric definition you can determine the winning direction. If increasing this metric is good for your business and customer experience, select “increase”. If not, select “decrease”. The traffic type applies to the event type associated with the metric. Select traffic type “user”. To get started, select “count of events per user” and select an event you sent. Learn more about defining your metric.

Step 3: Create an alert policy

An alert policy allows you to configure a degradation threshold for your metric. If the feature flag causes the metric to go below or above that threshold you will be alerted via your configured notification channel. Learn more about creating an alert policy.

Step 4: Manage alerts

Any alerts which have been triggered will be displayed both on the "Targeting" and "Alerts" tab on the split page. When youn receiving an alert you can quickly rollback the split by clicking "kill" or dismiss the alert. Learn more about managing alerts.

Step 5: Review metric impact during a review period

The review periods will be preset via the admin settings. You can access your metric impact by searching for the split you want to see the impact for and navigating to the third tab on the right, “metrics impact”. Once here you will see the impact on your organization’s metrics, first ordered by key metrics and then ordered by the additional organizational metrics which reached statistical significance. Learn more about review periods and filtering your metrics.

Step 6: Analyze metric impact details and trends over time

Information such as the impact % , p-value, and error margin will be surfaced on the metric card. You can click on the metric card to access additional information. This additional information includes how the impact has changed over time, the metric dispersion and the sample size used to calculate the metric value. Learn more about metric details and trends.