Metrics impact

Overview

The Metrics impact tab shows the impact of your experiment or feature rollout on your customers. To make data-driven decisions on your features, it is critical to review and interpret the data that you have collected before deciding to roll out the functionality to more customers.

Whether the impact is statistically positive, negative, or inconclusive the insights are an incredibly valuable resource. The data on the Metrics impact tab allows you to ensure safe and reliable feature delivery while powering data-driven decisions.

A few quick tips:

• Understand how your most important metrics (overall evaluation criteria) were both positively and negatively impacted to learn more about what your customers expect and how you should change your feature functionality.
• Compare the actual impact with your team's preliminary hypothesis.
• Ensure that you understand the impact and tradeoffs on your organization's guardrail and performance metrics.
• Share the impact with your team!

Metrics impact tab

To view the impact of your experiments on your organization's metrics, go to the Metrics impact tab on a split page. The tab shows how your organization's metrics change when looking at customers in a particular treatment as well as when comparing treatments. Note, automatic calculations will be run for split versions which include a percentage targeting rule. On-demand calculations can be run at any time via the Recalculate metrics button.

The Metrics impact tab is clearly laid out in a few key sections.

A. Selection criteria
Select the version, targeting rule, and treatments that you want to compare. When you select the treatment, you can easily see the number of customers in that treatment.

B. Summary
View how long your experiment has been running, and the last update time for the metrics displayed below. You can also force a recalculation of your metrics via the Recalculate metrics button, this recalculation usually takes around 5 minutes but is dependent on the length of your experiment and the size of your data.

C. Filters
Filter down to metrics with a positive or negative impact by clicking the tile. You can deselect and view all by clicking the tile again.

D. Key metrics
Select key metrics for this split. If you are releasing a new feature behind this split, display the key success metrics for this feature release here.

E. Organizational metrics
The remainder of your organization's metrics are displayed here. Any metrics that changed in a statistically positive and negative way are displayed first.

Learn more about applying filters to your data and understanding the impact on your customers.

If you have any additional questions or need help troubleshooting, email us at support@split.io.