Metrics impact tab

The Metrics impact tab shows the impact of your experiment or feature rollout on your customers. To make data-driven decisions on your features, review and interpret the data that you collect before deciding to roll out the functionality to more customers. The data on this tab allows you to ensure safe and reliable feature delivery while powering data-driven decisions. Before getting started, review the following:

• Understand how your most important metrics (overall evaluation criteria) were both positively and negatively impacted to learn more about what your customers expect, and how you should change your feature functionality.
• Compare the actual impact with your team's preliminary hypothesis.
• Ensure that you understand the impact and tradeoffs on your organization's guardrail and performance metrics.
• Share the impact with your team.

Viewing metrics

To view the impact of your experiments on your organization's metrics, from your selected feature flag, click the Metrics impact tab within the feature flag configuration. The tab shows how your organization's metrics change when looking at customers in a particular treatment and when comparing treatments.

Note: Automatic calculations are run for feature flag versions that include a percentage targeting rule. Click the Recalculate metrics button to run the on-demand calculations at any time.

The Metrics impact tab is described as follows:

• View impact for. Select the version, targeting rule, and treatments that you want to compare. When you select the treatment, you can see the number of customers in that treatment.
• Summary of key and organizational metrics. View how long your experiment has been running, and the last update time for the metrics displayed below. You can also force a recalculation of your metrics by clicking the Recalculate metrics button. This recalculation usually takes around 5 minutes but is dependent on the length of your experiment and the size of your data.
• Filter metrics. Filter down to metrics with a positive or negative impact by clicking the tile. You can deselect and view all by clicking the tile again.
• Key metrics. Select key metrics for this feature flag. If you are releasing a new feature behind this flag, display the key success metrics for this feature release.
• Organizational metrics. The remainder of your organization's metrics are displayed. Any metrics that changed in a statistically positive and negative way are displayed first.

About recalculating metrics

The following discusses the conditions when the recalculate button is enabled or disabled.

Enabled recalculate button

The recalculate button is enabled if the following conditions occur:

• A version is created before the retention period. This version ran longer than the current 90 day data retention for your organization. When you recalculate this feature flag, it only uses data from the last 90 days.
• Others. If you create or modify a metric after the last updated metric impact calculation, recalculate the metric to get the latest results. Note: Most recalculations take up to five minutes, but can take longer, depending on the size of your data and the length of your experiment.

Disabled recalculate button

The recalculate button is disabled if the following conditions occur:

• When it is an older version and the Retention Period (RP) expires. The data retention period lapsed.
• When no impressions are received since the creation of the version. No traffic is received since this version was created. To enable the recalculation, make sure your SDK is configured properly and that traffic is being sent.
• When the last impression time of a version is before the retention period. No traffic was received in the data retention period.
• A forced experiment is already scheduled. A calculation is in progress. To recalculate, wait until it finishes.

For detailed information about specific metric cards, refer to Understanding metric impact. Learn more about applying filters to your data and understanding the impact on your customers.

If you have questions or need help troubleshooting, contact support@split.io.