What happens on my metric cards when I don't have events coming in for one treatment? Will I get statistical significance? How should I view my metric card?
The system will test the difference between any mean values, even if one of them happens to be 0. As a result, you can still get statistically significant results, a card that turns red or green.
In this scenario we use the absolute change instead of the relative change. If one treatment is 0 and the other treatment isn't 0, as long as there is a relevant sample size and the appropriate variance in the data, the stats calculations are not impacted.
In this case, the table on the Details and Trends page will show the appropriate data for the treatments.
In addition to seeing no events in the table for one of the treatments, the chart will be blank, reinforcing that we are using the absolute change.