3

If I run an A/B for the recommended duration which was 2 weeks and after that duration, the test results are not statistically significant at 95% significance level. If the PM comes back to me and says he is okay with a lower lift(instead of 10% now the lift is 5%) which gives us a duration of 3 weeks after running power analysis. Lift was used to calculate the new sample size and duration. Should we let the test running for another week or stop it? My understanding is we should terminate and start a new test for 3 weeks.

Peppy
  • 31
  • 2
  • 1
    You are going to have to describe how you used your "lift" in the calculation of the results being not statistically significant – Henry May 13 '21 at 09:20
  • 1
    Highly relevant: https://stats.stackexchange.com/questions/20676 and https://stats.stackexchange.com/questions/310119. – whuber May 13 '21 at 13:01

0 Answers0