0

I want to run a A/B test where option B is more profitable than option A. Option A has a response rate of 70%. Hence i want to achieve a response rate of 70% for both B as well. I want no statistical difference between the two response rates. Lets say i ran N cases through both A and B and received 70% response rate in both cases. I want to conclude that there is no statistically significant difference between the two proportions.

Edit:-Any difference less than 50 bps(0.5%) would be considered immaterial and i want to draw the conclusion at 95% confidence and 80% power

What should be the minimum N and how would i determine it?

IamNotLegend
  • 79
  • 4
  • 1
    That's not a conclusion you can make. All you can say from such data is that you cannot *detect* a difference of some specified size. Thus, to ask a question like this, you need to stipulate (a) how small a difference you consider to be immaterial and (b) the confidence with which you want to draw your conclusion. – whuber Dec 06 '17 at 18:27
  • 1
    Any difference less than 50 bps would be considered immaterial and i want to draw the conclusion at 95% confidence and 80% power. What should be the minimum sample size now? – IamNotLegend Dec 06 '17 at 21:01
  • A similar question: https://stats.stackexchange.com/questions/235750/experimental-design-on-testing-proportions/270076#270076 – kjetil b halvorsen Dec 06 '17 at 21:26
  • It doesn't answer my question. – IamNotLegend Dec 06 '17 at 21:31
  • Agreed--it answers a different question. Your improved formulation is excellent (although it would be good to spell out what "bps" is--not everyone will interpret it as basis points or even know what a basis point is). Please incorporate that information in your question (not everyone consults the comments). – whuber Dec 06 '17 at 22:17

0 Answers0