4

I need to test whether response rate for Test B is larger than response Rate for Test A (i.e. AB Test).

  • Null Hypothesis: is Response Rate (Test B) <> Response Rate (Test A).
  • Alternative Hypothesis: Response Rate (Test B) > Response Rate (Test A).

As far as I know a z-test for 2 proportions can be used for this. A chi-square test which is a generalised z-test can also be used.

Question:

  1. Which test is more suitable for my situation?

  2. I ran both test and the p-values for both are different (approximately 20 points difference). How can I explain the difference of the p-values?

Ferdi
  • 4,882
  • 7
  • 42
  • 62
kyg
  • 71
  • 3
  • http://stats.stackexchange.com/questions/141547/are-different-p-values-for-chi-squared-and-z-test-expected-for-testing-differenc – SmallChess Dec 16 '16 at 07:42
  • It is difficult to comment on this. If the proportions are based on independent identically distributed Bernoulli trials then you could be testing the difference between two binomial proportions. Because of the central limit theorem in large samples a two sample z-test could be used as an approximation. I don't see how a chi-square test is a generalization of a z-test. Your notation for the null and alternative is confusing. Does <> mean = ? If so you are suggesting a one-sided test. – Michael R. Chernick Dec 16 '16 at 19:08
  • Hi Michael. Yes the alternative hypothesis is a one-sided test – kyg Dec 19 '16 at 00:31

0 Answers0