2

I have produced 50 units and after testing, I notice that 3 of them have a failure. So I change my production process and produce another 50 units. This time, I detect no failures. Did I resolve the issue? Can I quantify how sure I can be that the problem is solved?

How should I approach this problem from a statistical point of view?

Finally, what if the situation was reversed, i.e. I had zero failures when measuring 50 units but after a change in the process, I find 3 failures in 50 units. Is this same situation as before? Could I conclude that the process is significantly different?

Shapul
  • 73
  • 3
  • 1
    See https://stats.stackexchange.com/questions/134380/how-to-tell-the-probability-of-failure-if-there-were-no-failures – Tim Jan 13 '16 at 19:06

1 Answers1

4

The problem is certainly a statistical problem. Given two series of outcomes one with 3 failures and 47 successes and one with 0 failures and 50 successes how likely it is that these two sequences came from a "same source" with some unknown probability of generating successes and failures?

This is exactly what statistical tests answer, and that is the p-value of the statistical test.

In this case you should start on the chi square but if you read the details of the chi square test, it does not work when when one of the "cells" is lower than 5 (and you have two of them). So you will need the Yates correction or the Fisher exact test...

As for the 2 question, the order is not important. Whether you have first the 50/0 or the 47/3 sequences, it does not mater for the statistical test: the statistical tests assumes that the two sequences came from the "same" source!

Jacques Wainer
  • 5,032
  • 1
  • 20
  • 32
  • 1
    So I read a little bit about the chi square and the fisher exact tests. If I understand correctly, I should form 2x2 table and the compute e.g. `chisq.test(matrix(c(0, 3, 50, 47), nrow=2))` which gives me (with a warning) a p-value of 0.241. It means I don't have enough evidence to reject the idea that the two observations come from the same distribution. (fisher exact test gives a slightly different p-value of 0.2424). Does that sound right? – Shapul Jan 14 '16 at 01:07
  • 1
    Yep - that is right! – Jacques Wainer Jan 14 '16 at 01:28