1

I need your help. I'm designing a study for the following problem. There is a feature A (implemented by many software from my specific context) and I'm 100% sure that if I ask the users they will agree that any software (from my specific context) is useless without feature A. Now I want designed software B without feature A, which will be accepted by the users. So basically I want proof that feature A is in general (in my specific context) not necessary.

The study design is the following:

  • Survey 1:
    • I ask the users if they need feature A, so the answers are "yes/no"
    • this question is necessary because there is no data to this question but I'm 100% sure, that most of the users will state "yes"
  • Users test my designed software without feature A
  • Survey 2:
    • I again ask the users if they need feature A, so the answers are again "yes/no" and I expect that the users will change their mind.

Now I want proof that feature A is not necessary. So I have 2 dependent samples (exactly the same users) with dichotomous dependent (feature A necessary: yes/no) and independent (software tested/not tested) variables.

My question is, which statistical test can I use here to proof my hypothesis? As the variables are both nominal only chi-squared homogeneity test comes to my mind. Are there some better tests I can use or can I somehow make a better study design?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
qwer1993
  • 25
  • 4

1 Answers1

1

The chi-squared test is intuitive, as it deals with 2x2 tables of counts, which is what you will have at the end of your study. However, it doesn't answer the question that you want to ask. Instead, you need a test of the equality of the marginal proportions of the table. That test is McNemar's test. I discuss it here and here. In brief, you are conducting a binomial test of the two off diagonal cells.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650