1

I am designing a questionnaire to compare user experience of usage of a two prototypes of a software - prototype A and B. Users will rate prototype A for a task and then prototype B for performing the same task based on a 5 point Likert scale. The questionnaire will contain 5 questions related to the various measures of user experience and contain ratings for each measure. How can i compare the Likert ratings for the two prototypes using statistical tests?

iceman
  • 115
  • 3
  • Median for central tendency, inter-quartile range for dispersion, [Wilcoxon's signed-rank test](http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) for testing median difference. – Penguin_Knight Feb 27 '13 at 17:34
  • @Penguin_Knight; i was reading the wikipedia article on the Wilcoxon signed-rank test..is there an example with a software package that i can follow such as Matlab/Excel/SPSS, particular to my case? i am getting confused by the volume of stats jargon(being a software dev and not a ux specialist)..thanks – iceman Feb 27 '13 at 17:40
  • Here is a site on [how to do it in four common software](https://wikis.uit.tufts.edu/confluence/display/SSSI/Wilcoxon+Signed-Rank+Test) – Penguin_Knight Feb 27 '13 at 17:43

1 Answers1

2

Just to summarize the comments into an answer.

For comparing ordinal variables like Likert's scale, you may consider using median to represent the central tendency, and inter-quartile range to show the dispersion.

If you would like to formally test the difference in mean rank, you can use Wikcoxon's signed-rank test. Here is a wiki describing how to do it is SPSS, Stata, SAS, and R.

If you would like to test if the medians are equal, sign test is another option, as suggested by ttnphns in the comment.

Penguin_Knight
  • 11,078
  • 29
  • 48
  • 1
    **Wilcoxon sign-rank test** is a 2-related-samples test which tests H0 that the difference between the two variables is symmetric about 0. **Sign test** tests H0 that this difference has median 0. – ttnphns Feb 27 '13 at 18:31
  • @ttnphns Thanks for the correction. I have edited the answer to reflect the difference. – Penguin_Knight Feb 27 '13 at 18:37
  • Please don't be hurry. `difference in mean rank`, `the medians are equal`. How do you see it with a _paired-sample_ test? – ttnphns Feb 27 '13 at 18:48
  • @ttnphns In that case I am probably confused by your phrase "between the two variable is symmetric about 0." Since this is a wiki, please feel free to edit my answer. – Penguin_Knight Feb 28 '13 at 00:44
  • @Penguin_Knight do i have to do a-priori power analysis for determining the sample size for administering 5 Likert scale questionnaire..or can i randomly choose the number as 10 participants? – iceman Feb 28 '13 at 13:09
  • @iceman I do believe so. I am not very familiar with effect sizes of non-parametric tests, it is certainly possible. Here is a [SE thread](http://stats.stackexchange.com/questions/30455/how-to-determine-the-effect-size-of-a-wilcoxon-rank-sum-test-in-r) and an [article](http://onlinelibrary.wiley.com/doi/10.1111/j.1541-0420.2008.01062.x/abstract;jsessionid=2F19AE99435F55C2C460D0CFC3C39182.d01t02) for your reference. – Penguin_Knight Feb 28 '13 at 14:06
  • I found a link mentioning the statistics to calculate:http://www.icbl.hw.ac.uk/ltdi/cookbook/info_likert_scale/index.html . is there a simple worked out example for Likert items treated as ordinal data? – iceman Mar 08 '13 at 01:05
  • @iceman, try Google [wilcoxon's Signed-rank test example](https://www.google.com/search?q=wilcoxon+signed+rank+test+example&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a) and [sign test example](https://www.google.com/search?q=sign+test+example&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a). Good luck! – Penguin_Knight Mar 08 '13 at 02:25