1

I'm new to this and I'm trying to analyse the results of a questionnaire right now. It's a questionnaire where users get to decide how a certain feature in a program will work.

There are descriptions of three ways the feature could work and they are supposed to rank them starting by their favorite. That's the result:

+---------------------+--------+--------+--------+--------+
|       Ranking       | Rang_1 | Rang_2 | Rang_3 | Rang_4 |
+---------------------+--------+--------+--------+--------+
| FeatureDescription1 |     13 |     21 |     12 |      4 |
| FeatureDescription2 |     18 |     17 |     14 |      1 |
| FeatureDescription3 |     14 |     11 |     20 |      5 |
| FeatureDescription4 |      5 |     14 |      5 |     40 |
+---------------------+--------+--------+--------+--------+

One way I found was to assign each rank a value (1,2,3,4), and then just do it like:

(13 * value1 + 21 * value2 + 12 * value3 + 4 * value4) / sum(13+21+12+4)

Any idea what's the best way for that? Right now I just used the Rank 1 values and choose the winner based on that. Is there something wrong with that?

Thanks!

mediocre
  • 111
  • 2
  • This http://stats.stackexchange.com/q/139015/3277 question is similar. There is an answer which reflects on just _some of_ many possible ways to analyze ranking question frequency data. Just for your info. – ttnphns Sep 04 '15 at 10:58
  • Thanks, that's useful and I haven't found that in my previous searches. Naively I'd say I could just use the Rank 1 to choose the "winner" and only fall back on the others if there's a draw between two Feature Descriptions. Is there anything wrong with that approach? – mediocre Sep 04 '15 at 11:14

1 Answers1

1

Is it more important to win friends (by picking their favourite) or avoid making enemies (by not picking their worst-ranked choice)? In either case, feature description #2 would be the right one.

Actually you have a stronger result for your data. If (big "if") the business benefit of a choice whose rank-counts are $(r_1,r_2,r_3,r_4)$ -- e.g. 13,21,12,4 for option 1 -- can be described as a weighted sum of rank-counts with decreasing weights ... $$ \textrm{benefit} \equiv B(r_1,r_2,r_3,r_4) = \sum_i \alpha_i r_i \\ \alpha_1 > \alpha_2 > \alpha_3 >\alpha_4 $$ ... then option 2 is necessarily the best choice.

To see this, set $\alpha_1=1$ and $\alpha_4=0$ without loss of generality, and write the benefit in terms of $(\alpha_2,\alpha_3)$: $$ B_1 \equiv B(13,21,12,4) = 13 + 21\alpha_2 + 12\alpha_3 \\ B_2 \equiv B(18,17,14,1) = 18 + 17\alpha_2 + 14\alpha_3 \\ B_3 \equiv B(14,11,20,5) = 14 + 11\alpha_2 + 20\alpha_3 \\ B_4 \equiv B(5,14,5,40) = 5 + 14\alpha_2 + 5\alpha_3 $$ Now #1 beats #2 iff $4\alpha_2>5+2\alpha_3$ : impossible because $\alpha_2<1$.

And #3 beats #2 iff $6\alpha_3>4+6\alpha_2$ : impossible because $\alpha_3<\alpha_2$.

And #4 beats #2 iff $13+3\alpha_2+9\alpha_3 < 0$ : obviously impossible!

Creosote
  • 2,102
  • 8
  • 11