3

I am developing an inventory tool that has 21 items. I need to determine the weight of each of the items, as the presence of some may give a higher overall rating to a scenario than the presence of others. I need to be able to add up all those present to determine an overall weighting also.

I have 20,000 respondents, and when thinking about each of the 21 constructs of the same idea, participants chose either not representative, somewhat representative, highly representative of the construct.

Can I run a PCA to determine which of the 21 items are more representative than others, and thus warranting a higher weighting? Is this possible with only 3 points? Or rather, is it fairly widely accepted to do so? It's a high impact study, so I don't want my methodology shot down.

Kate
  • 76
  • 4
  • If the Likert rating scale (of whatewer granularity) is scale (i.e. interval or ratio) for you, you may do linear PCA. But if you believe it is ordinal - then you may not. In this case you [may do some](http://stats.stackexchange.com/a/22380/3277) nonlinear/categorical form of PCA, such for example as PCA with optimal scaling (CATPCA) or logistic factor analysis used in Item Response Theory. – ttnphns Sep 30 '14 at 01:52
  • Also, the description of your problem in your question is a bit vague, so one may hesitate about whether PCA or factor analysis or something altogether different is what you need. – ttnphns Sep 30 '14 at 01:56
  • 1
    Thank you for your response, it is appreciated. We essentially ask participants "When considering this case study, how bad do you believe the actions were?" with a three level response; not at all bad, somewhat bad, or extremely bad. – Kate Sep 30 '14 at 15:45
  • 1
    Once we have our results to the part above, I then want to be able to use these results to determine how 'bad' a case is overall, based on which items are present. For example, if items 1, 5 & 21 are present in the case, perhaps it is still only a 'slightly' bad case. But if item 3 is present, maybe it could be an 'extremely bad' case, because the weighting on item 3 might still be more than the other 3 items combined, for example. – Kate Sep 30 '14 at 15:48
  • @Kate, I am not sure I fully understood your problem. So you have 20000 people answering how bad they see each of the 21 items? And you want to have some measure of "badness" for each item? Is that correct? I don't see why you would want to use PCA for that; can't you simply average the scores across all people? You will get average badness for each item, and this seems to be what you are after. Apart from that, I suggest you update your question with all the clarifications. – amoeba Dec 16 '14 at 00:22
  • 1
    @amoeba thank you for your reply. We can average results, and get percentages. Item 1, for example, we had not at all bad = 2%, somewhat bad = 43%, and extremely bad = 55%. We have responses like this across all 21 items. We can therefore rank the items, by either ordering the responses from highest level of 'badness' to lowest based on percentage of people who scored items as extremely bad, or by combining somewhat and extremely bad scores to state the rank of items from most bad to least bad based on the percentage of participants who found each item somewhat, if not especially bad. – Kate Jan 09 '15 at 15:02
  • 1
    What I want to be able to do, however, is group the items into factors, so that when applying this measure to a scenario, each item present in that scenario is worth a different weight overall. Each of the items measures a different construct of 'bad' and so the tool is more like an evidence inventory (testing for the presence or absence of each item) rather than a tool measuring different parts of the same construct. So once I have assigned weights, the presence of a high weighted item in a given scenario may equal the same 'badness' as a scenario with 3 x low weighted items present. – Kate Jan 09 '15 at 15:11
  • 1
    I have since started running the survey now asking participants to rate the items on a scale of 1-100, so I can run PCA. I couldn't get past the problem of only having 3 levels to work with, so I re-issued the survey. – Kate Jan 09 '15 at 15:13

0 Answers0