3

  1. If the items to be "summed" or combined to create an overall index are collectively the underlying construct (i.e. I am trying to measure compliance to an intervention which has different components), wouldn't combining all the components for IRT violate the assumption of unidimensionality since they represent the intervention itself? If so, do you have a reference for this?

  2. Are there any good references that say if Likert scores are summed into a scale and the Cronbach's alpha is relatively high (~0.8), this is appropriate to use as an index?

  3. Are there any other tests of validity I should do other than Cronbach's alpha if all I have done is sum the responses?

Thanks in advance, any advice is appreciated!

ttnphns
  • 51,648
  • 40
  • 253
  • 462
oncearunner
  • 53
  • 1
  • 5
  • Could you be more precise about the dimensionality of the construct? Is it one construct with multiple components? Do you think the overall construct is unidimensional? – hplieninger Jul 24 '13 at 09:21
  • 1
    It is one intervention that has three components to it. Each component requires implementing a handful of items to be compliant. So the overall construct is compliance with an intervention, which is represented by 15 specific items that can be grouped into 3 categories. Does that answer your question? – oncearunner Jul 24 '13 at 10:44

1 Answers1

4

I don't fully understand question 1; IRT is a whole area which I used to know a little about but haven't looked at in a decade. But, in any case, adding scales that represent different things would certainly violate unidimensionality, regardless of whether it is IRT or classical test theory. But that is something that you can assess.

Regarding your second question, I have seen a few; Wikipedia refers to

Kline, P. (1999). The handbook of psychological testing (2nd ed.). London: Routledge

which I haven't seen; Googling around will find more (try Google Scholar, too).

Regarding your third question: Cronbach's alpha is not a measure of validity, it is a measure of reliability. Searching on "validity" will provide at least a start, but, in general, validity is much harder to assess than reliability.

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
  • Sorry I should clarify - for Q1, the scales all represent compliance to the same intervention which has multiple components, but they are measured on different frequencies/scales. Also, thank you for correcting me, I should have said reliability in Q3 and not validity. Are there any other tests of reliability is what I meant to ask. Thanks for your help! – oncearunner Jul 24 '13 at 10:52
  • Cronbach's alpha is pretty standard for reliability, but you might also want to check each item for e.g. all the responses (or nearly all) being the same (this will lower correlations and indicate an item that can be deleted. – Peter Flom Jul 24 '13 at 11:10
  • Thanks. I am getting an acceptable alpha (0.79) so hopefully that will suffice. Any thoughts on summing different scales if they represent the same construct? – oncearunner Jul 24 '13 at 11:24
  • There is no reason you can't sum different scales if they represent the same construct, you just have to go through the same steps. The scales are all made up of items; suppose all the items had been on one scale to start with? – Peter Flom Jul 24 '13 at 11:26
  • 1
    People interested in the IRT vs. sum score issue may have a look at the literature, e.g., [Culpepper (2013)](http://apm.sagepub.com/content/37/3/201) – hplieninger Jul 24 '13 at 11:28