Computation of composite score or summed score from Likert scale

Question

I am proposing a survey for my research proposal in speech therapy. There is a set of guidelines that speech therapists are supposed to follow, so that all questions are conceptually related - so that I can used a composite score if I understood that right.

My research question is Are speech therapists influenced by personal attributes in their choice of assessment? Hence I want to do a correlation between if somebody is bilingual and is then more likely to follow the guidelines (using the composite score/summary score).

I hope somebody can help.

Please indicate how often you use the following assessment procedure.

1 Never     2 Occasionally     3 Sometimes     4 Usually     5 Always


I assess all languages of the patient.                               1 2 3 4 5 
I use a language usage profile                                       1 2 3 4 5

I use the following individuals as interpreters during assessment:
... family and friends of the patient                                1 2 3 4 5
... bilingual coworkers                                              1 2 3 4 5 
... professional interpreters                                        1 2 3 4 5 

It is encouraged to use either bilingual coworkers or professional interpreters (no difference in preference) but family and friends should not be used hence I would not be able to use family and friends as a positive score (never would be good so score high) and I thought I could just average the score of the three items as I don't want the professional usage be weighted higher in the composite score than using the language profile and assessing all languages.

I was wondering if I would simply add up all the scores like this: (for example somebody answered like this)

assess all languages                                               (4) 
use language usage profile                                         (4)
use appropriate individuals                                        (4) 
 (e.g., never use family and friends             (5 as positive) 
      + frequently use bilingual coworkers       (2)
      + frequently use professional interpreters (5) 
      = 4)  

Which then as the composite score would be 4 + 4 + 4 = 12 out of a possible 15.

Then I was going to use Spearman's correlation as the question on bilingualism leads to a dichotomous variable and the composite score would be interval/ratio data?

Answer 1

I also wonder what exact questions you have. I am guessing that you are wondering several things:

1. Does your questionnaire (& associated scoring scheme) seem reasonable?
2. Can you treat the score derived from the questionnaire as interval/ratio data?
3. Is Spearman's correlation an appropriate test / measure of effect size?

If these are not your real questions, leave me a comment, or edit the question, and I can try to update this answer so that it would be more helpful. I will answer each possible question in turn.

Unfortunately, I have no substantive knowledge of your subject area, and not too much about psychometrics, either. One thing to think about when making a questionnaire is whether all questions are actually getting at the same underlying issue. I'm guessing your goal is to tap different aspects of whether therapists are aware of, and employ, correct procedures for working with bilingual clients. To the extent that all items (i.e., questions) are measuring that, combining them is quite reasonable. This can be checked with Factor Analysis, although I don't know if that's within your skill set. Another possibility is to make several different 2-way cross-tabulation tables, and see if most of the counts lie on or near the diagonal. This would mean that therapists who give a higher rating in response to one question also give a higher response to another. Spearman's correlations could serve this purpose as well; again, you are looking for strong, positive correlations. Finding that they all correlate with each other suggests treating them as 1 construct (i.e., combining them) is a reasonable thing to do. Others on CV have more expertise with psychometrics, perhaps they will weigh in with better advice.

Regarding the scoring scheme, I think it makes perfect sense to combine the last three questions first, so that that issue doesn't outweigh the others. One suggestion I would make is to take the average of the scores. In other words, divide the final sum by 3 (giving a final score of 4 for your example; n.b., this means averaging twice or you would lose the downweighting of the last 3 items). This shouldn't have any substantive effect, but it does put the score back in the original scale, which can make it clearer and more interpretable.

Whether or not Likert items (i.e., questions) and Likert scales (i.e., questionnaires) can be treated as interval data has long been a controversial issue (they are most certainly not ratio). As I understand it, there are a few principles: a) items with more response levels are likely to be closer to equal-interval; b) combining multiple items into a scale makes it likelier to be equal-interval; c) using numbers instead of just words primes respondents to think of the intervals as equal and respond accordingly; and (d) whether or not the values are interval is mostly a matter of theoretical belief, not something that can be tested and determined in practice. My guess would be that your scale may be interval enough for your purposes. Ultimately, it's still the case that using ordinal logistic regression is ideal, but I don't know if you have that in your toolbox. At least two different questions on CV have treated issues pertaining to Likert scales and may be worth your time.

I'm not sure that I would use Spearman's correlation for your main analysis. I suspect you have two groups: monolingual therapists, and bilingual therapists. (If you have some therapists who speak more than 2 languages, you would have to decide what to do, but if you just have one or two, I would just make is a >1 group.) This suggests something like a t-test, or Mann-Whitney U-test if possible (which I believe is equivalent to ordinal logistic regression when there are only two groups), makes the most sense. (If you'll be using SPSS, this tutorial may help with the U-test.)

To assess the magnitude of the effect, I would suggest you use a standardized mean difference. This is often referred to as 'Cohen's d'. It is simply the difference between the two means, divided by the pooled standard deviation:
$$ d=\frac{\bar{x}_2-\bar{x}_1}{SD_{pooled}} $$
d gives you a measure of how far apart the two means are in units of the population's standard deviation.