0

I have some data where participants watched some clips and evaluated them. All participants watched all the clips (within-subjects experiment). There are 3 categories of clips: positive, neutral and negative. Each category consists of 6 different clips. For the evaluation, the participants selected a continuous value between 0 and 1 for each clip. Also, they replied to a question on a 5-point Likert scale (Strongly disagree to Strongly agree).

I would like to see:

  1. If the continuous evaluations given by the participants differ significantly among the 3 categories of clips. And also if the difference is following the levels of the categories, eg. evaluations for negative are lower than neutral and positive, and so on.

  2. If the Likert replies change significantly with respect to the trial number. I have 18 trials/clips in total for each participant; does the Likert score increase or decrease significantly as the trials go by?

  3. If the Likert replies differ significantly between the 3 categories of clips.

I have 22 participants in total.

estraven
  • 75
  • 6
  • So what have you done so far? This looks like a problem for a GLM. Are you familiar with linear models? – user2974951 Jul 24 '19 at 11:57
  • I am a bit familiar with simple regression, but I am puzzled with the fact that for my independent variable for (1) which is categorical there are multiple observations for each participant. For each participant there are 6 items in IV level 'positive', 6 in 'neutral' and 6 in 'negative'. Every item is unique but there are many similarities since all the clips show the same actor gesturing, so there might be also some correlation because of this? I think I somehow need to compensate for that right? – estraven Jul 27 '19 at 10:19
  • My best guess for (1) is a Mixed Linear Model: valence_score ~ valence_category + (1|subject). For (2) likert ~ trial + (1|subject). For (3) likert ~ valence_category + (1|subject). For (2) and (3) maybe I need Generalized Linear Model since my response is ordinal? – estraven Jul 28 '19 at 12:17

1 Answers1

0

So in total every participant watched 18 clips, 6 of each kind. Every participant evaluated each clip with a continuous variable with values between 0 and 1, and also a likert scale variable from 1 to 5.

First you need to understand that neither of these variables are really normally distributed (probably), so you may need a more generalized linear model, a logistic regression model can be used for values between 0 and 1 (modeling probabilities), and an ordinal regression model for the likert scale variable.

As for the formulas, you guessed right

(1) valence_score ~ valence_category + trial + (1|subject)
So here you would estimate the effect of both the categories and the trials, while accounting for the correlation among the subjects.

(2) and (3) likert ~ valence_category + trial + (1|subject)
Note that in both cases you estimate all the effects simultaneously, not individually (with multiple models). This will avoid wrong conclusions due to confounders, correlations and such.

user2974951
  • 5,700
  • 2
  • 14
  • 27
  • Thank you very much. Would it be possible to advise me on which R functions I can use? I tried to figure it out but I got seriously confused. Seems hard to find GLM functions that take the mixed effects into consideration. I checked for instance polr() for (2) and (3). I also checked glmer with binomial family for (1) but I am not sure how to deal with it since there no p-values. I apologize in advance if asking these extended questions is not appropriate here. Please let me know if that is so, and I will create a new question. – estraven Jul 30 '19 at 13:12
  • @estraven For the logistic regression use the `glmer` function in the `lme4` package, see mkt's answer in this post for guidance https://stats.stackexchange.com/questions/418584/what-kind-of-curve-or-model-should-i-fit-to-my-percentage-data. For ordinal data see Ben's answer in this post https://stats.stackexchange.com/questions/238581/how-to-use-ordinal-logistic-regression-with-random-effects – user2974951 Jul 30 '19 at 13:17
  • Regarding `glmer` for (1), I only found an example with `glm` in mkt's answer, and it seems that it doesn't take into account random effects. I tried `glmer` with family=binomial, which defaults in a logit link function. I get a warning _In eval(family$initialize, rho) : non-integer #successes in a binomial glm!_. Also, I wonder after I read https://stats.stackexchange.com/questions/233366/how-to-fit-a-mixed-model-with-response-variable-between-0-and-1, whether it would be a good idea for my design to use `glmmTMB` with family=beta. – estraven Aug 04 '19 at 15:47
  • @estraven have a search on this site for what that error means https://stats.stackexchange.com/search?q=glmer+non-integer+%23successes+in+a+binomial+glm. As for beta regression, you can use it in your case, however beta only accepts values in (0,1), so you will have to transform your response, or do some other workaround for the values 0 and 1. There are some posts on the matter on this site as well. – user2974951 Aug 04 '19 at 16:30