0

I have data from a study in which subjects were listening to a musical piece and asked to press a key at certain moments. The time axis has been segmented into adjacent epochs of time (of varying durations) that have, for music analysis purposes, been labelled as type A, B or C. Adjacent epochs are not necessarily of the same type, and there are more type-B and type-C epochs than there are of type-A.

For each subject, I computed a keypress count within each epoch in the piece, and thus obtained cross-subject means for each epoch, several ones for each epoch type (A,B,C).

I would like to do a statistical test between these group means to check for a significant difference between A-B-C, but the problem I see is there being an unequal number of elements (epochs) in each of the 3 groups. I think a repeated-measures ANOVA is therefore probably unsuitable.

I also think based on this past CV post that the unequal-number-of-elements-in-each-group problem can be overcome by computing pair-wise differences when such pairs can be defined, and then running paired-samples t-test on the difference (A vs B, B vs C, A vs C). However, I am not sure this data is truly "paired" since the data points refer to epochs from various parts of the piece; or even really "repeated measures" in the traditional sense!

Does it even make sense to run a statistical test in this case, and if so, which test is appropriate?

z8080
  • 1,598
  • 1
  • 19
  • 38
  • How many subjects do you have and how many trials did each subject complete? All subjects were listening to the same musical piece and A, B, and C where the same for all subjects, right? Sharing some (sample if you cannot share all of your) data would facilitate getting an answer. – vkehayas Jul 01 '18 at 21:52
  • There are 32 subjects, who listened to the same piece; epoch types were defined based on absolute positioning within the piece, e.g. 0:12 to 1:25; and are thus the same for all subjects. Note that the observations that I would like to compare category-wise are at epoch level. – z8080 Jul 02 '18 at 11:41
  • OK, I see. So you mean that each subject heard one song with multiple A, B, and C epochs. Can you share some (toy) data? – vkehayas Jul 02 '18 at 14:51
  • Thanks for your help, I've created a spreadsheet with the data [here](http://www.filedropper.com/data_12). Just to reiterate that my question is really rather conceptual, and the data structure uploaded is very simple indeed: for each epoch type, I have a different number of scores (6 "A" scores, 13 "B" scores, 19 "C" scores), and the question is, does it make sense to claim a significant difference between the three means, when there are unequal numbers of data points contributing to each. Again, each score is computed across subjects, and each subject contributes to each of the 3 epoch types – z8080 Jul 03 '18 at 15:25
  • The reason I keep asking for the data is that I am afraid the description is not clear enough (at least for me) and the simplest way to understand the problem would be to look at your data. Unfortunately, the data you shared do not have all the necessary information. I was expecting to see a [tall table](https://en.wikipedia.org/wiki/Wide_and_narrow_data) with key variables that are missing, such as time and subject. From what I see in your linked data, am I to guess that each epoch can be A, B and C at the same time? Otherwise what does a row designate? Are these data from a single subject? – vkehayas Jul 04 '18 at 08:24
  • From what I have understood so far, I think a Poisson mixed model would be an option, but I am not sure yet. – vkehayas Jul 04 '18 at 08:26
  • I see now that the data you shared are most likely averages over subjects. Do you have any reason to suspect that the order of epochs matters? For example, is it more (or less) likely for an A epoch type to be followed by a B epoch type? If so, your model would need to include some information about the order or timing of the epochs. Unless you music piece is completely non-stationary, I suspect that the order matters. What is certain is that any statistical analysis should not be done on the averages you provided, but preferably on the raw observations. – vkehayas Jul 04 '18 at 09:56
  • The cells in the table I shared refer to epochs, and are indeed averages over subjects, as I in fact mentioned in my last comment. I'm not sure if sharing more complete data (what values went into each epoch average) would necessarily help, since the question really is about something else. Also, in my attempt to simplify the problem, I may have mislead readers in the OP when I said the epochs were of fixed duration. In fact, consecutive chunks of time (representing an epoch of one type or another) could be of any duration. There is no prior expectation about the order in which epochs appear. – z8080 Jul 04 '18 at 10:58
  • I know it's a bit convoluted, but I hope it makes sense now. I can share all the subject-wise values for each epoch, if you think it helps. I definitely thought that any statistical test would just be done on the epoch-type group level. – z8080 Jul 04 '18 at 11:00

1 Answers1

1

Based on your clarifications in the comments, I think you should proceed with a Poisson mixed-effects model, a special case of a generalized linear mixed-effects model (GLMM).

If clicks is the number of clicks a subject, subject, performed on a given epoch, epoch, from the set {A, B, C}, then you could start with a model like this one (I am using the R package lme4):

library(lme4)
glmm = glmer(clicks ~ 1 + epoch + (1|subject), family = 'Poisson')

Since your response variable is counts of a process, number of clicks, then the Poisson distribution can be used as a model. Models based on distributions such as the Normal or $t$, are unlikely to provide a good fit in this case.

This approach can deal with unequal samples between epochs and takes into account the variability across subjects through the random effects term (1|subject). This is the equivalent of performing a paired $t$-test or a repeated-measures ANOVA. The reason you need to include such a term is that variability between subjects is expected to exist and you expect responses of a certain subject to be correlated. If you perform the analysis on subject averages or ignore the subject identification by removing the random effects term, you are throwing away existing information in your data that may or may not change the results and their interpretation.

Models such as this one are called mixed-effects, since they combine both fixed effects, (epoch in this case), for which we want to get an estimate of their effect on the response variable, and random effects, for which we assume that they have no overall influence on the response but contribute towards its variance. You will find plenty of posts in this site for more information on and specifically models.

Without knowing more details about your exact experimental conditions, I would advise to also have a look at whether the sequence of epochs and their duration are important by including them as variables in the model.

vkehayas
  • 701
  • 5
  • 13
  • Thanks!! Will study your answer in more detail tomorrow, for now I awarded the bounty as it would otherwise soon expire. Thanks again.. – z8080 Jul 04 '18 at 20:54