I have four different groups of analysts, each of which was given a random sample from a pool of possible images to classify in an online double-blind study. These analyst groups were created after the blind study based on their answers on a set of questions using PCA and K-medians - the groups are supposed to reflect a level of 'expertise'. However, as each analyst was given a random sample, I want to be sure that the group-wise comparisons of variables like analyst accuracy and confidence aren't impacted by the different samples each analyst was shown. I have done group-wise Kruskal-Wallis H Test and Dunn's test with Bonferroni correction on the counts of different images presented to each analyst and this resulted in a significant difference between certain groups. In other words, since analysts in different groups were shown significantly different samples, are my group-wise comparisons valid? Should I consider Generalized Linear Mixed Model or some other statistical method that takes the differences in samples into account? Can you suggest a reasonable strategy on how to go about this (in R or Python)?
Asked
Active
Viewed 5 times
