0

I hope this message finds everyone safe and well.

I want to estimate the Rasch item difficulty parameters for my test items (dataset below). However, I have two challenges:

(1) Item scores are continuous between 0 and 1 (item_score)

(2) Test is adaptive and thus, items (item_id) vary across persons (person_id)

Is there an R package to take (1) and (2) into account (I highly appreciate a demonstration)?

--Many thanks

#===== Dataset in R
dat <- read.csv('https://raw.githubusercontent.com/izeh/n/master/g.csv', stringsAsFactors = F)
user7148318
  • 9
  • 4
  • Does this https://stats.stackexchange.com/questions/44515/how-is-item-response-theory-irt-called-for-continuous-response?rq=1 help? – mdewey May 02 '20 at 15:14
  • @mdewey, not really I have stated specific challenges that require a different answer. – user7148318 May 02 '20 at 15:40

1 Answers1

0

Edit: This answer is not correct. It's not clear to me what OP is looking for, but take a look at the discussion at https://chat.stackexchange.com/rooms/107527/discussion-between-jeremy-miles-and-user7148318 before answering.

There is no package that can do this (as far as I know).

The lavaan package for sem does not give standard errors for latent variable estimates (and I don't know you can predict a model with new data).

You might be able to do this with merTools::predictInterval(), however I'm having trouble understanding your data. What identifies the question?

You see to have very large number of questions (~ 16000?) - each of which is not answered by very many people, so your standard errors will be high. Have I understood that correctly?

My guess is that the will look like:

library(dplyr)
library(lme4)
library(merTools)
dat <- read.csv('https://raw.githubusercontent.com/izeh/n/master/g.csv', stringsAsFactors = F)

dat$item_id <- factor(item_id)

dat_no_1 <- dat[dat$person_id != 1, ]
dat_no_1 <- dat[dat$person_id > 1, ]

fit1 <- lme4::lmer(item_score ~ as.factor(item_id) +
                     (1 | person_id),
                   data = dat_no_1)
system.time(
  pred1 <-
    merTools::predictInterval(fit1,
                              newdata = dat_no_1,
                              n.sims = 999)
)

I can't test this though, as I run out of memory.

Note that this ignores the non-normality issue.

Jeremy Miles
  • 13,917
  • 6
  • 30
  • 64
  • First of all, thank you! so, the test is computer adaptive. It assigns items to test takers based on their response (i.e., if correct makes the next one harder, if incorrect make the next one easier). The `item_id` identifies the specific question targeted by each test taker. Each test for each person consists of 5 different `item_type`s. Speaking of `lavaan`, it seems [**this page**](https://cengiz.me/posts/crm-stan/) talks about a transformation on the responses and than using `lavaan`, do you think it makes sense for this data? BTW, what are the item difficulty parameters in your code? – user7148318 May 02 '20 at 20:49
  • Item difficulty parameters are the intercepts. The code i wrote is the first step - you get the item parameters (these are stored in fit1) then you predict the CI of person 1. If the CI is sufficienlty small, you can stop. If not, give them another item - so you'd need to wrap this in some sort of loop. The system.time() part is the time it takes to score the individual, I don't know how long this would take - if it's too long, presumably it's not feasible. – Jeremy Miles May 02 '20 at 21:01
  • Do you really have 16k items? – Jeremy Miles May 02 '20 at 21:02
  • !! OK, that's a lot. You definitely can't use lavaan for this. Can you run the code I put in the answer? (I can't on my current machine.) – Jeremy Miles May 02 '20 at 21:17
  • You can't test the cat on any person who has taken an item that no other person has taken. – Jeremy Miles May 02 '20 at 21:18
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/107527/discussion-between-jeremy-miles-and-user7148318). – Jeremy Miles May 02 '20 at 21:21