modelChoose {irtProb} | R Documentation |
Functions to help to choose the best models from the m4pl person response models family.
The function meanModels
gives the mean of selected and available statistics
from a m4plModelShow
result. The function modelChoose
shows which
model(s) are choosen for each subject, while the function modelChooseAdd
adds a supplementary logical variable indicating which model(s) are choosen for each subject
to the data.frame returned previously by a m4plModelShow
call .
meanModels( modelShow, statistics=c("T","S")) modelChoose( modelShow, criteria = "BIC", tol = 0.2) modelChooseAdd(modelShow, criteria="LL")
modelShow |
data.frame: result returned by a m4plModelShow call. |
criteria |
character: criteria used to choose between models (LL, AIC or BIC). |
tol |
numeric: tolerance around the choose criteria, so that more models can be considered. |
statistics |
character: a vector of variables for which means of statistics
from m4plModelShow will be computed. |
A tolerance varying between 0.10 and 0.20 is suggested. Lower the value, less models are choosen.
meanModels |
data.frame: this function return a data.frame of means for each choosen variables. |
modelChoose |
list: return a list of the models(s) choosen for each subject. |
modelChooseAdd |
data.frame: return a data.frame adding a logical indicator of the choosen model(s). |
Gilles Raiche, Universite du Quebec a Montreal (UQAM),
Departement d'education et pedagogie
Raiche.Gilles@uqam.ca, http://www.er.uqam.ca/nobel/r17165/
Blais, J.-G., Raiche, G. and Magis, D. (2009). La detection des patrons de reponses problematiques dans le contexte des tests informatises. In Blais, J.-G. (Ed.): Evaluation des apprentissages et technologies de l'information et de la communication : enjeux, applications et modeles de mesure. Ste-Foy, Quebec: Presses de l'Universite Laval.
Raiche, G., Magis, D. and Beland, S. (2009). La correction du resultat d'un etudiant en presence de tentatives de fraudes. Communication presentee a l'Universite du Quebec a Montreal. Retrieved from http://www.camri.uqam.ca/camri/camriBase/
Raiche, G., Magis, D. and Blais, J.-G. (2008). Multidimensional item response theory models integrating additional inattention, pseudo-guessing, and discrimination person parameters. Communication at the annual international Psychometric Society meeting, Durham, New Hamshire. Retrieved from http://www.camri.uqam.ca/camri/camriBase/
## GENERATION OF VECTORS OF RESPONSE # NOTE THE USUAL PARAMETRIZATION OF THE ITEM DISCRIMINATION, # THE VALUE OF THE PERSONNAL FLUCTUATION FIXED AT 0, # AND THE VALUE OF THE PERSONNAL PSEUDO-GUESSING FIXED AT 0.30. # IT COULD BE TYPICAL OF PLAGIARISM BEHAVIOR. nItems <- 40 a <- rep(1.702,nItems); b <- seq(-5,5,length=nItems) c <- rep(0,nItems); d <- rep(1,nItems) nSubjects <- 1; rep <- 100 theta <- seq(-1,-1,length=nSubjects) S <- runif(n=nSubjects,min=0.0,max=0.0) C <- runif(n=nSubjects,min=0.3,max=0.3) D <- runif(n=nSubjects,min=0,max=0) set.seed(seed = 100) X <- ggrm4pl(n=nItems, rep=rep, theta=theta, S=S, C=C, D=D, s=1/a, b=b,c=c,d=d) ## Results for each subjects for each models essai <- m4plModelShow(X, b=b, s=1/a, c=c, d=d, m=0, prior="uniform") ## Means of choosen variables from m4plModleShow previous call round(meanModels(essai, statistic=c("LL","BIC","T","C")), 2) ## Which model(s) are the best for each subject (LL criteria) res <- modelChoose(essai, criteria = "LL", tol = 0.2); res ## Which model(s) are the best for each of the first 20 subjects (BIC criteria) res <- modelChoose(essai[which(essai$ID < 20) ,], criteria="BIC"); res ## A look at the 15th subject parameters estimation for each models essai[which(essai$ID == 15) ,] ## Add the logical critLL variable to the data frame criteria <- "LL" res1 <- modelChooseAdd(essai, criteria=criteria) # Create a charcater string composed from "crit" and criteria crit <- paste("crit",criteria,sep="") # To Show only the lines where the models is choosen according to critLL res2 <- res1[which(res1[crit] == TRUE),]; mean(res2$T);sd(res2$T,na.rm=TRUE);res2 # Give the mean for choosen variables from m4plModelShow meanModels(essai, statistic=c("LL","BIC","T","SeT","S","C","D")) # Tabulate only the choosen models table(res2$MODEL) # Show the information for the subject for which the model TSCD was choosen res2[which(res2$MODEL == "TSCD") ,] # Show only the results for the 5th subject, the with the TSCD model choosen res1[which(res1$ID == 5) ,] # Same, but without critLL essai[which(essai$ID == 5) ,] ## Simulation whith cheating # High proficiency students responding at random to 20 # easiest ones) XHProficiency <- X pourcHasard <- 0.20; nHasard <- abs(dim(X)[2]*pourcHasard) XHProficiency[,1:nHasard] <- rbinom(dim(X)[1]*nHasard, 1, 0.5) XHProficiency <- m4plModelShow(XHProficiency, b=b, s=1/a, c=c, d=d, m=0, prior="uniform") XHProficiency <- modelChooseAdd(XHProficiency, criteria=criteria) XHProficiency <- XHProficiency[which(XHProficiency$critLL == TRUE),] mean(XHProficiency$T);sd(XHProficiency$T,na.rm=TRUE);XHProficiency[1:10,] meanModels(XHProficiency, statistic=c("LL","BIC","T","SeT","S","C","D")) table(XHProficiency$MODEL) # Low proficiency students responding at random to 20 # (more difficult ones) XLProficiency <- X pourcHasard <- 0.20 nHasard <- abs(dim(X)[2]*pourcHasard) XLProficiency[,(nItems-nHasard+1):nItems] <- rbinom(dim(X)[1]*nHasard, 1, 0.5) XLProficiency <- m4plModelShow(XLProficiency, b=b, s=1/a, c=c, d=d, m=0, prior="uniform") XLProficiency <- modelChooseAdd(XLProficiency, criteria=criteria) XLProficiency <- XLProficiency[which(XLProficiency$critLL == TRUE),] mean(XLProficiency$T);sd(XLProficiency$T,na.rm=TRUE);XLProficiency[1:10,] meanModels(XLProficiency, statistic=c("LL","BIC","T","SeT","S","C","D")) table(XLProficiency$MODEL) # High proficiency students giving incorrect responses to 20 # (easiest ones) XHProficiency <- X pourcCheat <- 0.20; nCheat <- abs(dim(X)[2]*pourcCheat) XHProficiency[,1:nCheat] <- rep(0, dim(X)[1]*dim(X)[2]*pourcCheat) XHProficiency <- m4plModelShow(XHProficiency, b=b, s=1/a, c=c, d=d, m=0, prior="uniform") XHProficiency <- modelChooseAdd(XHProficiency, criteria=criteria) XHProficiency <- XHProficiency[which(XHProficiency$critLL == TRUE),] mean(XHProficiency$T);sd(XHProficiency$T,na.rm=TRUE);XHProficiency[1:10,] meanModels(XHProficiency, statistic=c("LL","BIC","T","SeT","S","C","D")) table(XHProficiency$MODEL) # Low proficiency students giving correct responses to 20 # (more difficult ones) XLProficiency <- X pourcCheat <- 0.20 nCheat <- abs(dim(X)[2]*pourcCheat); XLProficiency[,(nItems-nCheat+1):nItems] <- rep(1, dim(X)[1]*dim(X)[2]*pourcCheat) XLProficiency <- m4plModelShow(XLProficiency, b=b, s=1/a, c=c, d=d, m=0, prior="uniform") XLProficiency <- modelChooseAdd(XLProficiency, criteria=criteria) XLProficiency <- XLProficiency[which(XLProficiency$critLL == TRUE),] mean(XLProficiency$T);sd(XLProficiency$T,na.rm=TRUE);XLProficiency[1:10,] meanModels(XLProficiency, statistic=c("LL","BIC","T","SeT","S","C","D")) table(XLProficiency$MODEL)