Daim {Daim}R Documentation

Diagnostic accuracy of classification models.

Description

Estimation of prediction error based on cross-validation (CV) or various bootstrap techniques.

Usage

Daim(formula, model=NULL, data=NULL, control = Daim.control(),                  
                thres = seq(0,1,by=0.01), cutoff = 0.5, 
                labpos = "1", returnSample = FALSE,
                cluster = NULL, seed.cluster = NULL, ...)

Arguments

formula formula of the form y ~ x1 + x2 + ..., where y must be a factor and x1,x2,... are numeric or factor.
model function. Modelling technique whose error rate is to be estimated. The function model returns the predicted probability for each observation.
data an optional data frame containing the variables in the model (training data).
control See Daim.control.
thres a numeric vector with the cutoff values.
cutoff the cutoff value for error estimation. This can be a numeric value or a character string. If the cutoff set to:
".632" - the estimated cut-point corresponding the .632 estimation of the sensitivity and the specificity.
".632+" - the estimated cut-point corresponding the .632+ estimation of the sensitivity and the specificity.
labpos a character string of the response variable that defines a "positive" event. The labels of the "positive" events will be set to "pos" and other to "neg".
returnSample a logical value for saving the data from each sample.
cluster the name of the cluster, if parallel computing will be used.
seed.cluster an integer value used as seed for the RNG.
... additional parameters.

Value

a list with the following components :

call the matched call.
formula the formula supplied.
method the list of control parameters.
err632p the .632+ estimation of the misclassification error.
err632 the .632 estimation of the misclassification error.
errloob the LOOB estimation of the misclassification error.
errapp the apparent error.
sens632p the .632+ estimation of the sensitivity.
spec632p the .632+ estimation of the specificity.
sens632 the .632 estimation of the sensitivity.
spec632 the .632 estimation of the specificity.
sensloob the LOOB estimation of the sensitivity.
specloob the LOOB estimation of the specificity.
sensapp the apparent sensitivity.
specapp the apparent specificity.
roc a data frame with estimated values of sensitivity and specificity for a variety of cutoffs.
sample.roc a list in which each entry contains the values of the ROC curve of this special sample or cross-validation run.
sample.data a data frame with the results of this particular sample or cross-validation run.

References

Werner Adler and Berthold Lausen (2009).
Bootstrap Estimated True and False Positive Rates and ROC Curve.
Computational Statistics & Data Analysis, 53, (3), 718–729.

Tom Fawcett (2006).
An introduction to ROC analysis.
Pattern Recognition Letters, 27, (8).

Bradley Efron and Robert Tibshirani (1997).
Improvements on cross-validation: The.632+ bootstrap method.
Journal of the American Statistical Association, 92, (438), 548–560.

See Also

plot.Daim, performDaim, auc.Daim, roc.area.Daim

Examples


  library(ipred)
  data(GlaucomaM)
  head(GlaucomaM)

  mylda <- function(formula,train,test){
        model <- lda(formula,train)
        predict(model,test)$posterior[,"pos"]
  }

  ACC <- Daim(Class~.,model=mylda,data=GlaucomaM,labpos="glaucoma")
  ACC
  summary(ACC)

  ####
  #### for parallel computing with snow cluster
  ####
 
  # library(snow)
  ### 
  ### create cluster with two slave nodes

  # cl <- makeCluster(2)

  ###
  ### Load used library on all slaves and execute the Daim in parallel
  ###

  # clusterEvalQ(cl, library(ipred))
  # ACC <- Daim(Class~.,model=mylda,data=GlaucomaM,labpos="glaucoma",cluster=cl)
  # ACC

  ####
  #### for parallel computing with multicore package 
  #### you need only to load this library
  ####

  # library(multicore)
  # ACC <- Daim(Class~.,model=mylda,data=GlaucomaM,labpos="glaucoma")
  # ACC

  library(randomForest)

  myRF <- function(formula,train,test){
        model <- randomForest(formula,train)
        predict(model,test,type="prob")[,"pos"]
  }

  ACC2 <- Daim(Class~.,model=myRF,data=GlaucomaM,labpos="glaucoma",
        control=Daim.control(number=25))
  ACC2
  summary(ACC2)

  ####
  #### for parallel computing with snow cluster
  ####

  # library(snow)
  ### 
  ### create cluster with two slave nodes

  # cl <- makeCluster(2)

  ###
  ### Load used library on all slaves and execute the Daim in parallel
  ###

  # clusterEvalQ(cl, library(randomForest))
  # ACC2 <- Daim(Class~.,model=myRF,data=GlaucomaM,labpos="glaucoma",cluster=cl)
  # ACC2

  ####
  #### for parallel computing with multicore package
  ####

  # library(multicore)
  # ACC2 <- Daim(Class~.,model=myRF,data=GlaucomaM,labpos="glaucoma")
  # ACC2


[Package Daim version 1.0.0 Index]