trainNWS {caretNWS} | R Documentation |
This function sets up a grid of tuning parameters for a number of classification and regression routines, fits each model and calculates a resampling based performance measure.
trainNWS(x, y, method = "rf", ..., metric = ifelse(is.factor(y), "Accuracy", "RMSE"), maximize = ifelse(metric == "RMSE", FALSE, TRUE), sleighObj = NULL, trControl = trainNWSControl(), tuneGrid = NULL, tuneLength = 3)
x |
a data frame containing training data where samples are in rows and features are in columns. |
y |
a numeric or factor vector containing the outcome for each sample. |
method |
a string specifying which classification or regression model to use. Possible values are: lm , rda , lda , gbm , rf , nnet , multinom , gpls , lvq , rpart , knn , pls , pam , nb , earth , treebag ,
svmpoly , svmradial , fda , bagEarth , bagFDA , glmboost , gamboost , blackboost , ada , ctree , cforest , sddaLDA , sddaQDA , LogitBoost , J48 , M5Rules , slda , superpc , enet and lasso . See the Details section below. |
... |
arguments passed to the classification or regression routine (such as randomForest ). Errors will occur if values
for tuning parameters are passed here. |
metric |
a string that specifies what summary metric will be used to select the optimal model. Possible values are "RMSE" and "Rsquared" for regression and "Accuracy" and "Kappa" for classification.(NOTE: If given, this argument must be named.) |
maximize |
a logical: should the metric be maximized or minimized? |
sleighObj |
a sleigh object for an existing nws session. This object is not destroyed after trainNWS closes. |
trControl |
a list of values that define how this function acts. See trainNWSControl . (NOTE: If given, this argument must be named.) |
tuneGrid |
a data frame with possible tuning values. The columns are named the same as the tuning parameters in each
method preceded by a period (e.g. .decay, .lambda). See the function createGrid in this package for more details.
(NOTE: If given, this argument must be named.) |
tuneLength |
an integer denoting the number of levels for each tuning parameters that should be
generated by createGrid . (NOTE: If given, this argument must be named.) |
trainNWS
can be used to tune models by picking the complexity parameters that are associated with the optimal resampling statistics. For particular model, a grid of parameters (if any) is created and the model is trained on slightly different data for each candidate combination of tuning parameters. Across each data set, the performance of held-out samples is calculated and the mean and standard deviation is summarized for each combination. The combination with the optimal resampling statistic is chosen as the final model and the entire training set is used to fit a final model.
Currently, the trainNWS
function does not support model specification via a formula. It assumes that all of the predictors are numeric (perhaps generated by model.matrix
).
A variety of models are currently available. The table below enumerates the models and the values of the method
argument, as well as the complexity parameters used by trainNWS
.
Model | method Value | Package | Tuning Parameter(s) |
Recursive partitioning | rpart | rpart | maxdepth |
ctree | party | mincriterion |
|
Boosted Trees | gbm | gbm | interaction depth , |
n.trees , shrinkage |
|||
blackboost | mboost | maxdepth , mstop |
|
ada | ada | maxdepth , iter , nu |
|
Boosted regression models | glmboost | mboost | mstop |
gamboost | mboost | mstop |
|
logitboost | caTools | nIter |
|
Random forests | rf | randomForest | mtry |
cforest | party | mtry |
|
Bagged Trees | treebag | ipred | None |
Neural networks | nnet | nnet | decay , size |
Partial least squares | pls | pls, caret | ncomp |
Support Vector Machines (RBF) | svmradial | kernlab | sigma , C |
Support Vector Machines (polynomial) | svmpoly | kernlab | scale , degree , C |
Relevance Vector Machines (RBF) | rvmradial | kernlab | sigma |
Relevance Vector Machines (polynomial) | rvmpoly | kernlab | scale , degree |
Least Squares Support Vector Machines (RBF) | lssvmradial | kernlab | sigma |
Gaussian Processes (RBF) | guassprRadial | kernlab | sigma |
Gaussian Processes (polynomial) | guassprPoly | kernlab | scale , degree |
Linear least squares | lm | stats | None |
Multivariate adaptive regression splines | earth | earth | degree , nprune |
Bagged MARS | bagEarth | caret, earth | degree , nprune |
M5 Rules | M5Rules | RWeka | pruned |
Elastic Net | enet | elasticnet | lambda , fraction |
The Lasso | enet | elasticnet | fraction |
Supervised Principal Components | superpc | superpc | n.components , threshold |
Linear discriminant analysis | lda | MASS | None |
Stabilised Linear discriminant analysis | slda | ipred | None |
Stepwise Diagonal Discriminant Analysis | sddaLDA , sddaQDA | SDDA | None |
Logistic/multinomial regression | multinom | nnet | decay |
Regularized discriminant analysis | rda | klaR | lambda , gamma |
Stabilised Linear Discriminant Analysis | slda | ipred | None |
Flexible discriminant analysis (MARS) | fda | mda, earth | degree , nprune |
Bagged FDA | bagFDA | caret, earth | degree , nprune |
C4.5 decision trees | J48 | RWeka | C |
k nearest neighbors | knn3 | caret | k |
Nearest shrunken centroids | pam | pamr | threshold |
Naive Bayes | nb | klaR | usekernel |
Generalized partial least squares | gpls | gpls | K.prov |
Learned vector quantization | lvq | class | k |
By default, the function createGrid
is used to define the candidate values of the tuning parameters. The user can also specify their own. To do this, a data fame is created with columns for each tuning parameter in the model. The column names must be the same as those listed in the table above with a leading dot. For example, ncomp
would have the column heading .ncomp
. This data frame can then be passed to createGrid
.
In some cases, models may require control arguments. These can be passed via the three dots argument. Note that some models can specify tuning parameters in the control objects. If specified, these values will be superseded by those given in the createGrid
argument.
The vignette entitled "caret Manual – Model Building" in the caret package has more details and examples related to this function.
caretNWS
is a parallel version of the train
function in the caret package. This function implements a combination of sequential and parallel jobs. For example, if a PLS model with 10 candidate values for the number of retained components is requested along with 10-fold cross–validation, the 10 jobs for each caudate tuning parameter are submitted in parallel. Once finished, the process is started again with the second candidate tuning parameter etc.
The function also has a fault-tolerance function where any jobs that are abnormally long (say 20x the longest job to date) can be killed automatically. In this way, a problem with a worker node may not destroy the results to date. For example, if the models are tuned using 25 bootstrap results, the failure of one node will cause the function to tune the model with the remainder and keep going.
The nwsStart
can be dependent on the configuration of the user's system. The example function given in the package may not work out of the box.
A list is returned of class train
containing:
modelType |
an identifier of the model type. |
results |
a data frame the training error rate and values of the tuning parameters. |
call |
the (matched) function call with dots expanded |
dots |
a list containing any ... values passed to the original call |
metric |
a string that specifies what summary metric will be used to select the optimal model. |
trControl |
the list of control parameters. |
finalModel |
an fit object using the best parameters |
trainingData |
a data frame |
resample |
A data frame with columns for each performance
metric. Each row corresponds to each resample. If leave-one-out
cross-validation or out-of-bag estimation methods are requested,
this will be NULL . The returnResamp argument of trainNWSControl |
perfNames |
a character vector of performance metrics that are produced by the summary function |
maximize |
a logical recycled from the function arguments. |
Max Kuhn
train
, createGrid
, createFolds
## Not run: data(iris) TrainData <- iris[,1:4] TrainClasses <- iris[,5] knnFit <- trainNWS(TrainData, TrainClasses, "knn", tuneLength = 10, trControl = trainNWSControl(method = "boot")) ## End(Not run)