evimp {earth}R Documentation

Estimate variable importances in an "earth" object

Description

Estimate variable importances in an earth object

Usage

evimp(obj, trim=TRUE)

Arguments

obj An earth object.
trim If TRUE (default), delete rows in the returned matrix for variables that don't appear in any subsets.

Value

A matrix showing the relative importances of the variables in the model. There is a a row for each variable. The row name is the variable name, but with -unused appended if the variable does not appear in the final model.

The columns of the matrix are:
col: column index of the variable in the x argument to earth.
used: 1 if the variable is used in the final model, else 0. Equivalently, 0 if the row name has a -unused suffix.
nsubsets: variable importance using the "number of subsets" criterion. Is the number of subsets that include the variable (see below).
gcv: variable importance using the GCV criterion (see below).
rss: ditto but for the RSS criterion.

The rows are sorted on the nsubsets criterion. This means that values in the nsubsets column decrease as you go down the column. The values in the gcv and rss columns decrease except where the gcv or rss ranking differs from the nsubsets ranking.

Additionally, there are unnamed columns after the gcv column and the rss column. These have a 0 where the ranking using the gcv or rss criteria differs from that using the nsubsets criterion. Equivalently, there is a 0 for values that do not decrease as you go down the gcv or rss column.

Note

Estimating variable importance

Establishing predictor importance is in general a tricky and even controversial problem. There is no completely reliable way to estimate the importance of the variables in a standard MARS model, unless you make further lengthy tests after the model is built. The evimp function just makes an educated (and in practice useful) guess as described below.

Three criteria for estimating variable importance

The evimp functions uses three criteria for estimating variable importance.

1. The nsubsets criterion counts the number of model subsets that include the variable. Variables that are included in more subsets are considered more important.

By "subsets" we mean the subsets of terms generated by the pruning pass. There is one subset for each model size, and each subset is the best set of terms for that model size. (These subsets are specified by $prune.terms in earth's return value.) Only subsets that are smaller than or equal in size to the final model are used for estimating variable importance.

2. The rss criterion first calculates the decrease in the RSS for each subset relative to the previous subset. (For multiple response models, RSS's are calculated over all responses.) Then for each variable it sums these decreases over all subsets that include the variable. Finally is scales these decreases so the maximum decrease is 100. Variables which cause larger net decreases in the RSS are considered more important.

3. The gcv criterion is the same, but using the GCV instead of the RSS. Adding a variable can sometimes increase the GCV. When this happens, the variable could even have a negative total importance, and thus appear less important than unused variables.

Note that using RSq's and GRSq's instead of RSS's and GCV's would give identical estimates of variable importance.

Example

a <- earth(O3 ~ ., data=ozone1, degree=2)
evimp(a, trim=FALSE)
Yields the following matrix:
              col used nsubsets    gcv      rss
    temp        4    1       10 100.00 1 100.00 1
    humidity    3    1        8  12.68 1  14.78 1
    ibt         7    1        8  12.68 1  14.78 1
    doy         9    1        7  11.26 1  12.93 1
    dpg         6    1        5   6.75 1   7.84 1
    ibh         5    1        4   9.58 0  10.46 0
    vis         8    1        4   4.38 1   5.30 1
    wind        2    1        1   0.74 1   0.98 1
    vh-unused   1    0        0   0.00 1   0.00 1
The rows are sorted on nsubsets. We see that temp is considered the most important variable, followed by humidity, and so on. We see that vh is unused in the final model, and thus is given an unused suffix and a 0 in the used column.

The col column gives the the column indices of the variables in the x argument to earth after factors have been expanded.

The nsubsets column is the number of subsets that included the corresponding variable. For example, temp appears in 10 subsets and humidity in 8.

The gcv and rss columns are scaled so the largest net decrease is 100.

The unnamed columns after the gcv and rss columns have a 0 if the corresponding criterion increases instead of decreasing (i.e. the ranking disagrees with the nsubsets ranking). We see that ibh is considered less important than dpg using the nsubsets criterion, but not with the gcv and rss criteria.

Other techniques

Running plotmo with ylim=NULL (the default) gives an idea of which predictors make the largest changes to the predicted value (but only with all other predictors at their median values).

You can also use drop1 (assuming you are using the formula interface to earth). Calling drop1(my.earth.model) will delete each predictor in turn from your model, rebuild the model from scratch each time, and calculate the GCV each time. You will get warnings that the earth library function extractAIC.earth is returning GCVs instead of AICs — but that is what you want so you can ignore the warnings. (You can turn off just these warnings by passing warn=FALSE to drop1). The column labeled AIC in the printed response from drop1 will actually be a column of GCVs not AICs. The Df column is not much use in this context. Remember that this technique only tells you how important a variable is with the other variables already in the model. It does not tell the effect of a variable in isolation. Note that drop1 drops predictors from the model while earth's pruning pass drops terms.

You will get lots of output from drop1 if you built your original earth model with trace>0. You can set trace=0 by updating your model before calling drop1. Do it like this:
my.model <- update.earth(my.model, trace=0).

Remarks

This function is useful in practice but the following issues can make it misleading.

MARS models have a high variance — if the data changes a little, the set of basis terms created by the forward pass can change a lot. So estimates of predictor importance can be unreliable because they can vary with even slightly different training data.

Colinear (or related) variables can mask each other's importance, just as in linear models. This means that if two predictors are closely related, the forward pass will somewhat arbitrarily choose one over the other. The chosen predictor will incorrectly appear more important.

For interaction terms, each variable gets credit for the entire term — thus interaction terms are counted more than once and get a total higher weighting than additive terms (questionably). Each variable gets equal credit in interaction terms even though one variable in that term may be far more important than the other.

One can question if it is valid to estimate variable importance using model subsets that are not part of the final model. It is even possible for a variable to be rated as important yet not appear in the final model.

An example of conflicting importances (however, the results are fine with the default pmethod):
evimp(earth(mpg~., data=mtcars, pmethod="none"))

Acknowledgment

Thanks to Max Kuhn for the original evimp code and for helpful discussions.

See Also

earth

Examples

data(ozone1)
a <- earth(O3 ~ ., data=ozone1, degree=2)
evimp(a, trim=FALSE)

[Package earth version 2.0-2 Index]