grpreg {grpreg}R Documentation

fit a group penalized regression path

Description

Fit paths for group lasso, group bridge, or group MCP at a grid of values of the regularization parameter lambda. Fits linear and logistic regression models.

Usage

grpreg(X, y, group=1:ncol(X), family="gaussian", penalty="gMCP", lambda,
n.lambda=100, lambda.min=ifelse(n>p,.001,.05), lambda.max, lambda2=.001,
eps=.005, max.iter=1000, delta=1e-8, gamma=.5,
a=ifelse(family=="gaussian",3,30), verbose=FALSE, monitor=NULL,
warn.conv=TRUE)

Arguments

X The design matrix, without an intercept. grpreg standardizes the data and includes an intercept by default.
y The response vector.
group A vector of consecutive integers describing the grouping of the coefficients (see example below).
family Either "gaussian" or "binomial", depending on the response.
penalty The penalty to be applied to the model. One of "gLasso" for group lasso, "gBridge" for group bridge, or "gMCP" for group MCP.
lambda A user supplied sequence of lambda values. Typically, this is left unspecified, and the function automatically computes a vector of lambda values that ranges uniformly on the log scale from lambda.min to lambda.max.
n.lambda The number of lambda values. Default is 100.
lambda.min The smallest value for lambda, as a fraction of lambda.max. Default is .001 if the number of observations is larger than the number of covariates and .05 otherwise.
lambda.max The largest value for lambda. Default is to use the smallest value for which all penalized coefficients are 0. This can be calculated for group lasso and group MCP, but not for group bridge. If lambda.max is left unspecified for group bridge, grpreg will make a rough guess.
lambda2 By default, a small L2 penalty is included alongside the group penalty. lamdba2 controls the magnitude of this penalty, as a fraction of lambda.
eps Convergence threshhold. The algorithm iterates until the relative change in any coefficient is less than eps. Default is .005. See details.
max.iter Maximum number of iterations. Default is 1000. See details.
delta The amount by which the group lasso penalty is bounded away from 0. Smaller values of delta produce more accurate results, but converge more slowly. See example.
gamma Tuning parameter of the bridge penalty; defaults to 1/2
a Tuning parameter of the group MCP penalty - defaults to 3 for linear regression and 30 for logistic regression;
warn.conv Should the function give a warning if it fails to converge? Default is TRUE. See details.
verbose Get a running update on what the algorithm is doing. Default is FALSE.
monitor Monitor the iterations of a vector of covariates. If set to a numeric vector, for example c(3,5), the algorithm will display iterates of the third and fifth covariates as it progresses.

Details

The sequence of models indexed by lambda is fit using a locally approximated coordinate descent algorithm. For logistic regression models, some care is taken to avoid model saturation; the algorithm may exit early in this setting. The objective function is defined to be

1/(2*n)RSS + penalty

for "gaussian" and

-1/nobs loglik + λ*penalty

for "binomial", where the likelihood is from a traditional generalized linear model for the log-odds of an event.

This algorithm is stable and generally converges quite rapidly to values close to the solution. However, especially when p is large compared with n, grpreg may fail to converge at low values of lambda, where models are nonidentifiable or nearly singular, Often, this is not the region of the coefficient path that is most interesting. The default behavior warning the user when convergence criteria are not met may be distracting in these cases, and can be modified with warn.conv (convergence can always be checked later by inspecting the value of iter).

If models are not converging, increasing max.iter may not be the most efficient way to correct this problem. Consider increasing eps, increasing n.lambda, or increasing lambda.min in addition to increasing max.iter.

Value

An object with S3 class "grpreg" containing:

beta The fitted matrix of coefficients. The number of rows is equal to the number of coefficients, and the number of columns is equal to n.lambda.
family Same as above.
lambda The sequence of lambda values in the path.
lambda2 Same as above.
penalty Same as above.
df A vector of length n.lambda containing estimates of effective number of model parameters all the points along the regularization path. For details on how this is calculated, see reference.
iter A vector of length n.lambda containing the number of iterations until convergence at each value of lambda.

Author(s)

Patrick Breheny <patrick.breheny@uky.edu>

References

Breheny, P. and Huang, J. (2009) Penalized methods for bi-level variable selection. Statistics and its interface, 2: 369–380.

See Also

plot and select methods.

Examples

data(birthwt.grpreg)
X <- as.matrix(birthwt.grpreg[,-1:-2])
y <- birthwt.grpreg$bwt
group <- c(1,1,1,2,2,2,3,3,4,5,5,6,7,8,8,8)
fit <- grpreg(X,y,group,penalty="gLasso")
plot(fit)
fit <- grpreg(X,y,group,penalty="gBridge",lambda.max=.08)
plot(fit)
fit <- grpreg(X,y,group,penalty="gMCP")
plot(fit)
select(fit,X,y)

# Role of delta
# Note that fit2 cuts down on the # of iterations,
# but creates artifacts near 0
fit1 <- grpreg(X,y,group,penalty="gLasso")
fit2 <- grpreg(X,y,group,penalty="gLasso",delta=1e-4)
par(mfrow=c(2,1))
plot(fit1)
plot(fit2)
dev.off()
plot(fit1$lambda,fit1$iter,type="l")
lines(fit2$lambda,fit2$iter,lty=2)

# Logistic regression
y <- birthwt.grpreg$low
fit <- grpreg(X,y,group,penalty="gLasso")
plot(fit)
fit <- grpreg(X,y,group,penalty="gBridge",lambda.max=.03)
plot(fit)
fit <- grpreg(X,y,group,penalty="gMCP")
plot(fit)
select(fit,X,y)

[Package grpreg version 1.1 Index]