5

This is my first time using lars, so this question is probably obvious.

When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept.

I thought part of how LASSO worked was shrinking coefficients, which to me would imply you need an intercept to make up for the shrinking coefficients.

Obviously I am missing something here, so I would really appreciate your help.

Jeremy Miles
  • 13,917
  • 6
  • 30
  • 64
EvKohl
  • 1,090
  • 2
  • 10
  • 14
  • Note that centering both response & predictors is usual in LASSO because it puts the intercept - which you don't want to shrink - to zero before fitting. R's `lars` package does this by default, storing the response & predictor sample means (as well as the $L_2$-norms used for scaling) so that you can recover the intercept (by extracting them & calculating it, or by using @Dimitriy's neat `predict` method). I'm not familiar with Stata, but you should be able to recover the intercept by manual calculation at least. – Scortchi - Reinstate Monica Jan 27 '15 at 18:00

1 Answers1

5

lars in Stata matches the defaults in R (standardize all variables to have unit L2 norm and include an intercept). The intercept itself, however, is not reported in either package. I think I figured out how to get it in R using predict and setting all the covariates to zero, but I can't seem to successfully replicate this with Stata.

Here's the Stata code (which calls R using rsource from SSC) showing the equivalence:

set more off
capture ssc install rsource
sysuse auto, clear
saveold "/Users/dmasterov/Desktop/auto_v12.dta", replace

rsource, terminator(END_OF_R) 

library(lars)
library(foreign)
data<-read.dta("/Users/dmasterov/Desktop/auto_v12.dta")
attach(data)
x<-data[c("mpg","headroom","trunk","weight","length","turn","displacement")]
x<-as.matrix(x)
y<-price
lars <- lars(x,y, type="lar", normalize = TRUE, intercept = TRUE)
lars
coefficients(lars)
summary(lars)
predict(lars,data.frame(mpg=0,headroom=0,trunk=0,weight=0,length=0,turn=0,displacement=0),s=7)$fit
detach(data)

END_OF_R

sysuse auto, clear

lars price mpg headroom trunk weight length turn displacement, algorithm(lars)

At first, my intuition was that you should be able to take the lars coefficients and use them in constrained regression in Stata to get the intercept. However, I cannot seem to match the number I get in R with this method. I get essentially the intercept you would get if you included no constant, but a vector of ones in X whose coefficients you penalized.

dimitriy
  • 31,081
  • 5
  • 63
  • 138