LASSO closed form with two regressors, JRSSB eq. (6)

Question

I was having look at the orginal Tibshirani paper, JRSSB 1996. In particular, I am trying to understand his equation (6), which says that the LASSO estimates $(\hat\beta_1,\hat\beta_2)$ in the case of two regressors will be

$$\hat\beta_1=[s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2]^+\qquad\qquad(1)$$

and

$$\hat\beta_2=[s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2]^+\qquad\qquad(2)$$

where $(x)^+$ means, I think, "positive part of $x$" (i.e. $(x)^+=\max(x,0)$) and $s$ is our "budget"

$$|\hat\beta_1|+|\hat\beta_2| \leq s$$

The OLS estimates $\hat\beta^{(ols)}_1$, $\hat\beta^{(ols)}_2$ need to be positive and

$$\hat\beta^{(ols)}_1+\hat\beta^{(ols)}_2 \geq s.$$

Now, my question: if $\hat\beta_2$ is shrunk to zero, that means

$$\max[s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2,0]=0$$

or

$$s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2 < 0$$

or

$$s/2 < (\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2$$

Plugging this into (1) yields

$$\hat\beta_1 = s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2 > s/2+s/2=s$$

Hence, the "budget constraint"

$$\hat\beta_1+\hat\beta_2\leq s$$

would be violated as $\hat\beta_1>s$ and $\hat\beta_2=0$.

Where's my mistake?

Answer 1

If $$\max[s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2,0]=0$$

then

$$s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2 \leq 0.$$ But you wrote $$s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2 < 0.$$ Now $$s/2-(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2 \leq 0$$ implies $$s/2\leq (\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2.$$ Next we have: $$\hat\beta_1=[s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2]^+=max[s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2,0].$$ Therefore, $$s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2\geq s/2+s/2.$$ So $$s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2\geq s \geq 0\qquad\qquad (1)$$ Hence using (1) we have: $$\hat\beta_1=max[s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2,0]=s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2 \qquad\qquad (2)$$ But you found $$\hat\beta_1>s.$$ Finally using (2) we have: $$|\hat\beta_1|+|\hat\beta_2|=|\hat\beta_1|+0=|\hat\beta_1|=|s/2+(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2| \leq s/2 +|(\hat\beta^{(ols)}_1-\hat\beta^{(ols)}_2)/2|.$$ The last in-equality can be less than or equal to $s$ and nothing is violated.