4

Using R, I can only find tools for performing L1 and/or L2 regularized linear regression (lars, glmnet) and tools for constrained linear regression (quadprog , or lsei {limSolve} , where the inequality and equality constraints can be only given in the form Ax = b , Gx <= h).

It seems inutitive for me that the possibility of combining both should be required very oft when solving specific regression problems, but so far I havent been able to do it.

Instead of providing information on my specific set of constraints and algebraic system, I´d be interested to know if this is a problem I can actually solve using the above mentioned packages? Are there any packages at all in R built for both parameter regularization and specific parameter penalties?

Update: For better understanding: I am not trying to combine different regularization methods (like in elastic net), nor trying to combine different parameter constraints. My goal is to combine regularization with specific coefficient constraints, so for example:

Find the most sparse solution (penalizing absolute values through LASSO) of a linear regression y = bx which satisfy the coefficient constraints bA < h for some given matrix A and threshold h.

min($\parallel$ $\beta$x-y$\parallel_2^2$ + $\lambda$ $\mid$ $\beta$ $\mid_1$ ), s.t $\beta$ A $\leqslant$ h

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
hirschme
  • 750
  • 1
  • 7
  • 21
  • 3
    Stan lets you estimate any model you can dream of, at the cost of a steeper learning curve. – Sycorax Apr 07 '16 at 13:44
  • Thank you for the fast answer. Yes I have heard of rstan as a very complete modeling package. I just thought my problem is so basic, it should be possible to modifiy practically any linear regression tool in order to set specific constraints and a penalized norm to the estimated set. – hirschme Apr 07 '16 at 13:52
  • Actually, now that you mention it, I think that I answered a similar question like this before. There was a trick that would let you coerce the linear constraint to just be a regular linear model.... it's maybe a year old. I don't have time to look now, but perhaps you'd be able to find it. – Sycorax Apr 07 '16 at 13:56
  • You can search the site: http://stats.stackexchange.com/search?q=[r]+constraint+regression and you'll find multiple similar threads, e.g.: http://stats.stackexchange.com/questions/61733/linear-regression-with-slope-constraint – Tim Apr 07 '16 at 15:30
  • Well not really. That search only looks for parameter constraints. I wrote in my question I already know how to apply constraints (with packages like quadprog or limSolve). My problem is to apply BOTH constraints and regularization to the regression problem.. – hirschme Apr 08 '16 at 09:35
  • 2
    Presuming that you already know MATLAB, it is very simple to specify and solve problems of the type you have described using CVX http://cvxr.com/cvx/ . – Mark L. Stone Apr 08 '16 at 15:28
  • 1
    @Mark Thank you for your advise, it was really easy to implement my problem using CVX in matlab. I guess for this kind of tasks I will be moving away from R more into the wolfram alpha and mathworks world.. – hirschme Apr 12 '16 at 12:54
  • I think you want the ["elasticnet" ](https://cran.r-project.org/web/packages/elasticnet/ "elastic net")package which combines both L1 and L2 regularization see ["wikipedia" ](https://en.wikipedia.org/wiki/Elastic_net_regularization "wikipedia") for the definition – ashokragavendran Apr 07 '16 at 20:02

0 Answers0