Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [constrained-regression]

constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.

Wikipedia has an article https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)#Constrained_linear_least_squares and the more general https://en.wikipedia.org/wiki/Constrained_optimization with further references.

115 questions
23
votes
2 answers

The limit of "unit-variance" ridge regression estimator when $\lambda\to\infty$

Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum of squares as…
amoeba
  • 93,463
  • 28
  • 275
  • 317
20
votes
4 answers

Linear regression with slope constraint

I want to perform a very simple linear regression in R. The formula is as simple as $y = ax + b$. However I would like the slope ($a$) to be inside an interval, let's say, between 1.4 and 1.6. How can this be done?
Iñigo Hernáez Corres
  • 545
  • 1
  • 4
  • 9
19
votes
1 answer

How to achieve strictly positive forecasts?

I am working on a time series whose values are strictly positive. Working with various models including AR, MA, ARMA, etc, I couldn't find an easy way to achieve strictly positive forecasts. I'm using R for doing my forecasts, and all that I could…
Ho1
  • 327
  • 5
  • 14
15
votes
4 answers

How to fix one coefficient and fit others using regression

I would like to manually fix a certain coefficient, say $\beta_1=1.0$, then fit coefficients to all other predictors, while keeping $\beta_1=1.0$ in the model. How can I achieve this using R? I'd particularly like to work with LASSO (glmnet) if…
raco
  • 339
  • 1
  • 3
  • 11
10
votes
1 answer

Constrained Regression in R: coefficients positive, sum to 1 and non-zero intercept

I have the model that I need to estimate, $$ Y = \pi_0 + \pi_1 X_1 + \pi_2 X_2 + \pi_3 X_3 + \varepsilon, $$ with $\sum_k \pi_k = 1 \text{ for }k \geq 1$ and $\pi_k\ge0 \text{ for }k \geq 1$. Elvis answer to another question solves this for the…
user14281
10
votes
3 answers

Calculating the p-values in a constrained (non-negative) least squares

I have been using Matlab to perform unconstrained least squares (ordinary least squares) and it automatically outputs the coefficients, test statistic and the p-values. My question is, upon performing constrained least squares (strictly nonnegative…
cgo
  • 7,445
  • 10
  • 42
  • 61
8
votes
6 answers

Logistic regression classifier with non-negative weights constraint

My feature data is defined in such a way that I believe all weights must be non-negative. I am looking for a reference discussing how to optimize the weights of a logistic regression classifier with the constraint that the weights must be…
o17t H1H' S'k
  • 511
  • 6
  • 11
7
votes
3 answers

Linear Regression with individual constraints in R

I am using the nnls() function from the nnls package in R to do a linear regression for regressors $x_i$ and observations $y$. The function delivers beta coefficients $\beta_i\geq{0}, \forall i$. However, is it possible to apply the constraints only…
Dom
  • 353
  • 3
  • 9
6
votes
2 answers

Confidence intervals for maximum likelihood estimator with constraints

Let us suppose I have a maximum likelihood estimator for a multivariate parameter $\vec{\theta}$. The parameter is subject to the following constraints: $\theta_i \in [0,1]$ $\sum_i \theta_i = 1$ I want to calculate the confidence intervals for…
quant_dev
  • 634
  • 4
  • 13
6
votes
1 answer

Confidence interval for a constrained fit to Gaussian-like data

I'm working with data from an instrument which is expected a priori to produce Gaussian (normally) distributed data: \begin{equation} G = A\exp\left(-\dfrac{(x - \mu)^2}{\sigma} \right) \end{equation} The data are normally sparse, with only about…
Jeff T.
  • 61
  • 2
5
votes
1 answer

(Prediction) constrained OLS in R

I have a collection of $n$ datapoints $(y_i,\bf{x}_i)$ in $\mathbb{R}^{p+1}$ and would like to estimate the following model in R: $$\underset{\bf{b}\in\mathbb{R}^{p}}{\arg.\min}\;\sum_{i=1}^n(y_i-\bf{x}_i'\bf{b})^2$$ $$u.c.\;\;\;0\leq…
user603
  • 21,225
  • 3
  • 71
  • 135
5
votes
2 answers

Monotonic splines in Python

I am trying to find a procedure to fit data monotonically in Python. The data won’t be necessarily monotonic. I just would like to achieve a monotonic fit because of theoretical assumptions. I imagine that a way of doing that would be to run an…
Eaglez
  • 61
  • 4
5
votes
1 answer

What are drawbacks of isotonic regression?

I have been reading about isotonic regression and it seems like a great method that will give one a monotone regression function estimator and, moreover, is free of any tuning parameters. Why are people still using alternative approaches to such…
MerylStreep
  • 236
  • 1
  • 2
  • 9
5
votes
3 answers

regression with constraints

I have some domain knowledge I want to use in a regression problem. Problem statement The dependent variable $y$ is continuous. The independent variables are $x_1$ and $x_2$. Variable $x_1$ is continuous and positive. Variable $x_2$ is…
PolBM
  • 1,363
  • 11
  • 19
5
votes
1 answer

How do I fit a constrained regression in R so all coefficients are positive and above 0

I am trying to understand how to solve the below quadratic program: This is my model Y=π1X1+π2X2+π3X3+ε, My constraints are: - all weights πk>0 ; so each weight need to be above 0 My goal is minimize ∑(Yi−(π1Xi1+π2Xi2+π3Xi3))2 The code below show…
user4797853
  • 63
  • 1
  • 5
1
2 3 4 5 6 7 8