2

I have to estimate linear weight $\beta$ for regression $Y \sim \mathbf{X}$, where $Y$ are non-negative samples. If I perform vanilla regression (lets assume ridge regression) it will find $\beta$ such that most of estimated $\hat{Y}$ are non-negative. But some of them will be negative. But I need them to be invariably non-negative. In Bayesian framework, it implies I need a prior $\beta$ that is non-negative (ridge assume normal priors). Is there simple way to modify ridge regression to obtain this? Or putting non-negative priors and estimating via MCMC is only solution to it.

For further description of problem (why I need estimated $\hat{Y}$ to be positive). My full model is a hierarchical model where there is a variable $Z$ that is a linear model of latent variable $Y$ i.e $Z \sim N(\Theta^T \mathbf{Y}, \sigma^2)$. Observable variable are $Z$ and $X$. I need estimates of $\beta$ and $\theta$. I am currently estimating $\beta$ given Y. Given $\hat{Y}$ I am estimating $\theta$. I am iterating between this 2 steps. If $\hat{Y}$ is negative the problem become unidentifiable.

avi
  • 399
  • 3
  • 12
  • 5
    By the middle of the first paragraph you appear to confuse two separate things: the non-negativity of the responses $Y$ and the non-negativity of the parameter $\beta$ of the regression (I presume your "linear weight" is a parameter!). In the second paragraph you assert that $Y$ is "latent" and then immediately claim it is observable! Those inconsistencies appear to make this question unanswerable. – whuber Oct 24 '13 at 21:20
  • 2
    The usual assumption with linear regression is that the variance of the observations is constant. If $Y$ is nonnegative, but it's close enough to the zero bound that a fitted value could be negative, the constant variance assumption is untenable. The linearity assumption also becomes more difficult to maintain, unless you have an explicit supportable theoretical model for the process (rather than just throwing linearity at a problem for which it may not be reasonable). Look to your model; you may need a more suitable model. When you have one, the needed characteristics tend to come with it. – Glen_b Oct 25 '13 at 00:27
  • 3
    Based on the relative sophistication of your question (wrinkles pointed out by whuber aside), I assume that you've heard of GLM's? Would a log-linked gaussian or a gamma GLM not solve your problem? – generic_user Oct 25 '13 at 04:34
  • 1
    @whuber Corrected. Thanks for pointing out the mistake. Y is not observable. When I referred non-negativity I meant estimated $Y$ should be non-negative. beta have no constraints. I assumed linearity because of convenience of estimating parameters. The intuition of the model is similar to Hierarchical Bayesian- instead of sampling values are set it MAP estimates in each iteration. – avi Oct 25 '13 at 04:35

0 Answers0