1

For multiple least squares linear regression, can we actually specify a ratio between the coefficients as the prior? For example, for following linear model: $y = b_1*x_1 + b_2*x_2 + b_3*x_3$ ,can we specify $b1:b2:b3 = 1:2:3$?

Conditions:

  1. We don't know what exactly $b_1, b_2, b_3$ are, we only know they are expected to be proportional to each other.
  2. They are not exactly proportional, which means we cannot reduce it to univariate regression problem: $y = b_1*(x_1+2*x_2+3*x_3)$. The ratio between the effects are prior expectations, we want data can help adjust specific values for the coefficients.
kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
wuhabjmo
  • 11
  • 1

1 Answers1

0

I variant of your question is answered here: Linear model with constraints on coefficients in terms of ratios, but your question have a Bayesian twist.

First you will need some prior on $b_1$ (maybe a weak prior), then you write $b_2 = \gamma_2 b_1, b_3=\gamma_3 b_1$ and now you need a prior on $\gamma_2$ with prior expectation 2 and (maybe a small) variance, likewise for $\gamma_3$, but now with prior expectation 3.

It shouldn't be difficult to program such a model in bugs or stan.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • 1
    Thanks a lot! Kjetlil. So for the prior on $b_1$, I'm think about using one round of univariate regression first, like: $y = b_1(x_1+2*x_2+3*x_3)$ to calibrate $b_1$ as the prior you mentioned. And then use this prior on $b_1$ to do the following multiple regression using MCMC. Is this something sound reasonable? Also can you please point me some specific parts of bugs/stan that I should look at to implement? I'm more familiar with R. are there something related too? Thanks. – wuhabjmo Aug 28 '18 at 21:11