8

I've got a problem where

$$y = a + b $$

I observe y, but neither $a$ nor $b$. I want to estimate

$$b = f(x) + \epsilon$$

I can estimate $a$, using some sort of regression model. This gives me $\hat b$. I could then estimate

$$\hat b = f(x) + \epsilon$$

First problem: a regression model to predict $a$ could lead to $\hat b$ being negative, which wouldn't make any sense. Not sure how to get around this (not the sort of problem I've dealt with a lot) but seems like the kind of thing that others deal with routinely. Some sort of non-gaussian GLM?

The main problem is how to account for the uncertainty in the main model that comes from estimating $\hat b$. I've used multiple imputation before for missing covariates. But this is a missing "latent parameter." Alternatively, it is outcome data, which seems OK to impute. However I often hear of EM used for "latent" parameters. I am not sure why, nor do i know whether EM is any better in these contexts. MI is intuitive both to understand, implement, and communicate. EM is intuitive to understand, but seems more difficult to implement (and I haven't done it).

Is EM superior for the sort of problem I've got above? If so, why? Second, how does one implement it in R for a linear model, or for a semiparametric (GAM) model?

Community
  • 1
generic_user
  • 11,981
  • 8
  • 40
  • 63

1 Answers1

1

Whether or not it makes sense to use GLMs depends on the distribution of $y$. I'd be inclined to use a nonlinear least squares model for the whole thing.

So if your regression model is $a = Z\alpha+\nu$ where $Z$ are the predictors and $\alpha$ are the parameters in the regression model for $a$, and your model for $b$ is $b = f(x)+\epsilon$ but where $f(x)$ is restricted to be non-negative, you could write $f(x) = \exp(\psi(x))$ and fit a model like this:

$$ y = Z\alpha+\exp(\psi(x))+\eta $$

where $\eta$ is the sum of the two individual noise terms. (If you really intend that $y=a+b$ with no error at all, you have to do it differently; that's not really a stats problem as much as an approximation problem and you would probably want to look at infinity-norms then.)

If you put say a cubic regression spline in for $\psi$ that would be one easy way of getting some general smooth function in. That model could be fitted by nonlinear least squares. (Indeed, some algorithms can take advantage of the linearity of $a$ to simplify and speed up calculation.)

Depending on what you assume about $y$ or $f$, there are other things you might do instead.

That doesn't really address the imputation issue yet. However, this sort of model framework can be inserted into something like your suggestion of using EM.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thanks for the comment. Indeed, y = a+b with no error (or more precisely, the error is ignorable and ignored). Even more precisely, y, a, and b are all undefined below zero. So my regression where I model a cannot give me $\hat a$ that has elements less than zero. I've been getting around this by bottom-coding the fitted values (and top-coding so that they don't drive b--> <0), but this is a hack and there might be more elegant solutions. – generic_user Feb 06 '13 at 06:33
  • That's a lot of pertinent information that ought to be spelled out in your question, I think. – Glen_b Feb 06 '13 at 10:22