4

Estimating confidence intervals for non-normally distributed residuals can be accomplished using bootstrapping procedures, sandwich estimators or quantile regression.

But is there a way to calculate $\beta$ confidence intervals given the estimated value? That is, for each $\hat{Y}$ we know the residuals variance for this predicted value region, so rather than applying the same (robust) intervals to all predicted values, isn't it possible to adjust the intervals conditional to $\hat{Y}$?

In other words, the residuals could be split in different bins with their own distribution as the value of $\hat{Y}$ increases, then specific confidence intervals could be applied to each bin.

UPDATE I'm going to elaborate a bit more because I'm surprised there isn't a simple answer to this question (or an obvious flaw). Let's take the residuals analysis from this thread:

Residuals vs. Predicted

Here we have a loss of predictive power at the sides of the chart, that is for $\hat{Y} ~ (0.694, 2.23], \hat{Y} ~ (3.51, 9.53]$. Clearly the confidence intervals are not uniform across all values of $\hat{Y}$.

Wouldn't make sense to adjust the $\beta$ confidence intervals given $\hat{Y}$?

Community
  • 1
Robert Kubrick
  • 4,078
  • 8
  • 38
  • 55
  • If $Y_i = X_i\beta + \epsilon_i$ where $\epsilon \sim (0, \sigma^2_i)$ do you want a confidence interval for $Y_i$ or $E[Y_i]=\hat{Y}_i$ or $\beta$? – conjugateprior Mar 21 '13 at 15:27
  • The residuals are not normally distributed for my specific problem. I don't think this question would make sense if they were. In any case, I'm looking for $\beta$ confidence intervals, conditional on $\hat{Y}$. – Robert Kubrick Mar 21 '13 at 15:44
  • How about a mixture model of the residuals? – abaumann Mar 22 '13 at 09:48
  • @abaumann At first view it looks like mixture models are more than I need. I'm surprised there isn't a simple answer to this question. Isn't this a common problem? Covariates can loose their predictive power for certain values, i.e. when they tend to 0. Thus for $\hat{Y}$ close to 0 we should have larger $\beta$ confidence intervals. – Robert Kubrick Mar 22 '13 at 12:30

3 Answers3

1

I don't think you need different confidence intervals for the model parameters at different points. Your estimates for the model parameters will depend on the whole set of data and on the assumptions made for the distribution of the errors, but they will be one set of estimates for the entire model (independent of location in parameter space). Clearly, in this case you should not use the identically-independently-distributed assumption (you do not have 'identically').

Your tolerance intervals for the predictions would be different at different points. This is not simple to do and I am not sure this is covered in standard software packages. You may want to look at: Tolerance Intervals in a Heteroscedastic Linear Regression Context with Applications to Aerospace Equipment Surveillance, International Journal of Quality, Statistics, and Reliability Volume 2009 (2009), Article ID 126283, 8 pages http://dx.doi.org/10.1155/2009/126283

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
user130505
  • 11
  • 1
1

Perhaps you could take a look at the GAMLSS model of Rigby and Stasinopoulos. As in regression, this model allows to model how E[Y] varies as a function of predictor values. In addition, GAMLSS can model the spread of the residuals by a second, independent function of predictors. If needed, you could even model third and fourth moments of a distribution to account for skewness and kurtosis in the same way. The GAMLSS package in R has around 70 built-in distributions, so the model is extremely flexible, but still easy to use.

Stef van Buuren
  • 2,081
  • 15
  • 13
1

Have you looked at the Delta Method? It relates change in variance of one variable to another through the derivative (or partial derivative for multivariate methods) using the Taylor series.

http://www.stanford.edu/class/cme308/notes/TaylorAppDeltaMethod.pdf

The truth is that you can relate any scaled value of the standard deviation of one variable to another. The constant transports through the integral sign.

In light of the "pseudo-sigma" this suggests that quantiles can also be transported, as long as the Taylor Series approximation is valid.

EngrStudent
  • 8,232
  • 2
  • 29
  • 82