4

My dependent variable is, by origin, absolute residual* left after some regression; it is distributed half-normally. Now I plan to use Generalized linear model in SPSS (GENLIN) to regress it on some predictors (totally different from those which produced the residuals). What distribution type should I use for the DV? For continuous data, GENLIN offers Gamma, Inverse Gaussian and Tweedie distributions. Which to choose to model the half-normal? Or should I apply special transforms before? And what link function would be most appropriate? What can you advice on that? Thanks.

*More precisely, I'm analysing positive residuals separately and negative residuals separately.

ttnphns
  • 51,648
  • 40
  • 253
  • 462
  • Why are you using an absolute residual as a dependent variable? If you're interested in how the variance depends on some predictors, there may be better ways of doing it. – onestop Nov 12 '11 at 13:59
  • Actually, I'm interested in half of the residuals at a time. I study positive residuals and negative residuals separately, as DVs on their own, with potentially different predictors and models. No, I'm not interested in variance. – ttnphns Nov 12 '11 at 14:49
  • If you had an unscaled normal (such as you might approximately have from raw residuals) before you split them, and we ignore that the residuals won't be exactly identically distributed nor independent, then the ***square*** of the half-normal would be scaled chi-square(1) (gamma with shape parameter $\frac12$). (In R you can specify the deviance parameter when asking for the summary table, I assume something similar is possible in SPSS). As for what link function is suitable, that really depends on your model; but note that a log-link would work as well for squared values as absolute ones. – Glen_b May 28 '17 at 07:04

2 Answers2

3

Even though you state you are not interested in the variance your exercise is very similar to econometric models trying to estimate the variance as a function of some regressors. They typically work in a two step approach like you suggest here, estimate a model for the mean, take the residuals from this first model, and then fit a seperate model on the residuals (which may or may not have different regressors than the mean model).

If the first model is OLS and you square the residuals, a gamma model is often used as the link function (with the relationship between the residuals and a chi-square distribution the motivating factor I imagine). But in your case you have several complicating factors that make the choice of the link function less clear:

  • The residuals from generalized linear models are more complicated than residuals from OLS (and are not even well defined - you have several options to choose from)
  • You want to use the absolute value of the residual (instead of the square)
  • You want to seperately estimate models for negative and positive residuals (This seems like a mistake to me, I would consider stacking the equations and using dummy variables to distinguish between negative and positive residuals in whatever subsequent equation you want to examine.)

Because it is unlikely you will be able to even approximately derive a distribution of the residuals, an ad-hoc approach of simply using a distribution that will reasonably approximate the residual distribution seems called for. The distribution will look different for different types of generalized linear models (e.g. a logistic regressions residuals will look different than a Poisson regression) so it is hard to give general advice. A gamma distribution may work alright for the residuals from a Poisson regression, but I'm not sure it would for the residuals from a logistic regression (at least in the typical residual plots I have seen for these types of models). Gung gives other possibilities that aren't in the exponential family that may be considered as well.

For an example application of estimating the variance as a function of regressors (along with a brief review of the econometrics literature that has similar applications), see

Western, B., & Bloome, D. (2009). Variance function regressions for studying inequality. Sociological Methodology, 39(1), 293-326. Pre-print PDF Here

Community
  • 1
Andy W
  • 15,245
  • 8
  • 69
  • 191
2

I can't tell exactly what you're doing, or why, and it seems a little strange, to be honest. I'm guessing that you have a regression model, and believe that those who are above the line are from a different underlying (latent) group from those who are below the line, and hope to find out something about these 'groups'. Perhaps not, but if so, I'm not sure this is a legitimate strategy. I have difficulty imagining how the distance above a regression line can be determined by one set of explanatory variables, and the distance below the line by a different set. There is a great deal of real noise in data and trying to explain it is more likely to lead to phantoms than knowledge. But I don't mean to scold.

Perhaps a tobit model would be appropriate. The tobit is based on a probit model with an underlying normal distribution, but where some proportion of the data have been censored. Recognize that this means the model is assuming the other half of the data are a part of the picture, but just have been hidden from the model, which is true in your case. The non-zero level of intrinsically unexplainable variance that must exist in your data necessitates that some proportion of the negative residuals really belong to the set of your positive residuals and vice versa, so the assumption the model is making is justifiable. Nonetheless, running two tobits would allow you to try to model your residuals with different explanatory variables.

I have no idea how to conduct a tobit regression in SPSS; I'm not sure the software supports that analysis. If you want to know a little bit more about it, and how to conduct it with other software, UCLA's website has a nice, concise, clear description (of course) of it for R. If you want more theoretical background on it, J Scott Long's book is very accessible. It requires calculus, but he steps through it very gently.

Good luck.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • `how the distance above a regression line can be determined by one set of explanatory variables, and the distance below the line by a different set` Imagine the residuals are deviations of body weight from normal (median) body weight. Binary variable "endocrine diagnosis" (Present/Absent) will strongly positively correlate with positive residuals, extent of overweight (because of obesity cases), but will not correlate or will weakly negatively correlate with negative residuals, extent of underweight. – ttnphns Nov 13 '11 at 04:44
  • That helps to clarify the situation you are asking about. But, I wonder if a simple logistic regression with "endocrine diagnosis" regressed onto BMI (the full range, not just above the median) will work. Patients who are underweight will have a very low probability of ED, but the model can handle that just fine. If you are particularly concerned about the symmetry of the logistic transformation, cloglog or loglog could be used instead. – gung - Reinstate Monica Nov 13 '11 at 06:36
  • Deviations (either only positive or only negative) from median body weight have to be DV, and different diagnoses will constitute set of IVs of the model. The only thing that stops me is that I don't know how to be, given that the distribution of DV is thus half-normal. I never worked with such distributed DVs before. – ttnphns Nov 13 '11 at 09:09
  • You could try a tobit model, then. If you're just looking at ED vs. no ED (i.e., just 1 factor), you could try a Mann-Whitney U test--it doesn't require assumptions about the distribution. Other than that, I don't know. – gung - Reinstate Monica Nov 13 '11 at 15:18
  • This got upvoted today, @ttnphns. I'm just seeing it again after years. I had forgotten about it. I wonder how useful this is. Would you prefer if I just deleted it? – gung - Reinstate Monica Sep 22 '17 at 20:01
  • I upvoted both answers here, but years ago. Are you feeling your answer is incorrect or irrelevant, to delete it? I don't see why you should want to. – ttnphns Sep 23 '17 at 06:49
  • OK, @ttnphns, just checking. – gung - Reinstate Monica Sep 23 '17 at 12:31