6

I am testing whether price per ounce of beer (continuous variable, range of values mostly between 0.1 and 0.5 dollars) and the presence of promotion, advertisement, and display (all binary) have effect on the total amount of ounces purchased (continuous variable). Here is my residual vs. fitted plot before the log transformation of y:

before log transformation of y

This is the residuals vs. fitted plot after the log transformation of y:

after log transformation of y

Heteroscedasticity is very high (White's general t statistics is nearly 800).

This is the histogram of the transformed y:

enter image description here

Any ideas or suggestions on how to improve my model or where to look for errors in order to improve the problem of heteroskedasticity are greatly appreciated.

Community
  • 1
Olga
  • 61
  • 1
  • 3

2 Answers2

4

Your response variable isn't really continuous. It is presumably discrete (you can't buy .5 ounces, and moreover, beers only come in certain ounce sizes). In addition, no one can buy less than 0 ounces (you can clearly see the floor effect in your top--untransformed--residual plot). As a result, using an OLS regression (that assumes normal residuals) is likely to be inappropriate. You should probably try to use Poisson regression. In fact, a zero-inflated Poisson, negative binomial, or zero-inflated negative binomial are more likely what you will end up needing.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • Here is a correction: My response variable is in fact continuous because I was able to calculate total ounces purchased per transaction regardless of size or form of beer packages. Any more suggestions? – Olga Jun 15 '14 at 18:06
  • 1
    The fact that you were able to get the exact number is nice, but those numbers only come in certain possible increments (eg, no .5's & no -3's). You would still be better served by using a count model. – gung - Reinstate Monica Jun 15 '14 at 18:18
  • 1
    You can even see this behavior in the log-transformed response histogram: you have "spikes" and deep valleys that arise because the response variable, while not necessarily integer-valued, is also not truly continuous because beer is typically sold in standardized volumes, so certain values are much more common than others. – heropup Jun 15 '14 at 20:12
3

Not only is your variable apparently discrete it clearly shows lack of fit at the left and right ends

enter image description here

Discreteness (red arrows) and lack of fit (green ellipses) apparent in residual plot.

You can't properly assess heteroskedasticity with a test statistic that assumes that the model for the mean is correct ... when it plainly isn't. Further, the fact that the t-value is large is unsurprising, since the sample size is huge. [A large t-statistic isn't saying "the heteroskedasticity is dramatic", it's saying "the sample size is big, so the standard error is tiny". The impact on your inference is measured more by something like the numerator of the $t$.

There may be hetero in that plot, but it's not terribly severe; there are more important issues to deal with first.

I'd suggest considering a gamma glm rather than fitting the logs with a linear model (presuming there are no exact zeros). Taking logs tends to make the discreteness in the low end perhaps "loom larger" than it would with a model on the original scale.

You should then work on the lack of fit problem, and then assess the degree of the hetero issue, but don't rely on a test statistic to assess the size/importance of it.

Glen_b
  • 257,508
  • 32
  • 553
  • 939