9

I am looking for any help, advice or tips in how to explain heterogeneity / heteroscedasticity to biologists in my department. In particular I want to explain why its important to look for it and deal with it if it exists, I was looking for opinions on the following questions.

  1. Does heterogeneity influence the reliability of random effect estimates? I am pretty sure it does, but I couldn't find a paper.
  2. How serious a problem is heterogeneity? I have found conflicting views on this, while some say that model standard errors etc. will be unreliable, I have also read that it is only a problem if the heterogeneity is severe. How severe is severe?
  3. Advice on modelling heterogeneity. Currently, I focus largely on the nlme package in R and the use of variance covariates, this is pretty simple and most people here use R so providing scripts is useful. I am also using the MCMCglmm package as well, but other suggestions are welcome, particularly for non-normal data.
  4. Any other suggestions are welcome.
gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
user3136
  • 609
  • 2
  • 6
  • 12
  • 1
    @user3136, please clarify is your main concern heterogeneity (different species with unique attributes, probably taken from known distribution) or heteroscedasticity (the property of random process to have time varying variance), as these two concepts are clearly different. Both problems are interesting though, the former leads to mixed effects models or random coefficient models, the latter has many cures to deal with (but is less important, since $OLS$ just is not efficient, but estimates are unbiased). – Dmitrij Celov Mar 30 '11 at 11:04
  • Hi, sorry about that. My concern really is about heteroscedasticty. One problem I have had is that these two terms (heteroscedasticity and heterogeneity) are used almost interchangeably. In this context , both are supposed to refer to the situation when the error in the residuals is not constant – user3136 Mar 30 '11 at 14:08
  • 4
    Not really so, sources of heterogeneity are many: differences in parameters (random parameters for instance), variables (usual regression thing), residuals (idiosyncratic part that may belong to different distributions, or distribution parameters may be different, heteroscedasticity belongs here, thus it is a separate case of heterogeneity), differences in functional form. So I would leave just the more particular term - heteroscedasticity. – Dmitrij Celov Mar 30 '11 at 15:38
  • Thanks Dimitrij, one question I had meant to ask was about the correct terminology in this area. – user3136 Mar 30 '11 at 16:10

3 Answers3

6

One option is to use a simulation. So set up a model where you specifically specify the heterogeneity suppose as $var(\alpha_i)=\overline{X}_i^2\sigma^2_u$. Then generate your data from this model, taking random intercepts as a simple example.

$$\alpha_i=\overline{X}_i u_i\;\;\;\;\;\; u_i\sim N(0,\sigma^2_u)$$

$$Y_ij=\alpha_{i}+\beta X_{ij} + e_{ij}\;\;\;\;\;\; e_{ij}\sim N(0,\sigma^2_e)$$

(hope this notation makes sense). I believe playing around with a set-up such as this will help you answer question 2). So you would fit this model using a random intercept, when in fact it should be a random slope (which gives you a partial answer to question 3 - random intercepts can account for "fanning" to a degree - this is "level 2 fanning"). The idea of the above is to try as hard as you can to break your modeling method - try extreme conditions consistent with what you know about the data, and see what happens. If you are struggling to find these conditions, then don't worry.

I did a quick check on heteroscedasticity for OLS, and it doesn't seem to affect the estimated betas too much. To me it just seems like heteroscedasticity will in some places by giving an under-estimate of the likely error, and in other places it will give an over-estimate of the likely error (in predictive terms). See below:

awaiting plot of data here, user currently frustrated with computers

And one thing which I always find amusing is this "non-normality of the data" that people worry about. The data does not need to be normally distributed, but the error term does. If this were not true, then GLMs would not work - GLMs use a normal approximation to the likelihood function to estimate the parameters, as do GLMMs.

So I'd say if estimating fixed effect parameters is the main goal then not much to worry about, but you may get better results for prediction by taking heteroscedasticity into account.

probabilityislogic
  • 22,555
  • 4
  • 76
  • 97
  • 1
    HI, thanks for the advice. I am currently working on some simulations so I hope they work out. As far as I know heteroscedasticty does not affect estimation of regression coefficinets, but it can over- or under-estimate the standard errors of these estimates. – user3136 Mar 30 '11 at 14:11
  • It actually does both (over- and under-estimate) if there is heteroscedasticity - in a similar way that "averaging" of ordinary data will both over and under-estimate the actual values. For a slope, you will have different degrees of accuracy at different points on the line. – probabilityislogic Mar 31 '11 at 03:32
  • In my particular field there is also a large reliance on significance testing and therefore p-values as well . So I think that fact that the SE can be over- and under-estimated may cause some problems if you base all inference on your p-values. – user3136 Mar 31 '11 at 09:57
  • I'd say you have bigger problems than standard errors if all inference is based on p-values and significance testing. These kinds of things encourage "mindless statistics". – probabilityislogic Mar 31 '11 at 12:10
  • I couldn't agree more, I think most people I speak to know that such an approach is suspect, but its difficult to stop them from focusing solely on p-values, often at the expense of everything else. – user3136 Apr 01 '11 at 08:19
  • @user3136 - and I think the main reason is that "mindless statistics" such as the use of p-values, is easy. It may be wrong, but easy often wins. And software is another killer - just contributes to the difficulty of doing anything else. you need to have something to put in their place. – probabilityislogic Apr 01 '11 at 10:01
6

Allometry would be a good place to start that will be familiar to biologists. Logaritmic transformations are often used in allometry because the data have a power-law form, but also because the noise process is heteroskedastic (as the variability is proportional to size). For an example where this has caused a severe problem, see "Allometric equations for predicting body mass of dinosaurs", where the conclusion that dinosaurs were only half the size previously though was incorrect because an invalid assumtion of homoscedasticity was made (see the correspondance for details).

Dikran Marsupial
  • 46,962
  • 5
  • 121
  • 178
0

The best FREE online resource I know for learning about heteroskedasticity is Prof. Thoma's ECON 421 lectures from 2011. Specifically lectures 1 - 7. His lectures are very organized and easy to follow along regardless of your discipline.

Here is the first lecture. You can Find the rest of the lectures from the Winter 2011 semester here as Well. http://www.youtube.com/watch?v=WK03XgoVsPM

Also, the corresponding website for Prof. Thoma's Econ 421 course has Homework Problems and as well as their solutions. For solutions that require software, the solution is detailed step-by-step using a combination of text, formulas, and screen shots from Eviews.

Eventhough the the steps used to solve the homework problems are detailed using screen shots from E-views, the solutions easily translate well into other statistical packages such as STATA or R stats.

There are no Solutions listed for the Homeworks from the 2011 semester, which is Prof. Thoma's last video taped semester. However there are homeworks available for his Winter 2012 semester.

Here is a link to the Homework solutions section of Prof. Thomas Winter 2012 421 class. Specifically here is the Solution to Homework 3 where heteroskedasticity is introduced to the homework sets. http://economistsview.typepad.com/economics421/2012/02/solution-to-homework-3.html

James Bradshaw IV
  • 1
  • 1
  • 1
    Thank you for these references, James. Here, however, the questioner is asking for *explanations* and *advice.* So that your answer can be relevant, would you be able to paraphrase specific "help, advice, or tips" in these references? – whuber Sep 21 '12 at 14:15