3

Non-statisticians tend to use step-wise regressions which is strongly argued by statisticians. This is something that I don't understand (unfortunately, I am not a statistician), but I just obey them. "Ok this is not a good way to do your modelling".

Here is (was) my model:

b <- lmer(metric1~A+B+C+D+E+F+G+H+I+J+K+(1|X/Y) + (1|Z), data = dataset) drop1 (b, test="Chisq")

(Just a small note: Watch out for the random effects in my model; random effects are Year, Month, Sampling.location; one of my variables is 1/0: I allready log-transformed my variables)

I am trying to find an exploratory model (with drop1 to reach final model) and evaluating it with my biological knowledge to see if the dependent ("metric" in this case) seems to be responding variables. I will repeat this process with 100 metrics just to evaulate which metrics seems to be responding environmental variables.

I was in the search for an acceptable model instead of stepwise according to the suggestions of statistics gurus.

However, there are lots of alternatives. I read alot, but still feel myself lost. Some say Lasso, some say elastic modelling, some say ridge regression... Which one fits for my purpose?

edit: meanwhile I have some tiny progress on glmulti. Is that a proper way of doing the thing that I aim?

Any advise for a better alternative and an easy model or a help page for dummies, or examples (that could be better) would be much appreciated.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
borgs
  • 115
  • 1
  • 1
  • 8
  • 1
    Don't take an egghead's word for it. Learn what you need to in order to understand the methods you want to use. Otherwise, you're pretty much guaranteed to make mistakes. – Kodiologist Apr 02 '17 at 18:27
  • Can you explain the term 'random effects' please? – Hugh Perkins Apr 02 '17 at 18:58
  • Well... I just follow what has been published in the area of ecology. Similar manuscripts report "year" (sampling date; I have 3 years observation on different months) as random effects. And my sampling year/month gives a nested random effect. However, I am not statistician, I follow mainly Zuur et al., 2009 (http://www.springer.com/gp/book/9780387874579) – borgs Apr 02 '17 at 19:04
  • Please notice that if you test 100 metrics with a given Type 1 error probability, you will have to adjust the Type 1 threshold used. To oversimplify things a bit: if there is a 5% probability of having a false positive from a test procedure, if we do that test procedure 100 times we would expect 5 false positives. – usεr11852 Apr 02 '17 at 23:08
  • @HughPerkins: Based on the context and the reference used see [here](http://stats.stackexchange.com/questions/4700) and [here](http://stats.stackexchange.com/questions/21760) for starters to get an idea of what *random effects* entail. (Also we are fortunate that a user named Ben Bolker is active in CV; he is a leading developer on mixed effects model in R; check his answers - when Ben talks, I take notes.) – usεr11852 Apr 02 '17 at 23:14

1 Answers1

5

This is a variable shrinkage problem with a bit of dressing because it has a mixed effect element.

You are correct about Lasso, Ridge Regression and Elastic regression being shrinkage methods. It seems though from your phrasing that you want to reduce the number of variables (rather than make the coefficients small, which is what can happen in ridge regression for instance).

If this is the case (and bearing in mind mixed effects) then the Lasso regression would be well suited to such an application. Observing that your original code is written in R, a useful package for implementing a mixed effect Lasso regression would be the ‘glmmLasso’. (Search CRAN for this package.)

You would want to learn more about Lasso regression. Chapter 6 of the book Introduction to Statistical Learning with Applications in R (ILSR) is easy reading. The book is available legitimately from the authors see here.

To answer your original question, which is "best", a statistician would answer "the one that minimises the out of sample error", an economist would answer "the interpretable one", your journal editor will say "the one that fits on a page". It's your hard work to balance (or ignore) these competing views. In short there is no "best" method, there are however sound methods. Start with the mixed effect Lasso (a sound method) and work from there.

PS: Please do not use the "drop" method you describe. It is essentially amounts to a stepwise method.

Community
  • 1
STP
  • 80
  • 1
  • 7