0

I just did a binary linear regression in R with a dataset that has 100000 lines. The output of the regression is, that almost every parameter is highly significant. I wouldn't expect that when I look at the boxplots. Did I do something wrong in my code or can that be right?

Call:
glm(formula = damage ~ dist_gerst + dist_gew + dist_hunt + dist_kart + 
    dist_mais + dist_raps + dist_road1 + dist_road2 + dist_road3 + 
    dist_road4 + dist_roada + dist_rog + dist_rmr + dist_ruben + 
    dist_sg + dist_wald + dist_hecke + dist_weize + dist_wg + 
    dist_bra, family = binomial(logit), data = data)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-3.3963  -0.0024   0.2446   0.4947   5.1474  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept) -9.7073574  0.5951280 -16.311  < 2e-16 ***
dist_gerst   0.0355374  0.0053699   6.618 3.65e-11 ***
dist_gew     0.0033584  0.0012147   2.765 0.005698 ** 
dist_hunt    0.0531545  0.0017567  30.259  < 2e-16 ***
dist_kart    0.0472300  0.0022333  21.148  < 2e-16 ***
dist_mais    0.0578135  0.0031780  18.192  < 2e-16 ***
dist_raps    0.0470257  0.0021689  21.682  < 2e-16 ***
dist_road1  -0.0135003  0.0023328  -5.787 7.15e-09 ***
dist_road2   0.0304884  0.0016027  19.023  < 2e-16 ***
dist_road3  -0.0003806  0.0011631  -0.327 0.743505    
dist_road4  -0.0515227  0.0048316 -10.664  < 2e-16 ***
dist_roada  -0.0186244  0.0050640  -3.678 0.000235 ***
dist_rog    -0.0403263  0.0031825 -12.671  < 2e-16 ***
dist_rmr    -0.1133255  0.0045872 -24.705  < 2e-16 ***
dist_ruben   0.1168154  0.0032703  35.721  < 2e-16 ***
dist_sg     -0.0450639  0.0020300 -22.199  < 2e-16 ***
dist_wald    0.1127090  0.0035169  32.047  < 2e-16 ***
dist_hecke   0.1065537  0.0028434  37.474  < 2e-16 ***
dist_weize  -0.1496686  0.0038303 -39.075  < 2e-16 ***
dist_wg     -0.0937316  0.0051061 -18.357  < 2e-16 ***
dist_bra    -0.0599667  0.0023916 -25.074  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 64559  on 53510  degrees of freedom
Residual deviance: 32357  on 53490  degrees of freedom
  (46231 observations deleted due to missingness)
AIC: 32399

Number of Fisher Scoring iterations: 9
meles
  • 151
  • 1
  • I first posted that on stackoverflow. Here are the answers I got there: See [Practical vs Statistical Significance](http://stats.stackexchange.com/q/81778/3601) over on the statistics stackoverflow site. – Aaron yesterday Also this answer (and others) to [Why is "statistically significant"](http://stats.stackexchange.com/a/79304/3601) not enough? – Aaron yesterday – meles Jan 15 '14 at 14:00
  • (part 2 of the old comments) boxplots show you the population distribution, similar to looking at the standard deviations rather than the standard errors of the mean. Try adding the notch=TRUE argument (see ?boxplot for details: it gives an approximate 95% confidence interval for the median). – Ben Bolker yesterday @Aaron: I guess you are referring to that: "With increasing sample size, small effects will become statistically significant". I was expecting that, but what would be the solution for it? Is there a method that's not depending on the sample size? – user24484 yesterday – meles Jan 15 '14 at 14:04
  • (part 3 of the old comments)One solution is simply not to look for statistical significance but instead for practical significance, that is, which terms have effects that are big enough for you to be interested in them. – Aaron yesterday I looked into the migration procedure, and since there are no answers, they do suggest that you simply delete it and repost, so that a moderator need not be involved. – Aaron 21 hours ago – meles Jan 15 '14 at 14:05
  • 4
    Yes. It's not only possible, but indeed highly likely. This is an utterly standard feature of significance tests. The main function of significance tests is, arguably, to avoid being fooled by chance-like fluctuations in small samples, but that is not your risk here. It is best to focus on confidence intervals for your parameters. (EDIT: You already have this kind of comment and links to relevant questions, so why is this being asked again?) – Nick Cox Jan 15 '14 at 14:07
  • @NickCox The problem is that the actual sample size is small (sorry I thought I had already mentioned that). I had just about 100 different areas with different sizes and it is not always possible to distinguish where one area ends and the next one starts. That's why I decided to use a grid (1x1m) and do the distance calculations (dist_gerst,...) for every gridcell. UPDATE: I added the ID of the areas as random factor to the model. – meles Jan 15 '14 at 15:25
  • Not sure I understand this. But (1) I gather that your sample size is 100,000, which I wouldn't call "small" in this context (2) it is now clear that you have spatial data (I suppose the names with prefix "dist" were a clue) but if data are (substantially) spatially dependent then standard significance tests don't apply any way! – Nick Cox Jan 15 '14 at 15:28
  • @Aaron Is there a way to get the effect size with the models summary? `summary(glm)` – meles Jan 15 '14 at 15:29
  • @NickCox any ideas how else I should test the significance of the parameters? I have to test which factors influence damage in fields by game. I mapped areas with damage and those without. In between there might be areas where it is not sure. So the dependent variable is damage(0,1) and the independets are dist_...(in classes from 1-100). PS. Should I post a new question under this circumstances? – meles Jan 15 '14 at 17:41
  • Yes. Post a new question. However, there is no recipe for spatial dependence except using an appropriate model, and it is very difficult to say what that should be. – Nick Cox Jan 15 '14 at 17:47
  • I submitted this as a [new question](http://stats.stackexchange.com/questions/82384/how-to-analyse-spatial-data-where-the-depending-variable-is-binary) – meles Jan 16 '14 at 09:34

0 Answers0