Normality assumption: how deal with them?

Question

I would like to ask whether there's a way to decide which test is the strongest in accomplishing the normality condition. Let's take this example, using a dataset I'm working on:

> data1
    age   fev  ht sex smoke
1     9 1.708 145   0     0
2     8 1.724 171   0     0
3     7 1.720 138   0     0
4     8 2.336 155   0     0
5     6 1.919 147   0     0
6     6 1.415  NA   0     0
7     8 1.987 149   0     0
8     9 1.942 152   0     0
9     6 1.602 135   0     0
10    8 2.193 149   0     0
11    6 1.878 135   0     0
12    5 1.400 124   0     0
13    5 1.256 133   0     0
14    4 0.839 122   0     0
15    9 2.988 165   0     0
16    8 2.980 152   0     0
17    9 2.100  NA   0     0
18    5 1.282 124   0     0
19    8 2.673 152   0     0
20    7 2.093 146   0     0
21    5 1.612 132   0     0
22    8 2.175 150   0     0
23    9 3.135 152   0     0
24    8 1.931 145   0     0
25    5 1.343 127   0     0
26    9 2.076 145   0     0
27    8 1.344 133   0     0
28    9 2.797 156   0     0
29    9 3.016 159   0     0
30    7 2.419 152   0     0
31    4 1.569 127   0     0
32    8 1.698 146   0     0
33    8 2.481 152   0     0
34    6 1.481 130   0     0
35    4 1.577 124   0     0
36    7 1.631 141   0     0
37    5 1.536 132   0     0
38    9 2.560 154   0     0
39    8 2.531 147   0     0
40    6 1.719 135   0     0
41    7 2.111 145   0     0
42    6 1.695 135   0     0
43    7 1.917  NA   0     0
44    8 2.144 160   0     0
45    9 3.029 156   0     0
46    8 2.215 152   0     0
47    8 2.388 152   0     0
48    8 1.292 132   0     0
49    9 2.574  NA   0     0
50    7 1.742 149   0     0
51    7 1.603 130   0     0
52    8 2.639 151   0     0
53    7 1.829 137   0     0
54    7 1.473 133   0     0
55    8 2.341 154   0     0
56    7 1.698 138   0     0
57    5 1.196 118   0     0
58    8 1.872 144   0     0
59    7 1.827 138   0     0
60    7 1.461 137   0     0
61    8 1.697 150   0     0
62    9 2.040 141   0     0
63    7 1.609 131   0     0
64    8 2.458 155   0     0
65    9 1.947 144   0     0
66    8 2.288 156   0     0
67    5 0.791 132   0     0
68    9 2.463 155   0     0
69    9 2.631 157   0     0
70    8 2.293 147   0     0
71    9 3.042 168   0     0
72    8 2.665 163   0     0
73    9 2.592 154   0     0
74    7 1.750 140   0     0
75    9 2.259 149   0     0
76    9 2.048 164   0     0
77    8 1.780 149   0     0
78    5 1.552 137   0     0
79    8 1.953 147   0     0
80    9 2.851 152   0     0
81    9 3.004 163   0     0
82    9 1.933 147   0     0
83    9 2.091 149   0     0
84    9 2.316 151   0     0
85    5 1.704 130   0     0
86    9 1.606 146   0     0
87    6 2.102 141   0     0
88    9 2.320 145   0     0
89    5 1.146 127   0     0
90    8 2.187 156   0     0
91    8 1.335 144   0     0
92    8 2.709 159   0     0
93    5 1.092 127   0     0
94    9 2.166 146   0     0
95    7 1.690  NA   0     0
96    8 2.145 151   0     0
97    7 2.095 145   0     0
98    6 1.697 140   0     0
99    9 2.455 152   0     0
100   9 2.130 150   0     0
101   8 2.993 160   0     0
102   9 2.529 150   0     0
103   7 1.726 135   0     0
104   9 2.442 156   0     0
105   4 1.102 122   0     0
106   9 2.056 160   0     0
107   8 2.305 164   0     0
108   9 1.969 150   0     0
109   8 1.556 149   0     0
110   3 1.072 117   0     0
111   8 1.512 135   0     0
112   7 1.370 140   0     0
113   6 1.338  NA   0     0
114   9 2.639  NA   0     0
115   4 1.389  NA   0     0
116   8 2.135 150   0     0
117   9 3.223 165   0     0
118   6 1.796 140   0     0
119   6 1.523 130   0     0
120   9 2.485 163   0     0
121   8 2.335 150   0     0
122   7 1.415 136   0     0
123   7 1.728 144   0     0
124   9 2.850 160   0     0
125   8 1.844 144   0     0
126   9 1.754 156   0     0
127   6 1.343 132   0     0
128   8 2.476 160   0     0
129   8 2.382 157   0     0
130   7 1.640 140   0     0
131   5 1.589 130   0     0
132   9 1.886 142   0     0
133   9 1.912 150   0     0
134   7 1.877 133   0     0
135   7 1.935 133   0     0
136   5 1.539 127   0     0
137   8 2.358 155   0     0
138   6 1.535 140   0     0
139   9 2.182 151   0     0
140   7 2.002 146   0     0
141   7 2.366 147   0     0
142   8 2.069 152   0     0
143   4 1.418 124   0     0
144   8 2.333 145   0     0
145   8 1.758 132   0     0
146   7 2.564 147   0     0
147   9 2.487 163   0     0
148   9 1.591 145   0     0
149   8 1.999 144   0     0
150   9 2.688 151   0     0
151   6 1.672 137   0     0
152   8 2.015 146   0     0
153   7 2.371 141   0     0
154   8 2.328 152   0     0
155   7 1.495 145   0     0
156  11 2.170 147   0     0
157  12 3.058 154   0     0
158  10 2.642 155   0     0
159  12 2.841 160   0     0
160  10 3.166 156   0     0
161  13 3.816 161   0     0
162  11 3.654 165   0     0
163  11 2.665 160   0     0
164  14 2.236 168   0     1
165  11 3.490 170   0     0
166  13 3.147 163   0     0
167  10 2.520 154   0     0
168  12 2.889 163   0     0
169  11 2.386 156   0     0
170  11 2.762 152   0     0
171  11 3.011 163   0     0
172  11 3.236 168   0     0
173  10 2.864 152   0     0
174  14 3.428 163   0     1
175  13 2.819 157   0     0
176  10 2.250 147   0     0
177  13 3.208 155   0     1
178  11 2.346 150   0     0
179  11 2.754 166   0     0
180  11 2.633 157   0     0
181  10 3.048 166   0     0
182  13 3.745 173   0     0
183  12 2.384 161   0     1
184  10 3.183 166   0     0
185  14 3.074 165   0     1
186  11 3.411 161   0     0
187  10 2.387 168   0     1
188  11 3.171 160   0     0
189  13 2.646 156   0     0
190  10 2.504 152   0     0
191  10 2.891 155   0     0
192  10 1.823 145   0     0
193  10 2.175 147   0     0
194  11 2.735 159   0     0
195  12 3.835 177   0     1
196  11 2.318 150   0     0
197  14 3.395 170   0     0
198  12 2.751 160   0     0
199  10 2.673 164   0     0
200  12 2.556 157   0     0
201  11 2.542 157   0     0
202  11 2.354 157   0     0
203  13 2.599 159   0     1
204  10 1.458 145   0     0
205  11 2.491 150   0     0
206  13 3.060 156   0     0
207  10 3.305 165   0     0
208  11 3.774 170   0     0
209  14 3.169 163   0     0
210  13 2.704 155   0     0
211  11 3.515 163   0     0
212  10 2.287 155   0     0
213  13 2.434 166   0     0
214  10 2.365  NA   0     0
215  13 3.086 171   0     1
216  12 2.868 157   0     0
217  10 2.813 156   0     0
218  12 3.255 168   0     0
219  10 3.413 168   0     1
220  10 2.975 160   0     1
221  11 3.223 164   0     0
222  11 2.606 165   0     0
223  11 3.169 159   0     1
224  10 2.358 150   0     0
225  12 2.347 156   0     0
226  10 2.691 170   0     0
227  11 2.827 159   0     0
228  14 2.538 180   0     0
229  10 3.050 152   0     0
230  12 3.079 152   0     0
231  13 2.216 173   0     1
232  12 3.403 157   0     0
233  12 3.501 164   0     0
234  11 2.578 160   0     0
235  13 3.078 168   0     1
236  12 3.186 170   0     1
237  11 2.081 160   0     0
238  11 2.974 157   0     0
239  13 3.297 165   0     1
240  10 2.250 147   0     0
241  12 2.752 161   0     0
242  11 3.102 163   0     1
243  10 2.862 155   0     0
244  13 2.677 170   0     1
245  11 3.023 171   0     0
246  11 3.681 173   0     0
247  13 3.255 169   0     0
248  10 2.356 154   0     0
249  12 3.082 161   0     0
250  13 3.297 165   0     1
251  11 3.258 160   0     0
252  11 2.362 155   0     0
253  11 2.563 160   0     0
254  13 3.331 166   0     0
255  12 2.417 155   0     0
256  12 2.759 156   0     1
257  11 2.953  NA   0     1
258  12 2.866 157   0     0
259  14 2.891 157   0     0
260  11 3.022 156   0     0
261  11 2.866 154   0     0
262  12 2.605  NA   0     0
263  13 3.056 160   0     0
264  12 2.569 160   0     0
265  11 2.501 157   0     0
266  13 3.785 160   0     1
267  13 2.449 160   0     0
268  10 3.073 168   0     0
269  10 2.688 157   0     0
270  11 2.689 156   0     0
271  14 2.934 163   0     0
272  10 2.435 165   0     0
273  10 2.838 160   0     0
274  12 3.035 157   0     0
275  12 2.714 166   0     0
276  10 3.086 157   0     0
277  12 3.519 166   0     0
278  12 3.341 166   0     0
279  10 3.038 165   0     1
280  10 2.568 161   0     0
281  12 3.001 161   0     0
282  10 3.132 151   0     0
283  13 3.577 161   0     0
284  12 3.222 155   0     0
285  11 2.822 157   0     0
286  11 2.140 154   0     0
287  14 2.997 164   0     0
288  11 3.120 155   0     1
289  11 2.562 159   0     0
290  12 3.082 164   0     0
291  13 3.152  NA   0     1
292  11 2.458 152   0     0
293  13 3.141  NA   0     0
294  12 2.579 160   0     0
295  11 3.104 171   0     1
296  11 3.069 165   0     1
297  18 2.906 168   0     0
298  19 3.519 168   0     1
299  15 2.635 163   0     0
300  15 2.679 168   0     1
301  15 2.198 157   0     1
302  19 3.345 166   0     1
303  18 3.082 164   0     0
304  16 3.387 169   0     0
305  16 2.903 160   0     1
306  15 3.004 163   0     1
307  17 3.500 157   0     0
308  16 3.674 171   0     0
309  15 3.122 163   0     1
310  15 3.330 174   0     1
311  16 2.608 157   0     1
312  15 2.887 160   0     0
313  16 2.981 168   0     0
314  15 2.264 160   0     1
315  15 2.278 152   0     1
316  18 2.853 152   0     0
317  16 2.795 160   0     1
318  15 3.211 169   0     0
319   9 1.558 135   1     0
320   9 1.895 145   1     0
321   8 1.735 137   1     0
322   8 2.118 154   1     0
323   8 2.258 147   1     0
324   7 1.932 135   1     0
325   5 1.472 127   1     0
326   9 2.352 150   1     0
327   9 2.604 156   1     0
328   7 2.578 159   1     0
329   3    NA 131   1     0
330   9 2.348 152   1     0
331   5 1.755 132   1     0
332   9    NA 166   1     0
333   9 2.725 150   1     0
334   8    NA 140   1     0
335   8    NA 145   1     0
336   8 2.004  NA   1     0
337   8 2.420 150   1     0
338   5 1.776 130   1     0
339   7 1.624 137   1     0
340   6 1.650 140   1     0
341   8 2.732 154   1     0
342   5 2.017 138   1     0
343   9 3.556 157   1     0
344   8    NA 138   1     0
345   6 1.634 137   1     0
346   9 2.570 145   1     0
347   8 2.123 152   1     0
348   8 1.940 150   1     0
349   6 1.747 146   1     0
350   9 2.069 147   1     0
351   8 1.962 145   1     0
352   9 2.715 152   1     0
353   9 2.457 150   1     0
354   9 2.090 151   1     0
355   7 1.789 142   1     0
356   5 1.858 135   1     0
357   5 1.452 130   1     0
358   9    NA 175   1     0
359   8    NA 160   1     0
360   8    NA 138   1     0
361   7 1.253 132   1     0
362   9 2.659 156   1     0
363   5 1.580  NA   1     0
364   9 2.126 157   1     0
365   9 2.964 164   1     0
366   7    NA 146   1     0
367   9 2.196 155   1     0
368   9 1.751 147   1     0
369   9 2.165 156   1     0
370   7 1.682 140   1     0
371   8 1.523 140   1     0
372   7 1.649 137   1     0
373   9 2.588 160   1     0
374   4 0.796 119   1     0
375   6 1.979 142   1     0
376   8 2.354 149   1     0
377   6 1.718 140   1     0
378   7 2.084 147   1     0
379   7 2.220 147   1     0
380   7 2.219 140   1     0
381   9 2.420 145   1     0
382   6 1.338  NA   1     0
383   8 2.090 145   1     0
384   8 1.562 140   1     0
385   9 2.650 161   1     0
386   8 1.429 146   1     0
387   8 1.675 135   1     0
388   8 2.069 137   1     0
389   6    NA 132   1     0
390   6 1.348 135   1     0
391   9 1.773 149   1     0
392   7 1.905 147   1     0
393   6 1.431 130   1     0
394   9    NA 164   1     0
395   9 2.135 149   1     0
396   6 1.527 133   1     0
397   8    NA 161   1     0
398   9 2.301 149   1     0
399   9    NA 163   1     0
400   8 1.759 135   1     0
401   6    NA 122   1     0
402   9 2.571 154   1     0
403   7 2.046 142   1     0
404   9 2.893 164   1     0
405   6 1.713 128   1     0
406   6 1.624 131   1     0
407   8 2.631 150   1     0
408   5 1.819 135   1     0
409   7 1.658 135   1     0
410   7 2.158 136   1     0
411   4 1.789 132   1     0
412   8 2.503 160   1     0
413   7 1.165 119   1     0
414   9 2.230 155   1     0
415   9 1.716 141   1     0
416   7 1.790 136   1     0
417   9 2.717 156   1     0
418   7 1.796 140   1     0
419   9 1.953 147   1     1
420   9 2.119 145   1     0
421   6 1.666 132   1     0
422   6 1.826 133   1     0
423   9 2.871 165   1     0
424   6 2.262 146   1     0
425   6 2.104 144   1     0
426   9 2.973 151   1     0
427   5 1.971 147   1     0
428   7 1.920  NA   1     0
429   9    NA 152   1     0
430   5 1.808 141   1     0
431   9 2.042 157   1     0
432   6    NA 126   1     0
433   9 3.681 173   1     0
434   8 1.991 151   1     0
435   8 1.897 141   1     0
436   8 2.016  NA   1     0
437   7 1.612 144   1     0
438   8 2.681  NA   1     0
439   8 2.010 140   1     0
440   8 1.744 133   1     0
441   9 2.076 154   1     0
442   8 2.435 151   1     0
443   8 2.303 145   1     0
444   9 2.246 161   1     0
445   9 3.239 165   1     0
446   9 2.457 156   1     0
447   7 2.056 137   1     0
448   8 2.226 145   1     0
449   9 2.833 156   1     0
450   6 1.715  NA   1     0
451   8 2.631 150   1     0
452   7 2.550 142   1     0
453   9 2.803 151   1     0
454   9 2.923 163   1     0
455   8 2.094 146   1     0
456   9 1.855 152   1     0
457   7 2.135 142   1     0
458   5 1.930 130   1     0
459   5 1.359 128   1     0
460   6 1.699 137   1     0
461   8 2.500 145   1     0
462   5 1.514 132   1     0
463   7 2.535 151   1     0
464   8 1.624 135   1     0
465   9 2.798 157   1     0
466   6    NA 135   1     0
467   9 1.869 145   1     0
468   4 1.004 122   1     0
469   6    NA 126   1     0
470   7 1.826  NA   1     0
471   8 1.657 142   1     0
472   5 2.115 127   1     0
473  11 2.884 175   1     0
474  10 2.328 163   1     0
475  14 3.381 160   1     0
476  11    NA 169   1     0
477  10 1.811 145   1     0
478  11 2.524 163   1     0
479  14 3.741 174   1     0
480  13 4.336 177   1     0
481  14 4.842 183   1     0
482  12 4.550 180   1     0
483  10 2.561 157   1     0
484  10 2.481 155   1     0
485  10 3.203 168   1     0
486  13 3.549 173   1     0
487  11 3.222 183   1     0
488  10 3.111 168   1     0
489  10    NA 160   1     0
490  10 2.246 154   1     0
491  10 1.937 157   1     0
492  10 2.646 152   1     0
493  11 2.957 164   1     0
494  11 4.007 170   1     0
495  10 3.251 168   1     0
496  13 4.305  NA   1     0
497  13 3.906 170   1     0
498  11 3.583 170   1     0
499  14 3.436  NA   1     0
500  11 3.058 155   1     0
501  10 3.007 157   1     0
502  10 3.489 169   1     0
503  14 4.683 174   1     0
504  10 2.352 156   1     0
505  11 3.108 164   1     0
506  13 3.994 170   1     0
507  12    NA 174   1     0
508  10 2.592 165   1     0
509  13 3.193 178   1     0
510  11 1.694 152   1     1
511  14 3.957 183   1     1
512  13 4.789 175   1     1
513  11 3.515 171   1     0
514  10    NA 166   1     0
515  11 2.463 164   1     0
516  11 3.111 171   1     0
517  10 2.094 149   1     0
518  11 3.977 179   1     0
519  10 3.354 160   1     0
520  13 3.887 171   1     0
521  11    NA 164   1     0
522  11 3.845 174   1     0
523  12 2.971 164   1     0
524  11 2.417 159   1     0
525  14 4.273 184   1     0
526  13 2.976 166   1     0
527  11 4.065 169   1     0
528  11 3.596 173   1     0
529  10 2.608 168   1     0
530  10 3.795 174   1     0
531  10 2.545 165   1     0
532  11 2.993 169   1     0
533  13    NA 173   1     1
534  10 2.855 164   1     0
535  11 2.988 178   1     0
536  11 2.498 152   1     0
537  11 2.887 159   1     0
538  11 3.425 166   1     0
539  10 2.696 168   1     0
540  14 4.309 175   1     1
541  11 4.593 175   1     0
542  14 4.111 180   1     0
543  12 1.916 154   1     0
544  10 1.858 147   1     0
545  10 3.350 175   1     0
546  10 2.901 151   1     0
547  12 2.241 163   1     0
548  13 4.225 188   1     0
549  12    NA 178   1     0
550  11 4.073 170   1     0
551  12 4.080 164   1     0
552  12 4.411 173   1     0
553  12 3.791 174   1     0
554  13    NA 171   1     0
555  11 2.465 152   1     0
556  12 3.343 173   1     1
557  10 3.200 165   1     0
558  12 2.913 163   1     0
559  13 4.877 185   1     0
560  12 3.279 179   1     0
561  10    NA 168   1     0
562  10    NA 133   1     0
563  12 3.751 183   1     1
564  10 2.758 166   1     0
565  10 2.201 154   1     0
566  10 1.858 150   1     0
567  10 1.665 145   1     0
568  12 4.073 174   1     0
569  13 4.448 175   1     0
570  13 3.984 180   1     0
571  12 2.304  NA   1     1
572  14 3.680 170   1     0
573  12 3.692 170   1     0
574  10 4.591 178   1     0
575  10 2.216 155   1     0
576  11 3.247 166   1     0
577  11 4.324 171   1     0
578  11 3.206 161   1     0
579  14 3.585 178   1     0
580  12 4.720 182   1     0
581  13 5.083 188   1     0
582  10 3.498 173   1     1
583  10 2.364 155   1     0
584  10 2.341 155   1     0
585  12    NA 160   1     0
586  11 3.078 171   1     0
587  11 3.369  NA   1     0
588  12 3.529  NA   1     0
589  10 3.127 157   1     0
590  11 3.320 166   1     0
591  11 2.123 165   1     0
592  14    NA 178   1     0
593  11 3.847 168   1     0
594  12 3.924 173   1     0
595  10 2.132 150   1     0
596  12 2.752 174   1     0
597  10 3.456 160   1     0
598  10 3.329 173   1     0
599  14 4.271  NA   1     0
600  12 3.530 163   1     0
601  11 2.928 166   1     0
602  12 2.332 145   1     0
603  14 2.276 168   1     1
604  10 3.110 164   1     0
605  11 2.894  NA   1     0
606  11 4.637 183   1     1
607  12 4.831 180   1     0
608  11 2.812 155   1     0
609  13 4.232 179   1     0
610  10 2.770 157   1     0
611  10 3.090 165   1     0
612  13 2.531 155   1     0
613  12 2.822 177   1     0
614  12 2.935 166   1     0
615  11 2.387 154   1     0
616  12 2.499 165   1     0
617  11    NA 170   1     0
618  11 3.280 168   1     0
619  11 2.659 163   1     0
620  12 4.203  NA   1     0
621  14 3.806 173   1     0
622  11 3.339 174   1     1
623  10 2.391 151   1     0
624  13 4.045 175   1     1
625  14 4.763 173   1     1
626  10 2.100 147   1     0
627  11 2.785 175   1     0
628  15 4.284 178   1     0
629  15    NA 180   1     1
630  19 5.102 183   1     0
631  16 3.688 173   1     1
632  17 4.429 178   1     0
633  15 4.279 171   1     0
634  15 4.500 178   1     0
635  17    NA 170   1     1
636  15 5.793 175   1     0
637  15 3.985 180   1     0
638  18 4.220  NA   1     0
639  17    NA 179   1     0
640  15 3.731 170   1     0
641  17 3.406 175   1     1
642  17 5.633 185   1     0
643  16 3.645 187   1     0
644  15 3.799 169   1     1
645  18    NA 170   1     1
646  16 4.070 177   1     1
647  17    NA 178   1     0
648  16 4.299 168   1     0
649  18 4.404 179   1     1
650  16 4.504 183   1     0
651  17 5.638 178   1     0
652  16 4.872 183   1     1
653  16 4.270 170   1     1
654  15 3.727 173   1     1
>

Using the continuous variable fev, I've run both Kolmogorov-Smirnov and Shapiro-Wilk tests. However, their outcome seems to conflict with each.

One-sample Kolmogorov-Smirnov test

data:  data1$fev
D = 0.049474, p-value = 0.09708
alternative hypothesis: two-sided

that let me lean for the normality assumption satisfaction, although in a very situation at limit, whereas the Shapiro-Wilk test doesn't:

Shapiro-Wilk normality test

data:  data1$fev
W = 0.972, p-value = 0.000000001751

How do you usually decide which test is the best to accept normality assumption? Which test or expedient do you refer to?

[Is normality testing 'essentially useless'?](https://stats.stackexchange.com/questions/2492/is-normality-testing-essentially-useless). Please search before posting. — Henrik, Feb 18 '21 at 12:14
I've edited the English slightly, although not the sentence containing "propend", where I am not confident of your meaning. — Nick Cox, Feb 18 '21 at 13:23
As others have flagged, this is a much discussed question. In broad terms, many consider the Kolmogorov-Smirnov test oversold and its continued discussion may owe more to respect for its first author, its generality and its elegance. More specifically, whenever parameters are estimated from the data, the standard procedure needs modification; I don't know R well enough to advise. The KS test is necessarily most sensitive at comparing the middles of distributions rather than their tails, precisely the opposite of what is of most concern in practice. — Nick Cox, Feb 18 '21 at 13:27
For your `fev` data medical statisticians would expect skewness from experience with such data and others would expect it on general grounds. I'll speak for those practitioners (from threads here, quite numerous, but I wouldn't want to say "most") who would never bother with any formal test but would carefully look at a normal quantile plot and consider working with say log or cube root transformation and/or using such a link in a generalised linear model. — Nick Cox, Feb 18 '21 at 13:30

score 1 · Accepted Answer · edited Feb 18 '21 at 13:24

If you have hundreds or thousands of data points, the output of normality tests is more likely to be that the data is not normally distributed. Because more data we have, more statistical power we have.

See following posts for related information.

Is normality testing 'essentially useless'?

Why is 600 out of 1000 more convincing than 6 out of 10?

Also make sure you specify mean and SD for ks test.

https://stackoverflow.com/questions/26715843/kolmogorov-smirnov-test-in-r

Normality assumption: how deal with them?

1 Answers1