2

I would appreciate some guidance regarding the use of a t test vs $\chi^{2}$.

I am looking at some small set of demographic data (n only 13). I have divided up the sample into two sets, those with Body Mass Index (BMI) >= 25 (n = 5) and those with BMI < 25 (n = 8) and computed the means and standard deviations for the various attributes/data points, these are all numeric values (i.e, weight, height, etc).

If I want to determine if the respective resulting means for both groups are statistically different from each other I would have used a t test since these are quantitative values. However, a related similar paper I'm reading about this uses a $\chi^{2}$ test to assess this in table 1 (http://link.springer.com/article/10.1186/1471-2393-13-115/fulltext.html#Tab1). Is this a correct thing to do? Is this because by splitting the sample into two, you now have two categories, hence the application of the $\chi^{2}$? This seems incorrect. I am trying to summarize our data using R, for now I'm concerned about the numerical data (I'll have to summarize the categorical answers too).

My understanding is that I would use the t test for numerical data, and $\chi^{2}$ for categorical. It's been a while since I've had stats, but I'm reviewing, so any guidance would be appreciated.

(Also, it seems to be a consensus from reading messages here that a test of normality for the data would not make much sense with such small n, comments?)

I.e., the question is about the appropriateness of the test as used here, not the study/design itself.

Levon
  • 433
  • 7
  • 16
  • 3
    Don't dichotomize body mass index unless you've good reason to think that people with a BMI of 25.1 are going to have the same distributions of other variables as people with a BMI of 40; & different ones from those with BMIs of 15 to 24.9. See [here](http://stats.stackexchange.com/questions/68834/). – Scortchi - Reinstate Monica Jul 30 '14 at 16:37
  • @Scortchi Thanks for the link. The reason I'm splitting the sample is because I'm interested in the difference between the two groups of people in terms of physical activity level, nutrition etc. and hope to see if there are differences based on their BMI classification (>= 25 overweight/obese vs < 25 "normal"/underweight). I have collected this information and computed their means - I now want to know if those differences are statistically significant between these two groups. I would have expected to use a t-test for this, rather than a chi-squared test as the similar paper does. – Levon Jul 30 '14 at 16:55
  • 2
    If you think biological differences are better modelled by people's BMI *classification* rather than by their BMI, then go ahead. And do read [Dr Harrell's page](http://biostat.mc.vanderbilt.edu/wiki/Main/CatContinuous) - he's a medical statistician. – Scortchi - Reinstate Monica Jul 30 '14 at 17:57
  • @Scortchi It's not really a matter of what I think, this is a follow up study based on work that classified the population in this manner and it's not up to me to redefine this :-/ (I'm part of a group working on this). I'm happy to learn something new, but these comments, while useful, don't address my specific question above. I am not modelling biological differences, but looking at the behavior of two groups defined this way. – Levon Jul 30 '14 at 21:45
  • 1
    The table in the article you cite reports an N of 582 and you have an N of 13. With your small sample size, you cannot afford any loss of statistical power. The need to maximize power with a very small N argues for going with the t test approach. If you have many various attributes, you might worry about (or at least consider) experiment-wise alpha. – Joel W. Jul 31 '14 at 00:03
  • @JoelW. Ah, ok thanks. Since my sample is so small, and I may not be able to be certain that the values come from a normal distribution, would it make more sense to use the non-parametric Wilcoxon test? – Levon Jul 31 '14 at 02:20
  • Behaviour *might* differ noticeably depending on which side of a BMI cut-off people fall on - say if those classified as overweight get different medical advice from others. Nevertheless it would be foolish to try to analyse this without also controlling for the BMI values. – Scortchi - Reinstate Monica Jul 31 '14 at 11:24
  • 2
    Are you interested in whether there is a relationship between BMI and each of the many attributes? If so, have you considered calculating a correlation coefficient between BMI and each? Also, can you gather more data? – Joel W. Jul 31 '14 at 11:37
  • @JoelW.: That would be informative. Indeed simple scatter plots would be. – Scortchi - Reinstate Monica Jul 31 '14 at 11:58
  • @JoelW. The data set that has already been collected,so no new data. Eventually I want to examine any relationship between the BMI classification and behavior. My planned approach: Divide the sample,compute/compare the respective means,test for significance (and possible CIs). Compute the correlation coefficient (and its significance) between BMI and other attributes,create plots to visualize the data (to help guide/identify possible relationships worth investigating). Any suggestions within my constraints would be greatly appreciated. I have been using other related papers to guide me so far – Levon Jul 31 '14 at 12:24
  • @JoelW. Just to add, I plan to correlate BMI to the other variables w/o classification (ie splitting it up into two groups) also as part of exploring the data set and discovering any type of relationships present. – Levon Jul 31 '14 at 13:27

1 Answers1

1

Noting the major blunder in the paper you reference of discarding much information about BMI from the outset by dichotomizing its calculated values, a prudent reader should be alert to the possibility of minor blunders. The labelling of the "Difference" column in Table 1 as "test for significance chi squared" appears to be one such. Elsewhere it is stated that "Differences between groups were assessed using independent sample t tests or Mann Whitney U tests for continuous variables, and chi squared tests for categorical variables"†. The pooled-variance estimate of the standard error for the difference in mean ages of the two groups is

$$\sqrt{\frac{(n_1-1)s^2_1+(n_2-1)s^2_2}{n_1+n_2-2}\cdot\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}$$

$$=\sqrt{\frac{(196-1)5.1^2+(386-1)5.2^2}{196+386-2}\cdot\left(\frac{1}{196}+\frac{1}{386}\right)} \ \text{years}$$

$$= 0.4532 \ \text{years}$$

so the cited p-value implies a difference in sample means of 0.1096 years if a pooled-variance, two-sided t-test were used, in agreement with the tabulated ages of 30.0 & 29.9 years for each group. Where medians rather than means are reported, in Table 3, the use of the Mann–Whitney U test is clearly indicated in the footnotes.

† The chi-square test the authors refer to is Pearson's chi-square test for association between categorical variables.

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248