2

I have three sets of data I am working with. The N of each are all vastly different and the means are also very different. I obviously don't want to interpret it so black and white because I know the sample sizes of each makes a difference. My data have the following descriptive statistics: 

     A            B            C
mean  3.04        3.51          6.60
SD   21.82768985  8.426020509  12.49522047
n    40823        10152        320426

How should I interpret them and how should I draw conclusions and insight?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
wizkids121
  • 171
  • 5

2 Answers2

1

The answer to this question depends on intent. If your intent is description, a mean is a mean, whether it is over 2 values or 200. However, things change a bit if your intent is one of inference. That is, in the context of a statistical test, you are using your sample means as a point estimate of the population means. When you conduct a statistical test, you are asking whether or not the population means are likely to be different, given your sample mean point estimates (and calculated standard error). In such inferences cases, you care about precision, and to a limit, larger sample sizes will give you more precise point estimates. It would help to know exactly what your N's are. For example, perhaps your group N's are 1000, 5000, and 100000; in the presence of normally distributed data, these could be trusted and compared meaningfully despite the large difference in N. If you are working with far fewer numbers, there must be some caution.

If you must interpret your three means with very large differences in N, my advice is to state explicitly what the N is (and probably standard deviation as well), so that the reader can judge for themselves whether or not your means are to be trusted.

HEITZ
  • 1,682
  • 7
  • 15
  • Thanks for the response. Okay, so basically we have three kinds of subscriptions models. For the purposes of righting it here, we have versions A, B, and C. A being the most comprehensive, C being the most basic. I have a sheet of data. What that sheet of data tells me is, using our mobile site, if someone hit a certain button that drives certain engagement. So here is a simplified look of what I’m dealing with: Subscription A| Total Subscribers: 40823|# Times Engagement Button Clicked: 124199 B|101528|356706 C|320426|2113399 – wizkids121 Aug 28 '15 at 04:22
  • With only those data, you can say very little. You would need the variability as well to conduct any kind of statistical test. – HEITZ Aug 28 '15 at 04:33
  • well that's the aggregate of it. It's about 450000 rows of data. – wizkids121 Aug 28 '15 at 04:35
1

The fact that you have groups with different sample sizes does not have any direct effect on whether it is reasonable to believe that the populations (e.g., their means) differ. To assess whether you might typically get group sample distributions that differ like yours do if the groups really were identical, we can perform a hypothesis test. A good statistical hypothesis test, such as the $t$-test, will appropriately take the two sample sizes into account. The $t$-test assumes:

  1. that the data are independent,
    (I cannot verify that and the validity of any conclusions based on the given descriptive statistics could be threatened if the data are not independent.)
  2. that the variances are equal,
    (This is not plausible from the information given, but the Welch version of the $t$-test can be applied, so we needn't be too concerned.)
  3. that the data within each group is normally distributed.
    (I can't tell if that's true from what is given, but the point of that assumption is to guarantee that the sampling distributions of the means would be normal, and hence that the sampling distribution of the $t$-statistic will be $t$. Your datasets are so large that that should be the case even if the data are not normal. Thus, we needn't be too concerned here either.)
  4. The $t$-test does not assume that the sample sizes are the same.

Based on these considerations, we should be comfortable conducting a $t$-test to compare any two of your groups. Here is an example (conducted with R): 

library(BSDA)  # we need this package for the tsum.test function
# test if C > B
tsum.test(mean.x=6.60, s.x=12.49522047, n.x=320426, 
          mean.y=3.51, s.y=8.426020509, n.y=10152,
          alternative="greater", var.equal=FALSE)
# 
#         Welch Modified Two-Sample t-Test
# 
# data:  Summarized x and y
# t = 35.726, df = 11613, p-value < 2.2e-16
# alternative hypothesis: true difference in means is greater than 0
# 95 percent confidence interval:
#  2.947723       NA
# sample estimates:
# mean of x mean of y 
#      6.60      3.51
gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • Thank you for that! So now I'm curious to know, what is that telling me? – wizkids121 Aug 28 '15 at 17:11
  • The probability of getting means that differ as much as yours do, if the true population means were identical, is essentially 0. It is unreasonable to believe that C isn't > B. – gung - Reinstate Monica Aug 28 '15 at 17:16
  • 1
    @gung: Yes, but here I would ask the OP another question: What is the nature of this data? with such huge sample sizes it is a bit difficult to trust independence!, the data probably would have some sub-structure that should be accounted for, like time order, multiple laboratories, whatnot? we do not know. – kjetil b halvorsen Aug 28 '15 at 17:35
  • @kjetilbhalvorsen, you're right; I was a bit too cavalier. I have changed what I said about independence. Let me know if you think it needs more. – gung - Reinstate Monica Aug 28 '15 at 18:14