t-test for differently-sized sample sets

Question

First, Please understand my lack of English skills.

I have a problem dealing with statistical hypothesis testing.

I'm in a situation to evaluate two independent samples t-test. There are two different-sized sample sets and these sets are sampled using method A & B (for method A, $n= 100$, and for method B, $n= 1000$). I collected data for 22 days.

I had two questions about people who used method A vs. method B to purchase the same item.

(Number of purchases) : Did people using method A purchase more of the item than people using method B? In this case, how can I perform a reasonable comparison? My problem is that there are 10 times more transactions from method B than from method A.
(cumulative sales aspect) : Did the people using method A spend more money on the item than people using method B? How can I perform a reasonable comparison in this case?

Use a difference of proportion test like [Fisher's exact test](https://en.wikipedia.org/wiki/Fisher%27s_exact_test). — Carl, Oct 10 '18 at 02:43
In order to get the best answer, I think it would be helpful if you could describe the data a little more. For example, *number of purchases*: is this always e.g. 0, 1, or 2 per person, or is there a wide range of possible results? Do you know the underlying distribution of this data? Or can you share a histogram for each group? — Sal Mangiafico, Oct 12 '18 at 17:42
Also, because you have reasonably large sample sizes, you are likely to find a significant *p*-value from a test, even if the effect size is small. Remember to take into account the effect size and what this means practically for the differences between the groups. — Sal Mangiafico, Oct 12 '18 at 17:44

score 0 · Answer 1 · answered Oct 10 '18 at 04:07

0

The general rule of thumb is that once $n$ gets above 30, it is reasonable to approximate it with the normal distribution. So 100 is large enough, and 1000 is way beyond the minimum.

If your null hypothesis is that the mean and variance are the same, and the alternative hypothesis is that the means and variances are different, then take the difference between the sample means and divide by the observed standard deviation, and use that as your $z-value$. To find the observed standard deviation, take $\sqrt{\frac {s_1^2}{n_1}+\frac {s_2^2}{n_2}}$, where $n_1$ and $n_2$ are the sample sizes and $s_1$ and $s_2$ are the sample standard deviations.

See https://www.stat.colostate.edu//~vollmer/stat307pdfs/book_ch18.pdf

If your null hypothesis is that the mean and variance are the same, and the alternative hypothesis is that the means are different but the variance is the same, then you can take $s_1=s_2=s$ where $s$ is the sample standard deviation over all the samples. Then you can factor out $s$ to get $s\sqrt{\frac {1}{n_1}+\frac {1}{n_2}}$.

BTW, the 't' in "t-test" is lowercase.

answered Oct 10 '18 at 04:07

Acccumulation

3,688
5
11

It is the difference of proportions that is important here. A Fisher exact test does not require approximations. – Carl Oct 10 '18 at 07:41
@Carl Fisher is used for classification. This is testing means. – Acccumulation Oct 11 '18 at 13:33
The question implies testing of whether the rates, not the amounts, are different. – Carl Oct 11 '18 at 21:53
1

@Carl Although the question isn't completely clear, #2 explicitly asks to compare *sales amounts*. In light of that, it's natural to understand #1 to compare *sales counts.* Neither is a proportion or rate. – whuber Oct 11 '18 at 22:00
@whuber I know, and I also edited the question to begin with and it depends how it is interpreted. I did the best I could with it. If you understand "people" to be taken as individuals, and the question as to the likelihood of an individual outcome then it is a rate question, and it is the only question that makes sense to me. Why would anyone care to know about group purchases? Why would one not adjust for population size to analyze individual behaviour? Note OP says "how can I perform a reasonable comparison? Because samples from method 'B' are 10 times more than samples from method 'A'" – Carl Oct 11 '18 at 22:08
@whuber I think this is a translation problem. In English, people is a collective noun. One translates *ludzie* from Polish to English as *people*, however, *ludzie* is plural, and *people* is collective, i.e., singular. I do not know what OP's native language group is, but I would point out that collective nouns are the exception, not the rule. I think in this case the better translation of meaning would be *person's* and not people. If you want, we can ask for clarification. – Carl Oct 11 '18 at 22:48
@Carl I'm not clear on what you're saying. Are you saying the OP is asking whether the percentage of people using method A who purchased C is different from the percentage of people using method B who purchased C? I read it as they all purchased C, but some people purchased C more. – Acccumulation Oct 11 '18 at 22:57
@Acccumulation Some research: OP's handle is myeonggyu, a famous Korean actor. *People* translates into Korean as 사람들, which is plural, not singular collective. Thus, when OP says *people* in English, OP should likely be understood as saying *person's* from the Korean perspective. For example, 신속히 완쾌하십시오. ...의 모든사람들이 사랑을 보냅니다. Translates as *Get well soon. Everybody here is thinking of you.* – Carl Oct 11 '18 at 23:05
@Carl That does not appear to address my question, and it's not clear what relevance at all it has. Also, using the apostrophe to denote plurals is not correct English. – Acccumulation Oct 11 '18 at 23:14
@Acccumulation In other word, Yes. I think OP is asking about percentage because of 1) linguistics and 2) common sense. – Carl Oct 11 '18 at 23:15
@Acccumulation What? Person's is the correct form for *possessive case* in English; it is singular not plural. Good grief. In means *of a person* not *people*. – Carl Oct 11 '18 at 23:19
@Acccumulation And... I did answer your question. It just took two comments to do it. Look, I am trying to help, politely, in fact. – Carl Oct 11 '18 at 23:32
Please clarify the question. Do you want a per person answer or a gross amount answer for groups of people? – Carl Oct 11 '18 at 23:59
@Carl So you think we should understand `I had two questions about people who used method 'A' to purchase an item 'C' compared to people who used method 'B' to purchase 'C'.` as meaning `I had two questions about a person's who used method 'A' to purchase an item 'C' compared to people who used method 'B' to purchase 'C'.` and `Did the people using method 'A' purchase more item 'C'` to mean `Did a person's using method 'A' purchase more item 'C'`? That doesn't make sense. – Acccumulation Oct 12 '18 at 02:42
Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/84369/discussion-between-carl-and-acccumulation). – Carl Oct 12 '18 at 07:00
1

I would like to take exception to the first line in the answer: *The general rule of thumb is that once n gets above 30, it is reasonable to approximate it with the normal distribution.* First, that's not a great description of what the central limit theorem (CLT) says. Second, that *n=30* rule of thumb is not really reliable. The n has to be larger for, for example, very skewed distributions. In general, I wouldn't advise using a t-test or z-test without understanding, or looking at, the underlying distribution of the data. – Sal Mangiafico Oct 12 '18 at 17:37
@SalMangiafico I wasn't speaking of the CLT in general, but t-tests with high df, although I could have been more clear. Given that the OP has decided to use a t-test, a df of 100 means that it's pretty much normal. – Acccumulation Oct 12 '18 at 18:32
@Acccumulation, I actually don't understand your response. What's normal? I assume you mean the distribution of the sample means. But n=100 isn't necessarily large enough to ensure this. It depends on the population distribution. – Sal Mangiafico Oct 12 '18 at 18:54
@SalMangiafico The t-distribution for degrees of freedom > 30 is very close to the normal distribution. – Acccumulation Oct 12 '18 at 19:38
@Acccumulation, I see. Your answer is contingent on the OP's plan to use *t*-test. I suppose my point comes down to the fact that without knowing more about the data, we don't know if the OP's plan is the best approach. – Sal Mangiafico Oct 12 '18 at 19:50
FWIW, https://stats.stackexchange.com/questions/69898 provides a real-world example of the failure of the $n\gt 30$ rule of thumb. One might want to be a little cautious with sales amounts, which potentially can be extremely skewed. (cc @Carl) – whuber Oct 12 '18 at 21:46
@whuber Wilcoxon-Mann-Whitney of gross and/or relative (=gross/{$n_1,n_2$}, measurements depending on what this unclear question is asking. The question merits closure for lack of clarity. – Carl Oct 12 '18 at 23:09
@Carl I have not voted to close because I find the key phrases "number of purchases," "more of the item," and "spend more money" to be sufficiently clear. – whuber Oct 12 '18 at 23:35

t-test for differently-sized sample sets

1 Answers1