1

I run into a conundrum when determining whether a small sample is appropriate for a t-test. I know that if I meet the assumptions, a t-test produces approximately correct inference with any sample size, hashed out really nicely here:

Is there a minimum sample size required for the t-test to be valid?

What would be the recommended approach if I have a small sample, and I'm unsure of the normality assumption? Because of the small sample, it's difficult to run any normality tests...perhaps the best answer is to use my domain knowledge of the variable to make an educated guess as to whether it's normal? Or perhaps a non-parametric test is more appropriate?

Note: I am aware that this small sample will lead to low power. I am more concerned about my tests being approximately correct.

Alex
  • 487
  • 2
  • 8
  • What is the null hypothesis you want to query? – Michael M Jul 30 '20 at 15:41
  • The null hypothesis will likely be that the means are equal. – Alex Jul 30 '20 at 15:59
  • 2
    Then I'd tend to a permutation t test. – Michael M Jul 30 '20 at 16:18
  • 2
    Here one of our favourite posts (about normality pretests): https://stats.stackexchange.com/questions/2492/is-normality-testing-essentially-useless – Michael M Jul 30 '20 at 17:35
  • That's the reference I linked. I didn't see anything in there that addressed my question but please point it out if I missed it. – Alex Jul 30 '20 at 20:16
  • 4
    I'd advise against testing assumptions (it answers the wrong question), and more strongly advise against doing it on the same data you want to use those assumptions for (e.g. it screws up the properties of the subsequent test). If you can't reasonably make the assumption (e.g. with domain knowledge) then unless the test is robust to it, (BOTH in terms of level and power) it's probably better not to assume it at all. On the other hand when sample sizes are extremely small, there can be problems with nonparametric tests (e.g. there may be a lack of suitable significance levels) – Glen_b Jul 31 '20 at 04:19
  • 1
    @Glen_b thank you. This is the type of answer I was looking for. – Alex Jul 31 '20 at 13:19
  • @Alex Essentially everything I said in that comment is given in a number of answers already on site. – Glen_b Jul 31 '20 at 17:38

0 Answers0