3

Background:

I have a sample that is very left skewed. The mean of this sample is 4.01 (min = 1, max = 5). I would like to know whether or not a subset of this sample is significantly different than the sample as a whole.

My use case:

I want to take a portion of this sample (344 of the 1140 responses) and use Welch's t-test to determine whether the mean of the subset is significantly different than the mean of the entire sample.

My questions:

  1. Am I correct in assuming that significance should not be measured (at least not using Welch's t-test) if your data is extremely skewed?
  2. Is there a way to determine statistical significance between a sample and a subset of that sample when the data is skewed?

All of my computations are in php, as I am a web developer. So if your answer involves using some sort of statistical software magic I won't have a clue what you're talking about...thanks in advance!

Community
  • 1
robin_sparkles
  • 33
  • 2
  • Phrases like "statistical software magic" are unhelpful. Does that mean you're unprepared to run a simple command in any software, for example? How skewed are the data? Is the distribution also discrete? (can you show some display?) Is it only shift in mean you care about, or is some generic sense of "larger" all that matters? – Glen_b May 14 '15 at 23:32

1 Answers1

0

Question 1:

Even if your population is highly skewed the distribution of your sample means may not be for a sufficiently large sample size. You can try this out by making multiple draws from your initial sample and plotting them.

This question may be useful in understanding under what circumstances the t-test isn't applicable. t-test on highly skewed data

Question 2:

It really depends on what you want to compare, a standard goto would be the (http://en.wikipedia.org/wiki/Mann–Whitney_U_test), but you are violating the independence assumption of the two populations. There is also simulation which may work, but it is a bit awkward due to the dependence again.

To answer this better you might want to give some feedback on what your goal with this test is.

Community
  • 1
Jonathan Lisic
  • 1,342
  • 7
  • 16
  • Thanks, @Jonathan. Unfortunately, I have no way of collecting a larger sample size. The survey from which I'm getting my data is now closed. Although, the sample size does represent a significant portion of the population (the population being all employees in this particular company). The _goal_ of my test is to determine whether the mean of the subset is significantly gt or lt than the mean of the entire sample. I will review the other post you linked, as I think that may offer a pretty good summary of what I'm looking for. – robin_sparkles May 14 '15 at 02:45
  • I'm just a bit confused about why you are trying to subsample, since you already have a larger sample. Are you trying to determine sample size for a future sample? – Jonathan Lisic May 17 '15 at 02:13