2

I have been studying an organisation with 58 people with a number of Likert style questionnaires. The data are not random samples but rather incomplete census's. For the 6 questionnaires I managed to get between 49 & 57 respondents. However when I come to combine questionnaire data, the number can fall to say 43 subjects.

The responses to the questionnaires have been uniformly non-normal and generally leptokurtic. NP bootstrap regression on this number of subjects gives significant results, however I am unsure whether this enables me to say anything about the organisation more generally. If you have any guidance on the limits of what I can say with this sort of data it would be helpful.

Many thanks for any advice.

Andy W
  • 15,245
  • 8
  • 69
  • 191
user14470
  • 21
  • 1
  • Have you got any information on the people who did not answer and how that compares to those who did answer? – Peter Flom Sep 30 '12 at 11:52
  • A couple of people were largely absent in most of the questionnaires, but otherwise it was people being exceptionally away or ill when I was doing a particular questionnaire. The missing responses seemed roughly normally distributed by rank (I have not confirmed this though). – user14470 Sep 30 '12 at 14:37

2 Answers2

1

If the population size is 58 and the variance of responses is given as σ$^2$ then for a random sample of size n from a population of size N the variance for the mean of the sample is

(σ$^2$/n)(1-n/N)

Now 1-n/N is the finite population correction. In your case if the smallest sample size is 43 for a population of size 58 this finite population correction factor is

1-43/58= 1-0.74=0.26 and when n=57 it is 1-57/58=0.017.

Even though your sample is not random this finite population correction factor does show how much your variance is reduced because the sample size n is close to the population size N.

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • Many thanks for this. If I may, a related question. Many of the calculations of sample size use a response percentage which is based on a binary response. As my data is ordinal and those that respond select on a scale of 1-5, it is not immediately clear how to handle this. – user14470 Oct 01 '12 at 13:04
  • Check the site. I think the @Gung and a few others who are more expert than me on ordinal data have commented on using averages of ordinal data as though it were interval. Maybe Gung or Peter Flom will see this and comment. – Michael R. Chernick Oct 01 '12 at 16:29
0

Technically speaking, the classical statistics (the one which talks about significance, as opposed to e.g. Bayesian Statistics) requires the populations to be infinite regardless it may or may not make sense in the real world.

In your case, (whether you take bootstrap or straightforward approach) you are in fact taking into consideration infinite number of parallel realizations of your company based upon eighter normal model or the explicit bootstrap realizations. The significance tells you, that among this population a given relationship exists (with the given type I error). It is a leap of thought, the one that is commonly done in this field. [See: e.g. Bostad W.M. - Introduction to Bayesian statistics (2007), p. 5]

On a side, note that any classical statistical test assumes also that the variables come from i.i.d. distributions (independent and identically distributed). This may be problem in organization where exist many interpersonal mechanisms and interactions which effectively change (may it be unify or diversify) the way people answer your question(aire). So you should be aware, that however small, your sample may still be overreplicated.

Adam Ryczkowski
  • 2,092
  • 2
  • 18
  • 33