1

I have data that includes the number of students in a class and the percentage of that group who achieved a preset pass level in a standard test. I have this data for a number if different schools in two population samples, about 30 schools in each. The class sizes differ considerably, so it seems take sense to use the percentage already given when calculating the t-test.

But I also know that percentages shouldn't be averaged. I could calculate the number of students from the data given, but this does not reflect class size, which seems important. The percentage "automatically" reflects the weighting of class size. Any advice or thoughts about this problem appreciated.

Example data to illustrate problem

No students    percent passed     calculated no passed

28                7%                2

79                7%                6

28               51%               14

58               50%               29

Thanks Tim

Peter Ellis
  • 16,522
  • 1
  • 44
  • 82
Tim Rowan
  • 11
  • 1
  • 1
  • 2
  • 3
    What are you trying to learn about, Tim? You mention "two population samples." This suggests you might not be interested in deciding whether two particular schools differ (obviously there are some huge differences even in your small example) but perhaps you would like to know whether the two populations have significantly different distributions of passing proportions? – whuber Jul 23 '12 at 16:48
  • 2
    Like @whuber, I think we need more information. But as a heads-up, I suspect you may ultimately need to use a multilevel generalized linear model. If you are not familiar with these, you could get an overview from some answers I have recently given, [here](http://stats.stackexchange.com/questions/20523/30909#30909) (for a little about generalized linear models), & [here](http://stats.stackexchange.com/questions/32419/32421#32421) (for a little about the multilevel options for GLiMs). However, if you are unfamiliar w/ these, you will probably need to work with a consultant. – gung - Reinstate Monica Jul 23 '12 at 17:05
  • I want to check whether the sample mean of to groups is drawn from the same population, using a t-test as the significance test. Not sure if this is how to do it? Tim – Tim Rowan Jul 23 '12 at 18:52
  • 1
    @TimRowan, in your comment and question you refer to "mean", what is this the mean of? Is it the mean pass rate of individual, or schools? or something else (eg average score on the original test). – Peter Ellis Jul 23 '12 at 20:16
  • Okay but the OP sounded like he wanted to do a simple t test comparing proportions. But my answer is defective because as Peter points out this is more complicated than would be accomplished by the tests I am suggesting. I think I should have looked at parts of the question more carefully and I am going to graciously drop my answer and apologize to my friends Gung and Huber for my hasty comments. The only way I could see a t test being remotely possible would be the scenario I imagined. Withdrawn. Those downvotes were well administered! See I can recognize my flaws. – Michael R. Chernick Jul 24 '12 at 00:07
  • First a quick thank you to you all for your help. This is more complex than I thought! Or I haven't explained it properly... I want to apply a t-test to compare two normally distributed samples. I am happy with this part and with the procedure. The only wrinkle in this case is that the variable of interest is expressed as percentage in the data set. What are the implications of this to the validity of the t-test? Do I need to calculate the underlying numbers (as per my example data)? If the answer is yes, does the problem (that I am calling 'weighting') have any relevance? How solved? Tim – Tim Rowan Jul 24 '12 at 03:54
  • @PeterEllis the mean I refer to is the mean of the dependent variable, which is performance in a test. The percentage expresses the proportion of those who passed. I have other percentages, such as the proportion who achieved over 80 percent etc. This is class level data and I don't have individual student marks. – Tim Rowan Jul 24 '12 at 04:25
  • The dependent variable is a score. It too is measured in percentage. Sowe should be careful not to confuse student test percentage with percentage of students passing. I don't see what are the normally distributed variables and what are the two groups. All i see is the number and percentage that pass the test at each school. The answer to this question depends on what role the proportions are playing in the analysis. If you are comparing proportions you don't need a t test. You could do a test for independence in a contingency table. – Michael R. Chernick Aug 23 '12 at 21:14
  • If it has to do with combining groups and taking averages than a t test test on the average over the combined samples should be okay. – Michael R. Chernick Aug 23 '12 at 21:14

1 Answers1

8

There are several issues here.

Q1. Can a t-test be used with data that is a percentage?

A - Strictly speaking a t-test should be used when the underlying population has a normal distribution and you have to estimate the mean and variance simultaneously. Your percentage pass rates are not going to normally distributed, but they may well be close enough that no harm is done by using a t-test. This is unlikely to be the big obstacle.

Q2. What is the right way to aggregate the percentage pass rates into an overall pass rate?

A - you should use a weighted average of the percentages, with the weights being the class sizes. This will give you a percentage (for each of your two groups of classes) that is equivalent to their overall individual percentage pass rate. There are several equivalent ways of calculating this; one is by a weighted average, the other is (as you have started doing) converting each class score into a number of passing students and a total number of students, summing these numbers and calculating the final percentage.

There are complications in estimating the variance to use for a t-test but I will ignore these because of Q3 below:

Q3. (implied) - is this procedure as a method of comparing pass rates between two groups appropriate?

A - No. The problem is that there are known to be very significant class (teacher) and school effects in educational performance. No simple t-test can take these into account, and a t-test will give you a big chance of showing something as significant when it isn't. Basically, the data in these grouped situations is not worth as much as its sample size suggests. This is because 30 students in one class are not really 30 independent data points - they may all be reflecting skill of the teacher (even putting aside other important known environmental variables, and their individual socio economic background). The accepted way to do the sort of inference you want is with a multi-level model, a special case of a mixed-effects model, which unfortunately is much harder to implement than a t test, but will give you much more reliable results.

Peter Ellis
  • 16,522
  • 1
  • 44
  • 82