What are the assumptions behind each of the following types of tests (i.e. what are the prerequisites to running each test).
I really am getting confused seeing what would warrant one over the other given criteria like (normality, variance, sample size etc.) Any help would be appreciated.
(Note: This is for a class assignment)
While this doesn't necessarily cover the assumptions as required for the assignment*, it may well be worth laying out some things briefly for these common tests (though in most cases, a judicious use of the search function here on CrossValidated will yield the required information).
*(for which see your notes)
There are two main questions to address:
(i) What assumptions are necessary to derive the distribution of the test statistic under the null hypothesis
(ii) Which of those assumptions are more critical or less critical?
I should probably also add some discussion on the inadvisability of performing these tests only after performing hypothesis test of one or more of their assumptions, but briefly, it seems to be fairly clear that the tests done conditional on such preliminary testing do not have their nominal properties, and several papers advise that when there's no clear reason to think they're satisfied a priori, it's generally better to simply do whatever would be appropriate when they are not satisfied. [There isn't an option for you, though, since you're required to do so.]
This is a test for paired (matched) samples when applied to the differences (that the population mean pair difference is zero); it also applies when testing a single sample against some hypothesized mean.
Assumptions:
a) independence across pair-differences (not within pairs; they're dependent)
b) constant variance of pair-differences
c) normality of pair-differences
The more critical assumptions are independence and constant variance.
It's difficult to test independence without a specific form of dependence to check for. If you have observations over time, obviously one should be concerned about dependence over time.
Equality of variance across pair differences is tricky to check for without a specific alternative (one possibility is to see whether absolute size of difference is related to the average of the two measurements - e.g. to check whether ${|y_i - x_i|}$ is related to ${(y_i+x_i)}/2$, though the $\frac{1}{2}$ could be omitted); that's one reasonably common way in which it might tend to be violated.
The normality assumption matters in small samples, and much less so in large samples; in very large samples, you may be able to just about ignore normality as long as the other assumptions hold.
2. Sign test
This is another paired/matched samples test that is also suitable as a one-sample test. It is basically a test of the null hypothesis that the population median of the pair-differences is zero. If suitable additional assumptions are made it can also be suitable as a test of means (specifically, if the assumption that the population mean is the same as the population median in the distribution of pair differences).
Main assumptions:
a) independence across pair-differences (not within pairs; they're dependent)
b) continuous distribution of pair differences; this avoids the issue of the difference being exactly zero (though suitable adjustment to the procedure can be made if it's not the case)
Again, independence is important, but it's difficult to test without a specific form of dependence. If you have a time series, obviously one should be concerned about dependence over time.
Null hypothesis under the assumptions yields a binomial distribution for the number of positive differences, but the normal approximation may be used if sample sizes aren't small.
This is the two-sample t-test with the Welch-Satterthwaite approximation to the degrees of freedom chisquare approximation for the unpooled variance estimate. It is for testing equality of population means for independent samples.
Basic assumptions:
a) independence both within and between groups
b) common normal distributions within each population, but not necessarily equal variance across them
Moderate non-normality is not particularly crucial in moderate samples, and hardly matters in large samples. It can matter in very small samples.
This is the two-sample t-test with the additional assumption of equality of variance. It is for testing equality of population means for independent samples.
Basic assumptions:
a) independence both within and between groups
b) equality of population variance across groups
c) common normal distributions within each population
Moderate non-normality is not particularly crucial in moderate samples, and hardly matters in large samples. It can matter in very small samples. As before independence and equality of variance matters more, but in equal-sized samples, the equality of variance matters less.
5. Two sample Proportions test
This is for testing equality of proportions between two independent samples.
It assumes:
a) independence, and
b) (within groups), constant success probability.
There are two versions, the pooled and the unpooled versions (see the above link which includes both in the list of common test statistics). Either is suitable under the null hypothesis, but you should at least be aware which one is being performed.
In large enough samples (large enough to apply the normal approximation, say), moderate deviations from constant success probability assumption is less crucial than independence.
6. F test for equality of variance
This is for testing equality of variance in two samples.
It assumes:
a) independence (within and across groups)
b) normality
It is very sensitive to both assumptions and is generally not recommended. There are better choices.
There are some situations when you might need to decide between 1. and 2. with paired samples (watch out for the differences between them), since none of the others are paired. You might need to decide between 3. and 4. (hint for people in a conundrum: if you're in doubt about whether the variance might differ substantially, use the unpooled one, though if sample sizes are equal the decision is less critical). 5. and 6. really aren't competing with anything in that list; if you're testing independent proportions, only one of the above tests really fits; similarly if you're testing variances, only one of the tests in the list fits.
