3

I have a sample size of 4 or 3. Each sample is the difference between climate variables (Temperature, vapor pressure, wind, solar radiation, etc.) under two different conditions (variable value inside - variable value outside. I cannot assume normality. My sample and population are continuous. I want to know if these differences are significantly different from 0.

I would like to test if the mean is significantly different than 0. I am considering using a t-test with mean = 0 for the null. But this test, assumes normality.

The other test I am considering is the Wilcoxon rank-sum test, but it looks like it only compares two samples. Can I use it to test against a mean of 0?

What other tests are available for small sample sizes where parametric assumptions are not necessarily met?

Edit Purpose of my study

I have weather stations collecting data inside and outside low-tech greenhouses. I am testing to see if the differences between the weather station data inside and outside is statistically significant.

Because I have an unequal number of replicates inside and outside the greenhouses, I calculated the difference for each variable between each weather station inside each greenhouse and the weather station outside. I was hoping to test the significance of the differences from zero rather than the original weather station data. However in order to use the t-test, I need to transform some of my data or find another test.

phaser
  • 245
  • 1
  • 9
  • 2
    You need either strong assumptions or a strong result to test small samples. That makes it difficult to supply any kind of recommendation based only on the sample size. Perhaps you could explain more about your sample and the assumptions you might be able to make about it? – whuber Jun 20 '17 at 18:59
  • @whuber I am trying to describe my experiment without giving to much away. Let me know if you need more information. – phaser Jun 20 '17 at 19:21
  • 4
    You might find this thread to be of some interest: https://stats.stackexchange.com/questions/9653 . – whuber Jun 20 '17 at 19:44
  • 1
    If basic assumptions aren't met for standard tests, permutation or randomization tests are often a good alternative. These are frequently used to test difference of mean between two groups. Unfortunately with only 3 or 4 data points the number of permutations is very small making this no where near as good as if you had a larger sample. – Patty Jun 21 '17 at 00:53

1 Answers1

1

A permutation test is possible, but as stated in my comment your small sample makes significantly it less powerful. The basic idea is as follows:

We have 4 data points $(X_1,Y_1),...,(X_4,Y_4)$ and we wish to test whether $\mu_X = \mu_Y$ without assuming normality. The difference between sample means $\bar{X}-\bar{Y}$ will be our test statistic.

If our two groups do indeed have equal mean, then randomly assigning our data points too each group should not change this test statistic significantly. That is, we have 8 data points:

$Z_1,Z_2,...,Z_8$ where $Z_1=X_1,Z_2=Y_1,Z_3=X_2,...$ etc.

Randomly assign our labels of 'Group X' and 'Group Y' to this data set. ie, randomly pick 4 values of $Z_i$ and put them in group $X$, and then place the other 4 in group $Y$. We will then obtain a new permuted data set:

$(X_1,X_2,X_3,X_4)^*$ and $(Y_1,Y_2,Y_3,Y_4)^*$

Calculate our test statistic for this new data set: $\bar{X}^*-\bar{Y}^*$

Do this for every way you can permute your data. This will give you a collection of test statistics. Compare your original test statistics to this empirical distribution of test statistics. If it is 'too extreme' (ie. It's absolute value is in the highest 5% or 10% of those generated) then reject the null hypothesis the two variables have equal mean.

Due to your small data size the number of permutations possible is very small however, so you may wish to pursue a different test. Permutation tests also have some assumptions which you should also consider. I just figured outlining one approach would be useful to you.

Patty
  • 1,599
  • 1
  • 8
  • 21