5

I am using the package compute.es in R to calculate effect size from a $t$-test. The $t$-test is marginally significant with $p=0.05657$.

When I run tes(t=2.3563, n.1=4, n.2=4) I get the following output (trimmed):

Mean Differences ES:  
d [ 95 %CI] = 1.67 [ -0.34 , 3.67 ]
var(d) = 0.67
p-value(d) = 0.09
U3(d) = 95.22 %
CLES(d) = 88.06 %
Cliff's Delta = 0.76

My question is what is the meaning of p-value(d) = 0.09 in this case? That is, what is the null hypothesis of the Cohen's $d$? And if it is the same as in the $t$-test, then why are the $p$-values different?

amoeba
  • 93,463
  • 28
  • 275
  • 317
Nadia
  • 51
  • 1
  • 2

1 Answers1

7

Cohen's d is a measure of the standardized difference in means.

(mean($X_1$)-mean ($X_2$))/sigma

So the null hypothesis tests whether this standardized difference is equal to zero. This is different from the original null hypothesis which tests whether the non-standardized difference in means is equal to zero.

It might help to see how Cohen's d can be calculated using t. You can find the details in the compute.es package documentation (http://cran.r-project.org/web/packages/compute.es/compute.es.pdf):

t = d$*$sqrt($n_1$*$n_2$/($n_1$ + $n_2$))

rearranging will give you d.

[Addition to the above:]

The d = 1.67 tells you that the difference between the two groups is about one and two-thirds of a standard deviation. The original p=0.05657 was calculated for the non-standardized difference in means. The t-statistics for this difference follows a 'central t-distribution' - that is, it is symmetric around 0. 'Central t-distributions' have one parameter: the degrees of freedom.

The difference with the effect size is that the t-statistic for it is non-centrally distributed (it is not symmetric around 0). 'Non-central t-distributions' have two parameters: the degrees of freedom and a 'non-centrality' parameter. Rather than getting into further details, you can find a straightforward introduction here:

Cumming, G. & Finch, S. (2001) A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 633-649.

They give a readable account of why the effect size has a non-central t-distribution on pages 549-551.

Also: you asked for an explanation of the p-value. P-values depend on the distribution of the test statistic which I've addressed in the above (the details of which are in the reference). Hopefully that helps! If you want an explanation of what a p-value is generally, then likely that would require a different post and has probably been asked elsewhere.

Jordan Collins
  • 520
  • 2
  • 7
  • 2
    I think the OP wants to know why it doesn't give the same p-value as the t-test, despite the fact it's testing the same $H_0$. – Jeremy Miles Mar 04 '15 at 21:15
  • yes, I am looking for the explanation about the p-value. The explanation for this function in the package reads: Converts a t-test value to an effect size of d (mean difference), g (unbiased estimate of d), r (correlation coefficient), z (Fisher’s z), and log odds ratio. The variances, confidence intervals and p-values of these estimates are also computed – Nadia Mar 05 '15 at 03:57
  • 1
    Although the original answer did not address the p-value discrepancy, it did address the explicit question of what the null hypothesis was. It did not deserve a downvote. – gung - Reinstate Monica Mar 05 '15 at 15:19