4

I browsed through several papers and discussion but I am still not sure what is correct.

I conducted two studies on the same topic using similar methods and the same design (independent t-tests). Using R I ran a meta-analysis on both studies based on their Cohen's d. I decided to do a fixed-effect analysis, however the output using metagen of the meta package provides both-fixed and random-analyses. Here is my output

          95%-CI                %W(fixed)  %W(random) 
1 0.4111 [-0.0299; 0.8520]      31.3       31.3 
2 0.2577 [-0.0398; 0.5553]      68.7       68.7

Number of studies combined: k = 2

                             95%-CI          z      p-value 
Fixed effect model   0.3057 [0.0591; 0.5524] 2.43   0.0151   
Random effects model 0.3057 [0.0591; 0.5524] 2.43   0.0151

Quantifying heterogeneity:
tau^2 = 0; H = 1.00; I^2 = 0.0%

Test of heterogeneity:
Q d.f.  p-value
0.32    1   0.5721

Details on meta-analytical method:
- Inverse variance method
- DerSimonian-Laird estimator for tau^2

Why is it that the random and the fixed effect model show the same results? Is this typical or due to only using two studies?

Having an tau^2 = 0 and H = 1, does this mean that the effects are homogeneous or is it due to fixing them for fixed effect model?

And does having an insignficant heterogeneity test mean that the effects or homogeneous or is this test reporting something that does not go along with the tau^2 and H data?

I am happy for any hints on how to fully understand how heterogeneous the effects are. Thanks in advance

enter image description here

klonkklonk
  • 51
  • 2
  • Please type your results in as text (someone will come along afterward to help you format it, if you don't know how) instead of posting an image of it. Images can't be read by screen readers. – gung - Reinstate Monica Aug 04 '17 at 12:45
  • thanks for pointing to that. I didn't really know how to format it correctly but figured it out now – klonkklonk Aug 04 '17 at 13:27
  • How do you compute .4111 and .2577 ? What is 1, 2 ? and .3057 for each ( fixed and random model). Let there be a clear idea in your question. You are interested in overall effect size or difference in modelling. –  Aug 05 '17 at 10:35
  • So .4111 and .2577 are the effect sizes (cohen's d) for each of the studies which I calculated in a different spreadsheet. 1 and 2 are the number of the studies and .3057 is the overall effect size (for fixed and random effects as it says). I am interested in the overall effect size and whether the two groups differ significantly from each other. My questions above referred to a) the difference between random and fixed effect models for only two studies and b) whether I have to be bothered by heterogeneity – klonkklonk Aug 05 '17 at 13:08
  • Whether you have interest in groups or studies ? –  Aug 06 '17 at 05:49

1 Answers1

4

To answer your questions more or less in order.

The value of $\tau^2$ is estimated as zero so under those circumstances the two models will give the same result. It is not because you have two studies.

With only two studies the estimate of $\tau^2$ will be very imprecisely estimated so in reality you can say little about the true amount of heterogeneity. Even though you specified a fixed effect model the estimate of $\tau^2$ is indeed zero as you can see by comparing the value of $Q$ to its df.

It would in general be a good idea to look at a forest plot to understand heterogeneity but in this case with so few primary studies it is unlikely to be informative.

mdewey
  • 16,541
  • 22
  • 30
  • 57
  • Thanks for your quick and detailed answer. So is there any reason based on the method or just using two studies that tau^2 is zero? What does the comparison of Q and df tell me? I am not familiar with which relationships hint at a specific value of tau^2. And, I added the forest plot. Could you derive any conclusion of heterogeneity from it? – klonkklonk Aug 04 '17 at 13:28
  • 1
    It is the amount of between study variability which counts, not the number of studies. Experiment by adding a constant to one of your estimates and its confidence interval limits (say 5) and you should see a different value for $\tau^2$. I do not think it is safe to draw any conclusions about heterogeneity from two studies. – mdewey Aug 04 '17 at 14:04
  • Okay, sound reasonable to draw any conclusions about heterogeneity from just to studies. The idea to add a constant was a very good one! Indeed, tau^2 does change and the results for fixed and random effects begin to differ – klonkklonk Aug 05 '17 at 13:11