1

I am using multi-level meta-analysis model in metafor package in R. The funnel plot for a random effect model (with moderator) and a mixed effect model (without a moderator) were drawn. Including the moderator improved the funnel plot; however, it is not a significant covariate in the analysis and therefore not part of my final model. How should I interpret the publication bias in the funnel plot of the random-effect model in this case?

daragh
  • 43
  • 6
  • What makes you say that there is small-study bias in these plots? – mdewey May 17 '16 at 07:07
  • @mdewey because there is some sort of asymmetry in the funnel plot of the random effect model and many dots (i.e. studies) lies outside the white area? – daragh May 17 '16 at 07:19
  • I agree that I don't see that much of small study effects. What does a regression (Egger, Peters) test tells you? – Giuseppe Biondi-Zoccai May 17 '16 at 08:15
  • @GiuseppeBiondi-Zoccai I am not sure that will help as there will be a few outliers which may influence things. – mdewey May 17 '16 at 17:39

1 Answers1

2

In the plots there seem to be several things to remark on.

1 - most studies fall within an intermediate range of precision especially in the second plot so any effect of the less precise studies is going to be quite small since there are few of them.

2 - it is difficult to see patterns when the density of points varies so much over the range as there will be more variability in the dense parts of the plot.

3 - there is much heterogeneity here and the search for yet more moderators seem more valuable than looking for small study effects.

mdewey
  • 16,541
  • 22
  • 30
  • 57
  • I searched and tried all possible moderators that I can collect from the studies but non-were significant. How can I argue that the slight asymmetery in the funnel plot won't affect the result of the meta-analysis? I have to add something on that in my discussion in the paper I am writing. Doing a `regtest` here is not applicable to multi-level mode! – daragh May 22 '16 at 12:47
  • @daragh, fortunately you can do a regtest although not with the built in function http://stats.stackexchange.com/questions/134768/metafor-package-in-r-ranktest-for-multivariate-meta-analysis/134855#134855 shows you how towwards the bottom of Wolfgang's answer. Note though that you only have a few small studies and they may have excessive influence in the regression model. – mdewey May 22 '16 at 14:29
  • Egger et al 1997 British Medical Journal 315 629-634 – mdewey May 23 '16 at 09:26
  • Without knowing the underlying scientific question it is hard to say. It looks as though you are just fitting a model which you could have fitted with rma.uni. If that is not true I suggest asking a new question and giving the scientific background in a bit more detail. – mdewey May 23 '16 at 12:20
  • @medwey I didn't mean to ask about the type of model. My question was about the correct way of testing asymmetry of funnel plots. The answer you referred me to says that to test for asymmety (fir multi-level model), use the predictior (say vi) the same way u use the moderator in the meta model. DOSE THIS MEAN TO ADD (vi) as a continuous moderator in the model and see if it is significant? – daragh May 24 '16 at 08:25
  • @daragh You asked me whether the model you had fitted was correct, and as I said, I cannot answer that. You may be adding the moderator correctly but I am not sure whether the model itself is correct. – mdewey May 24 '16 at 20:00
  • David Spiegelhalter has a paper titled *Funnel plots for comparing institutional performance*, published in Statist. Med. 2005; 24:1185–1202. It may contain something related to your questions. – Mike Hunter Jul 22 '16 at 00:51