Statistical Methods for Meta-Analysis

September 16, 2020

Alessio Crippa
Department of Medical Epidemiology and Biostatistics
Karolinska Institutet
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Systematic review

Fixed-effect model

Random-effects model

Heterogeneity

Model choice

Sensitivity analysis

Crippa Alessio
1
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Systematic review

Crippa Alessio
2
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Systematic review

The process of systematically reviewing and integrating research

evidence is described with different terms (systematic review,
meta-analysis, research synthesis, overview, pooling).

The aim of a systematic review is to evaluate and syntethise the

results published over time in the literature on a specific field.

Meta-analysis is the statistical method to combine or pool results

obtained from distinguished, but comparable studies.

Crippa Alessio
3
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Crippa Alessio
4
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Crippa Alessio
5
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Models for meta-analysis

Two common models:

I Common or fixed-effect model: one true effect size. Differences
in the observed effect sizes are only due to sampling error.
I Random-effects model: the true effect size might varies across
studies. Differences in the observed may also be due to
differences in the studies (mixes of participants,
implementations of interventions, etc.)

Crippa Alessio
6
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

The choice between the two models is critical. It doesn’t affect only
the computations, but also the goal of the analyses and the
interpretations of the results.

As formulas are not always intuitive, it helps to keep in mind that

the summary effect is a weighted mean of the observed effect sizes.

Crippa Alessio
7
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Fixed-effect model

Crippa Alessio
8
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Example of a fixed-effect analysis

Aptitude score at one college

Studies Weight Mean [95% CI]

Study A 11.06% 101.30 [98.30, 104.30]

Study B 11.37% 99.56 [96.60, 102.52]
Study C 11.11% 101.26 [98.26, 104.25]
Study D 12.90% 99.78 [96.99, 102.56]
Study E 53.56% 100.91 [99.54, 102.27]

Summary 100.00% 100.69 [99.69, 101.69]

96 98 100 102 104 106

Mean score

Crippa Alessio
9
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Fixed-effect model

All the studies share a common (true and unknown) effect size,
denoted by θ. Effect sizes might differ because of random variability.

Yi = θ + i

i is an index for the study i = 1, . . . , k

Yi is the observed effect size (mean) in the i-th study
θ is the unknown true parameter
i is residual deviation from the common effect in the i-th study

Crippa Alessio
10
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Study E

Study D

Study C

Study B

Study A

True effect

96 98 100 102 104

Crippa Alessio
11
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Inverse variance weighted mean

We aim to estimate the unknown true effects starting from the
observed effect sizes.

The most precise (which minimizes the variance) estimate is a

weighted mean with weights equal to the inverse of the study’s
variance:

1 1
Wi = =
VYi SE(Yi )2
VYi is the within-study variance for study, e.g. an estimate of the
(square) standard error of the observed mean score in the i-th study.

Crippa Alessio
12
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Point estimation

The weighted mean (M) is computed as

Pk
Wi Yi
M = Pi=1
k
i=1 Wi
the sum of the products Wi Yi (effect size multiplied by weight)
divided by the sum of the weights.

Crippa Alessio
13
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Variance and interval estimation

The variance and standard error of the summary effect are given by:

1
VM = Pk
i=1 Wi
p
SEM = VM

95% confidence interval for the summary effect can be calculated

using the normal approximation

M ± 1.96 × SEM

Crippa Alessio
14
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Hand calculations

study y se v w wy std_w
1 Study A 101.3 1.532 2.347 0.426 43.2 0.111
2 Study B 99.6 1.511 2.283 0.438 43.6 0.114
3 Study C 101.3 1.528 2.336 0.428 43.3 0.111
4 Study D 99.8 1.419 2.013 0.497 49.6 0.129
5 Study E 100.9 0.696 0.485 2.063 208.2 0.536

Crippa Alessio
15
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

101.301 · 0.426 + 99.556 · 0.438 + 101.257 · 0.428 + 99.775 · 0.497 + 100.906 · 2.063
M= =
0.426 + 0.438 + 0.428 + 0.497 + 2.063

43.169 + 43.611 + 43.345 + 49.571 + 208.182 387.878

= = = 100.7
3.852 3.852

1 1
VM = = = 0.26
0.426 + 0.438 + 0.428 + 0.497 + 2.063 3.852

√
M ± 1.96 × SEM = 100.7 ± 1.96 · 0.26 = (99.7, 101.7)

Crippa Alessio
16
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Hypothesis testing

A Z -value test can be used for testing the null hypothesis that the
true efftc is equal to zero H0 : θ = 0 vs H1 : θ 6= 0

M
Z=
SEM
A two-sided p-value is obtained from the cumulative standard
normal function

p = 2[1 − Φ(|Z |)]

Crippa Alessio
17
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Studies on Prophylactic Use of Lidocaine After a Heart

Attack (Hine et al. (1989))
Code available at https://rpubs.com/alecri/code_meta

Meta-analysis of death rates in 6 randomized controlled trials

evaluating mortality from prophylactic use of lidocaine in acute
myocardial infarction.

Research question: Is there a detrimental effect of lidocaine on

mortality?

The studies, taken individually, are too small to detect important

differences in mortality rates.

Crippa Alessio
18
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Random-effects model

Crippa Alessio
19
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Example of a random-effects analysis

Aptitude score at all colleges

Weight Mean [95% CI]

Study A 18.11% 94.83 [92.08, 97.58]

Study B 19.25% 98.70 [96.14, 101.25]
Study C 17.15% 101.05 [98.13, 103.97]
Study D 18.72% 99.99 [97.34, 102.63]
Study E 26.76% 98.16 [96.76, 99.56]

Summary 100.00% 98.50 [96.73, 100.27]

90 95 100 105

Mean score

Crippa Alessio
20
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

The assumption of a common effect size for all the studies is often
implausible.

We assume instead that there is a distribution of true effect sizes

and that those studies are somehow similar, so that it makes sense
to combine this information.

Our aim is to estimate the mean or summary effect of this

distribution.

Crippa Alessio
21
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Between-studies variation

Study E

Study D
ζC
Study C

Study B

Study A

True effect

96 100 104
Mean score
Crippa Alessio
22
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Within-study variation

Study E

Study D
ζC
Study C εC

Study B

Study A

True effect

96 100 104
Mean score
Crippa Alessio
23
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Random-effects model

Yi = µ + ζi + i

i is an index for the study i = 1, . . . , k

Yi is the observed effect size (mean) in the i-th study
µ is the overall mean of the (true) study-specific effects
ζi is distance between the overall mean and the study-specific (true)
effect
θi = µ + ζi is the (true) effect in the i-th study
i is residual deviation from the true effect in the i-th study

Crippa Alessio
24
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Key points

The distance from µ and θs depends on τ the standard deviation of

the distribution of the true effects (τ 2 is the variance)

The distance from θi and Yi depends on VYi , the sampling

distribution. It varies across studies.

Crippa Alessio
25
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Study E

Study D

Study C

Study B

Study A

True effect

95 100 105

Crippa Alessio
26
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Inverse variance weighted mean

We aim to estimate the overall mean of the distribution of unknown

effect sizes.

As before, the weights are proportion to the inverse of that study’s

variance. However, under random-effects model, the study’s
variances are given by τ 2 + VYi .

We need an estimate for the between-studies variance.

Crippa Alessio
27
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Der Simonian and Laird estimator of τ 2

Der Siminonian & Laird proposed a moment based estimate of the
between-studies variance.

 

 

Q − (k − 1)

 

T 2 = max Pk ,0
 Pk Wi2 
 i=1 Wi − i=1

 

P k 
i=1
Wi

where Q is the heterogeneity test statistic

P 2
k k
X i=1 W Y
i i
Q= Wi Yi2 − Pk
i=1 i=1 Wi

Crippa Alessio
28
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Point estimation

The random effects estimate is the weighted average

Pk
∗ Wi∗ Yi
M = Pi=1
k ∗
i=1 Wi
where the weights Wi∗ incorporate the between-studies variance:

1 1
Wi∗ = =
VY∗i VYi + T 2

Crippa Alessio
29
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Interval and hypothesis testing

1
VM ∗ = Pk ∗
i=1 Wi
95% confidence interval for the summary effect can be calculated
using the normal approximation

∗
M ± 1.96 × SEM

A test for the the null hypothesis that the mean effect µ is equal to
0 can be obtained as in the fixed-effect meta-analysis.

Crippa Alessio
30
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Hand calculations

study y se v w wy y2 wy2 w*
1 Study A 94.8 1.403 1.967 0.508 48.2 8993 4572 0.221
2 Study B 98.7 1.304 1.701 0.588 58.0 9741 5727 0.234
3 Study C 101.1 1.489 2.218 0.451 45.6 10212 4603 0.209
4 Study D 100.0 1.349 1.820 0.550 55.0 9998 5494 0.228
5 Study E 98.2 0.715 0.511 1.955 191.9 9635 18841 0.325

Crippa Alessio
31
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

k
X
Wi Yi2 =8993.249 · 0.508 + 9740.95 · 0.588 + 10211.553 · 0.451+
i=1

9997.518 · 0.55 + 9635.232 · 1.955 = 39237.17

X k
Wi Yi = 94.833·0.508+98.696·0.588+101.052·0.451+99.988·0.55+98.159·1.955 = 398.68
i=1
Pk Pk
i=1
Wi = 4.05 and i=1
Wi2 = 4.93

398.682
Q = 39237.17 −
4.05

11.272 − (5 − 1)
T2 = 4.93
= 2.566
4.05 − 4.05

Crippa Alessio
32
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Distributional assumptions
i ∼ N(0, VYi ) or θi ∼ N(θ, VYi )
I the moment-based estimator of τ 2 does not require additional
assumption about the distribution of ζi ;
I the pooled effect M ∗ and its SE are also independent on a
distributional assumption for the random-effects;
I confidence intervals relies on approximating distributions or
central limit theorem (k >> 0).
ζi ∼ N(0, τ 2 )
I alternative estimator of τ 2 ;
I facilitates computation of confidence intervals;
I allows to give predictions for new studies.
Crippa Alessio
33
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Studies on the Effectiveness of the BCG Vaccine Against

Tuberculosis (Colditz et al. (1994))

Meta-analysis of 13 studies examining the effectiveness of the

Bacillus Calmette-Guerin (BCG) vaccine against tuberculosis.

Research question: What is the overall effectiveness of the BCG

vaccine for preventing tuberculosis?

Additional question: Are there any moderator variables that may

potentially influence the size of the effect?

Crippa Alessio
34
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Heterogeneity

Crippa Alessio
35
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

The aim of a meta-analysis is not only to provide a summary

measure, but also to make sense of the pattern of the observed
results.

Additional important questions to address:

I Is there evidence of heterogeneity in true effect sizes?
I What is the variance of the true effects?
I What are the implications of the observed heterogeneity?
I What proportion of the observed dispersion can be attributed
to differences among studies?
I Can we explain (part of) the observed heterogeneity?

Crippa Alessio
36
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Heterogeneity across studies

Differences between studies might be due to:
I design, and follow-up
I populations, participants, patients
I treatment or exposure, intervention
I outcome definition

Heterogeneity usually refers to variation in the true effect sizes.

We want to describe and quantify this variation.

Problem: the variation in the observed effect sizes is partly spurious.

It consists of both (true) heterogeneity and random error.

Crippa Alessio
37
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

A set of alternative measures and statistics are available which

tackle different aspect of the heterogeneity:
I Q statistic;
I results from the test based on Q;
I between-studies variance (T 2 );
I between-studies standard deviation (T );
I ratio of true heterogeneity to total observed variation (I 2 )

Crippa Alessio
38
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Disentangle between-studies variation from observed

variation

The idea behind the Der Simonian and Laird estimator

1 Compute the total amount of variation observed across studies.

2 Estimate the expected variation under the fixed-effect model.
3 The excess variation (if any) is assumed to reflect differences in
effect sizes (statistical heterogeneity)

Crippa Alessio
39
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

The Q statistic

Q is a weighted sum of squares of the observed effect sizes, which

quantifies the total amount of variation observed across studies.

k k 
Yi − M 2
X X 
2
Q= Wi (Yi − M) =
i=1 i=1
SYi

where Wi and M are the weights and summary measure from a

fixed-effect model.

Q is a standardized measure which is independent from the metric

of the effect sizes.

Crippa Alessio
40
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Under the fixed-effect assumption, the expected value of Q is its

degrees of freedom E [Q] = df = k − 1

The excess variation is given by Q − df which will be attributed to

differences in the true effects from study to study.

Note: the excess variation might be negative if Q < df.

Q reflects the total dispersion while Q − df the excess variation.

They are not easy to interpret: they are sums, and so they depends
strongly on the number of studies.
Q is on a standardized scale. Some measures expressed as ratio or
on the same scale as the effect size might be preferable.

Crippa Alessio
41
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Testing the assumption of homogeneity in effects

Researchers are often interested in testing if heterogenity is

statistically significant.
The null hypothesis is that the underlying true effect size is the
same for all studies.

The p-value of the heterogeneity test is calculated comparing the

statistic Q with a Chi-Square distribution with degrees of freedom
equal to k − 1.

Usually, an α = 0.1 is chosen as cut off for the p-value.

Crippa Alessio
42
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

A statistically significant result means that there is evidence against

there being one common effect size.
Failing to reject the null hypothesis does not imply that studies are
homogeneous.

It is known that the test for heterogeneity

I has low power when only a few studies

I is possibly oversensitive when many studies

I is difficult to compare across meta-analyses

Note: both Q and the corresponding p-value do not estimate the

magnitude of the true dispersion.

Crippa Alessio
43
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Estimating τ 2
τ 2 is the variance of the true effect sizes. It is in the same metric as
the effect and reflects the absolute amount of variation.

Der Simonian
 andLaird proposed the estimator:
2 Q−df
T = max 0; C

As it might generate negative estimates, it is truncated to 0 in case

Q < df.
It doesn’t require any assumption about the distribution of
random-effects, it easy to compute and explain.

Alternative estimators exist. Many statisticians favor a restricted

maximum likelihood method.
Crippa Alessio
44
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Tau

τ refers to the actual standard deviation of the true effect sizes.

√
An estimator for τ is simply T = T 2 .

T in on the same scale of the effect size. It can be used to describe

the distribution of effect sizes about the mean effect.

e.g. (Under the normality assumption), we expect 95% of the true

effects will fall in the range M ∗ ± 1.96T

T 2 and T are absolute measures. Sometimes, it can be easier to

think about heterogeneity independent of the scale.

Crippa Alessio
45
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

A measure of heterogeneity
Higgins et al (2002) proposed a statistic I 2 which quantifies the
amount of total variability attributed to between study
heterogeneity.

Q − df Variancebetween
I2 = × 100% = × 100%
Q Variancetotal
Advantages of I 2 are:
I intuitive interpretation, it is a percentage
I it is easy to calculate from previously published meta-analysis
I the interpretation is not dependent on the choice of measure of
association
Crippa Alessio
46
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

I 2 reflects the extent of overlap of confidence intervals. It is

convenient to interpret as a measure of inconsistency, and not as a
measure of the real variation.

Note: The I 2 statistic is a descriptive statistic and not an estimate

of any underlying quantity.

Crippa Alessio
47
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Recap

Crippa Alessio
48
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

What to do in case of substantial heterogeneity?

If the studies are highly heterogeneous you need to determine how

to proceed.
I Don’t combine results
I Do a qualitative systematic review (describe trials and evidence
narratively)
I Combine with a random effects model
I Combine only a subset of initial studies (those that are similar)
I Try to explain heterogeneity:
I subgroup analyses
I meta-regression

Crippa Alessio
49
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Exploring Heterogeneity
I Exploring Heterogeneity
I do separate meta-analyses on subgroups of studies according to
study characteristics that may influence the pooled association
(gender, study design, geographical area, year of publication);
I compare the combined results with analogue to an ANOVA
model.
I Meta-regression
I similar to standard regression;
I the pooled effect size is modeled as a linear function of one or
more explanatory variables

Note: Both approaches may have low power, they require several
studies.
Crippa Alessio
50
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Studies on the Effects of Diuretics in Pregnancy

(Viechtbauer et al. (2007))

Meta-analysis on 9 studies examining the effects of diuretics in

pregnancy on various outcomes, including the presence of any form
of pre-eclampsia, perinatal death, stillbirth, and neonatal death.

Research questions: Are there any health-related conditions in the

use of diuretics during pregnancy? How heterogeneous are the
results?

Crippa Alessio
51
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Model choice

Crippa Alessio
52
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Fixed vs Random-effects model

The differences between the fixed and random-effects model are

critical with rispects to
I assumption: one true effect vs. a distribution of true effects;
I goals of the analysis: combining estimates vs. summarizing a
distribution;
I interpretation of the statistics: estimate of the mean vs mean
of the mean estimates;
I computation (estimate of τ 2 ).

Crippa Alessio
53
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Differences in the computation

The variance of the combined effect is larger in a random-effects

model.

It also 2
 includes τ , so the weights will be smaller, and similarly their
sum Var(M) = Pk 1 .
i=1
Wi

This is also true because it’s an estimate of a different parameter

(one mean vs mean of populations)

Crippa Alessio
54
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Differences in precision
Let us assume VYi = σ 2 and ni = n ∀i

1 1 σ2 σ2
VM = Pk = Pk ni
= Pk =
i=1 Wi i=1 σ 2 i=1 ni
kn

I standard deviation of the outcome (higher sd, less precise);

I sample size in the individual studies (n increases, more precise).

σ2 T 2
VM ∗ = +
kn k
I standard deviation of the true effect sizes (higher T 2 , less
precise);
I number of studies (higher k, more precise).
Crippa Alessio
55
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Difference in weights’ definition

The estimate of the combined effect is almost always different under

the two models (unless T 2 = 0), because the weights are different.
1
Wi = VYi : the information from smaller studies is minimal

Wi∗ = V 1+T 2 : T 2 reduces the relative differences among the

Yi
weights

In a random-effects model, large studies lose influence and small

studies gain influence.

Crippa Alessio
56
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Which model should we choose?

Fixed-effect model
- there is good reason to believe that all the studies are functionally
identical;
- the goal is to compute the common effect size (not be generalized
beyond the observed population).

Random-effects model
- the studies have enough in common but are not identical;
- the goal of the analysis is usually to generalize to a range of
scenarios.

Crippa Alessio
57
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Sensitivity analysis

Crippa Alessio
58
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Studies on Lung Cancer Risk from ETS Exposure

(Hackshaw et al. (1997))

Meta-analysis on 37 studies (4 cohort, 33 case-control) reporting

results on the risk of lung cancer from environmental tobacco smoke
(ETS) exposure from the spouse in women who are lifelong
nonsmoker.

Research questions: What is the accumulated evidence of

evidence on lung cancer and environmental tobacco smoke? How
robust are the combined results?

Crippa Alessio
59
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Publication Bias
Precision increases as study size increases.

Results from small studies will be more scattered than larger studies.

In the absence of bias, a plot of precision vs. the estimate (funnel

plot) will resemble a symmetrical inverted funnel.

If there is publication bias, because smaller studies showing no

statistically significant effects remain unpublished, then this will lead
to asymmetry in the funnel plot.

A pseudo-confidence region is usually represented within bounds

M ± 1.96SE.

Crippa Alessio
60
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Funnel plot
0
Standard Error

0.367
0.735

−1.5 −1 −0.5 0 0.5 1 1.5 2

Log Odds Ratio

Crippa Alessio
61
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Egger’s test
Inspection of the funnel plot might be subjective. Egger (BMJ,
1997) and colleagues proposed a simple test to detect asymmetry in
the funnel plot:

E [standardized effect] = a + b × precision

First, standardizes the study-specific effect sizes by subtracting the

pooled effect and dividing by standard error.

Second, fit a regression of standardized effect size against their

precision (inverse of the standard error).

Third, tests whether intercept is significantly different from zero.

Crippa Alessio
62
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Funnel plot asymmetry may reflect something other than publication

bias:
I there may be true heterogeneity leading to a different effect
depending on study size;
I the intervention effect may differ between small and large
studies.

One can view funnel plots as displaying evidence for “small study
effects” in general rather than publication bias in particular.

Crippa Alessio
63
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Trim-and-fill analysis

The trim and fill method is a nonparametric (rank-based) data

augmentation technique proposed by Duval and Tweedie (2005).

It estimates the number of studies missing due to the suppression of

the most extreme results on one side of the funnel plot.
The method augments the observed data so that the funnel plot is
more symmetric.

Note: the method examines the sensitivity of the results to a

particular form of publication bias. It does not provide a more
“valid” summary estimate.

Crippa Alessio
64
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

0
Standard Error

0.367
0.735

−1.5 −1 −0.5 0 0.5 1 1.5 2

Log Odds Ratio

Crippa Alessio
65
Systematic review Fixed-effect model Random-effects model Heterogeneity Model choice Sensitivity analysis

Cumulative meta-analysis

A cumulative meta-analysis describes the accumulation of evidence.

It consists of performing a new meta-analysis as the available

estimates are added to the analysis in (typically) chronological order.

It allows
I the study of trends in efficacy;
I to determine when a new treatment appears to be significantly
effective or deleterious.

Crippa Alessio
66