Scribd
Upload
86 views25 pages

Meta Analysis: Ruswana Anwar, Herman Wibisono

This document discusses meta-analysis, which is a statistical technique for combining the results of multiple scientific studies. It focuses on the direction and magnitude of effects across studies through effect sizes rather than just statistical significance. Meta-analysis is applicable when studies examine the same relationships and report quantitative, comparable results. Effect sizes standardize findings to directly compare studies and make meta-analysis possible. While meta-analysis provides benefits like protecting against overinterpreting individual studies, it also has weaknesses including the potential for selection bias and "apples to oranges" comparisons of studies.

Uploaded by

yuyu tuptup
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
86 views25 pages

Meta Analysis: Ruswana Anwar, Herman Wibisono

This document discusses meta-analysis, which is a statistical technique for combining the results of multiple scientific studies. It focuses on the direction and magnitude of effects across studies through effect sizes rather than just statistical significance. Meta-analysis is applicable when studies examine the same relationships and report quantitative, comparable results. Effect sizes standardize findings to directly compare studies and make meta-analysis possible. While meta-analysis provides benefits like protecting against overinterpreting individual studies, it also has weaknesses including the potential for selection bias and "apples to oranges" comparisons of studies.

Uploaded by

yuyu tuptup
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 25

Meta analysis

Ruswana Anwar, Herman Wibisono


The Logic of Meta-analysis
 Traditional methods of review focus on
statistical significance testing
 Meta-analysis changes the focus to the
direction and magnitude of the
effects across studies
 Direction and magnitude represented by
the effect size
Forms of Research Findings Suitable to Meta-
analysis

 Group contrasts
 Experimentally created groups
 Comparison of outcomes between treatment
and comparison groups
 Central tendency research
 Prevalence rates
When Can You Do Meta-analysis?
 Meta-analysis is applicable to collections of
research that
 Are empirical, rather than theoretical
 Produce quantitative results, rather than qualitative
findings
 Examine the same constructs and relationships
 Have findings that can be configured in a
comparable statistical form (e.g., As effect sizes,
correlation coefficients, odds-ratios, proportions,
etc.)
 Are “comparable” given the question at hand
Effect Size:
The Key to Meta-analysis
 The effect size makes meta-analysis possible
 It is the “dependent variable”

 It standardizes findings across studies such that

they can be directly compared


 Different meta-analyses may use different effect size
indices
Effect Size:
The Key to Meta-analysis
 Any standardized index can be an “effect size” (e.G.,
Standardized mean difference, correlation coefficient,
odds-ratio) as long as it meets the following
 Is comparable across studies (generally requires

standardization)
 Represents the magnitude and direction of the

relationship of interest
 Is independent of sample size
Which Studies to Include?
 It is critical to have an explicit inclusion
and exclusion criteria
 The broader the research domain, the
more detailed they tend to become
 Refine criteria as you interact with the
literature
Which Studies to Include?
 Components of a detailed criteria
 distinguishing features
 research respondents
 key variables
 research methods
 cultural and linguistic range
 time frame
 publication types
Searching Far and Wide
 The “we only included published studies
because they have been peer-reviewed”
argument
 Significant findings are more likely to be
published than nonsignificant findings
 Critical to try to identify and retrieve all
studies that meet your eligibility criteria
Searching Far and Wide
(continued)
 Potential sources for identification of
documents
 Computerized bibliographic databases
 Authors working in the research domain
 Conference programs
 Dissertations
 Review articles
 Hand searching relevant journal
 Government reports, bibliographies,
clearinghouses
Interpreting Effect Size
Results
 Cohen’s “Rules-of-Thumb”
 standardized mean difference effect size
 small = 0.20
 medium = 0.50
 large = 0.80
 correlation coefficient
 small = 0.10
 medium = 0.25
 large = 0.40
 odds-ratio
 small = 1.50
 medium = 2.50
 large = 4.30
The Effect Size
 The effect size (ES) makes meta-
analysis possible
 The ES encodes the selected research
findings on a numeric scale

12
The Effect Size
 There are many different types of ES
measures, each suited to different
research situations
 Each ES type may also have multiple
methods of computation
Interpreting Effect Size
Results

 These do not take into account the


context of the intervention
 They do correspond to the distribution
of effects across meta-analyses
The Standardized Mean
Difference
X  X G2 s12 n1  1  s22 n2  1
ES  G1 s pooled 
s pooled n1  n2  2

 Represents a standardized group contrast on


an inherently continuous measure
 Uses the pooled standard deviation (some
situations use control group standard
deviation)
 Commonly called “d” or occasionally “g”
The Odds-Ratio
 The odds-ratio is based on a 2 by 2
contingency table, such as the one below
Frequencies
Success Failure ad
Treatment Group a b ES 
bc
Control Group c d

The Odds-Ratio is the odds of success in the treatment group


relative to the odds of success in the control group.
Confounding of Study
Features
 Relative comparisons of effect sizes across
studies are inherently correlational!
 Important study features are often
confounding, obscuring the interpretive
meaning of observed differences
 If the confounding is not severe and you have
a sufficient number of studies, you can model
“out” the influence of method features to
clarify substantive differences
Strengths of Meta-analysis
 Imposes a discipline on the process of
summing up research findings
 Represents findings in a more
differentiated and sophisticated manner
than conventional reviews
 Capable of finding relationships across
studies that are obscured in other
approaches
Strengths of Meta-analysis

 Protects against over-interpreting


differences across studies
 Can handle a large numbers of studies
(this would overwhelm traditional
approaches to review)
Weaknesses of Meta-analysis
 Requires a good deal of effort
 Mechanical aspects don’t lend themselves to
capturing more qualitative distinctions between
studies
 “Apples and oranges” criticism
 Most meta-analyses include “blemished” studies to
one degree or another (e.g., a randomized design
with attrition)
Weaknesses of Meta-analysis
 Selection bias posses a continual threat
 Negative and null finding studies that you were

unable to find
 Outcomes for which there were negative or null

findings that were not reported


 Analysis of between study differences is
fundamentally correlational
Final Comments
 Meta-analysis is a replicable and
defensible method of synthesizing
findings across studies
 Meta-analysis often points out gaps in
the research literature, providing a solid
foundation for the next generation of
research on that topic
Final Comments

 Meta-analysis facilitates generalization


of the knowledge gain through
individual evaluations
Meta analysis

Ruswana Anwar, Herman Wibisono

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy