NMA using IPD from RCTs

What is it?

Network meta-analysis (NMA) is a common approach to synthesise the efficacy of multiple treatments, and to compare their relative efficacy (see Overview of evidence synthesis and NMA). NMA can be based on the raw individual participant data (IPD) from each study, but most are based on aggregate data (that is, published summary estimates of relative treatment effect, usually as identified through a systematic literature search). For further information, see the GetReal review on methods for IPD meta-analysis. The articles identified in this review can be found on Zotero.

Why is it useful?

• Increased precision and consistency: NMA based on IPD may give more precise estimated results and greater consistency than NMA based on aggregate data because participant-level characteristics can be better adjusted for. NMA based on aggregate data tends to have higher levels of inconsistency and/or between-study heterogeneity, imprecise treatment effects (meaning that there is low confidence in the results) and may yield biased results, giving invalid or misleading estimates of comparative treatment effects. See the box below.
• Supports inclusion of real-world evidence (RWE): Access to IPD may support the inclusion of RWE in analyses. More information on the use of NMA using RWE can be found on NMA incorporating RWE.
• Increases ability to do the following:
  • analyse all important outcomes
  • determine the validity of analysis assumptions
  • use the same unit and method of analysis for each trial
  • conduct more detailed analysis of time-to-event outcomes
  • achieve greater power for assessing interactions between effects of interventions and participant or disease/condition characteristics (known as effect modification)
  • conduct more complex analyses
  • use non-standard models or measures of effect. (Tierney et al., 2015, Debray et al., 2015)
• May help personalise treatment strategies: NMA based on IPD may help to tailor treatment strategies by:
  • including evidence from studies with diverse populations (for example, non-randomised studies of interventions) (Debray et al., 2015)
  • adjusting for factors that may influence the outcome (known as prognostic factors)
  • adjusting for treatment-covariate interactions
  • borrowing information across studies (for example, when outcomes of primary interest have not been fully measured in randomised controlled trials [RCTs]). (Debray et al., 2016 and case study: NMA using IPD from RCTs)
• May also help to:
  • clarify trial eligibility with trial investigators
  • improve the quality of the data (for example, to include trials that are not reported in full).
  • standardise outcome definitions across trials or translate different definitions to a common scale
  • update follow-up or time-to-event outcomes beyond those reported
  • clarify the trial design, conduct and analysis methods
  • overcome potential bias arising from drop-out related to the treatment (known as informative drop-out). (Tierney et al., 2015, Debray et al., 2015)

Box: Issues with NMA based on aggregate data

NMA based on aggregate data tends to have: Higher levels of inconsistency: this means that the comparative treatment effect of two interventions differ according to whether they are assessed head-to-head or indirectly through other treatment comparisons (see diagrams of networks on Overview of evidence synthesis and NMA). High between-study heterogeneity: this implies that the comparative treatment effect of two interventions substantially differs across the included RCTS. Very imprecise treatment effects: meaning that there is very low confidence in the results from NMA based on aggregate data. Biased results: meaning that estimates of comparative treatment effect are invalid or misleading. (note: The presence of inconsistency and heterogeneity in an NMA may degrade the validity and clinical usefulness of summary estimates of comparative treatment effect, and should therefore be properly addressed.)