Software for evidence synthesis and predictive modelling

What is it?

Existing software for evidence synthesis and predictive modelling were reviewed by the GetReal group (see Overview of Evidence Synthesis and Network Meta-analysis and Methods for Predicting Outcomes). The review was carried out to inform the development of the ADDIS software and aimed to identify comparable products, identify potential components for use in ADDIS and establish approaches and patterns in common (see ADDIS: Aggregate Data Drug Information System).

What were the findings of the review?

The software review identified two main types of software for evidence synthesis and predictive modelling:

Table 1. User interfaces identified for evidence synthesis and predictive modelling

Software Accessibility Uses
Evidence synthesis
ADDIS and GeMTC Open source (web-based) Advanced methods such as network meta-analysis and network meta-regression in the Bayesian framework.

See ADDIS: Aggregate Data Drug Information System and ADDIS

See GeMTC for Evidence Synthesis and GeMTC
Cochrane RevMan Open source Pairwise meta-analysis with subgroups using frequentist methods (also a tool for writing reviews according to Cochrane standards).

See Cochrane RevMan
Comprehensive Meta-analysis Commercial Complex pairwise meta-analysis and meta-regression using frequentist methods.

See Comprehensive Meta-analysis
MetaEasy Commercial (add-in for Microsoft Excel) Converting differently reported outcome data (for example, medians and confidence intervals versus means and standard errors) to the scale needed for analysis.

See MetaEasy
EPPI-Reviewer Commercial Systematic reviewing and meta-analysis, including network meta-analysis using frequentist methods (this software uses the ‘netmata’ and ‘metafor’ packages for R – see details below).

See EPPI-Reviewer
Predictive modelling
Microsoft Excel Commercial Economic modelling, including the implementation of multi-state models.
TreeAge Commercial Modelling geared towards healthcare. Modelling techniques include decision trees, Markov models, patient-level simulation and discrete event simulation

See TreeAge
ARENA Commercial Patient-level simulation and discrete event simulation.

See ARENA
SIMUL8 Commercial Patient-level simulation and discrete event simulation.

See SIMUL8

Table 2. R packages identified for evidence synthesis and predictive modelling

Package Accessibility Uses
Evidence synthesis
General purpose Markov chain Monte Carlo (MCMC) software such as BUGS, JAGS, or Stan Open source Estimating Bayesian models and conducting individual participant data (IPD) meta-analysis. Basic model structure is described in Dias et al, 2013.
gemtc Open source Network meta-analysis and network meta-regression in the Bayesian framework.
netmeta Open source Network meta-analysis using frequentist methods.
metafor and mvmeta Open source Multi-variate meta-analysis and pair-wise meta-regression using frequentist methods (both can be used for network meta-analysis but this requires significant expertise to set up).
lme4 (linear models) MASS (generalized linear models)
hglm (hierarchical generalized linear models)
nlme (non-linear mixed models)		 Open source Frequentist packages for IPD meta-analysis.
Predictive modelling (using multi-state models)
msm Open source Continuous time Markov models
gems, Epi, and simMSM Open source Estimating models with non-linear hazard functions, which need not be Markov models, using patient-level simulation.

What were the limitations of the review?

  • Not all user interfaces were reviewed in depth.
  • The review focused on R software statistical packages, although alternatives may be available for other platforms such as SAS or Stata.
  • For predictive modelling, only software for multi-state models and patient-level simulation were identified. Although these are useful for predicting longer term effects, they do not directly address the efficacy‑effectiveness gap (for a definition, see Clarify the Issues).

Key contributor

Gert van Valkenhoef, University Medical Center Groningen