Software for evidence synthesis and predictive modelling

What is it?

Existing software for evidence synthesis and predictive modelling were reviewed by the GetReal group. The review was carried out to inform the development of the ADDIS software and aimed to identify comparable products, identify potential components for use in ADDIS and establish approaches and patterns in common.

What were the findings of the review?

The software review identified two main types of software for evidence synthesis and predictive modelling:

Table 1. User interfaces identified for evidence synthesis and predictive modelling

Software Accessibility Uses
Evidence synthesis
ADDIS and GeMTC Open source (web-based) Advanced methods such as network meta-analysis and network meta-regression in the Bayesian framework.
Cochrane RevMan Open source Pair-wise meta-analysis with subgroups using frequentist methods (also a tool for writing reviews according to Cochrane standards).
Comprehensive Meta-analysis Commercial Complex pair-wise meta-analysis and meta-regression using frequentist methods.
MetaEasy Commercial (add-in for Microsoft Excel) Converting differently reported outcome data (for example, medians and confidence intervals versus means and standard errors) to the scale needed for analysis.
EPPI-Reviewer Commercial Systematic reviewing and meta-analysis, including network meta-analysis using frequentist methods. (this software uses the ‘netmeta’ and ‘metafor’ packages for R – see details below)
Predictive modelling
Microsoft Excel Commercial Economic modelling, including the implementation of multi-state models.
TreeAge Commercial Modelling geared towards healthcare. Modelling techniques include decision trees, Markov models, patient-level simulation and discrete event simulation.
ARENA Commercial Patient-level simulation and discrete event simulation.
SIMUL8 Commercial Patient-level simulation and discrete event simulation.

Table 2. R packages identified for evidence synthesis and predictive modelling

Package Accessibility Uses
Evidence synthesis
General purpose Markov chain Monte Carlo (MCMC) software such as BUGS, JAGS, or Stan Open source Estimating Bayesian models and conducting individual participant data (IPD) meta-analysis. Basic model structure is described in Dias et al 2013.
gemtc Open source Network meta-analysis and network meta-regression in the Bayesian framework.
netmeta Open source Network meta-analysis using frequentist methods.
metafor and mvmeta Open source Multi-variate meta-analysis and pair-wise meta-regression using frequentist methods (both can be used for network meta-analysis but this requires significant expertise to set up).
lme4 (linear models), MASS (generalized linear models), hglm (hierarchical generalized linear models), and nlme (non-linear mixed models) Open source Frequentist packages for IPD meta-analysis.
Predictive modelling (using multi-state models)
msm Open source Continuous time Markov models
gems, Epi, and simMSM Open source Estimating models with non-linear hazard functions, which need not be Markov models, using patient-level simulation.

What were the limitations of the review?

  • Not all user interfaces were reviewed in depth.
  • The review focused on R software statistical packages, although alternatives may be available for other platforms such as SAS or Stata.
  • For predictive modelling, only software for multi-state models and patient-level simulation were identified. Although these are useful for predicting longer term effects, they do not directly address the efficacy‑effectiveness gap (for a definition, see here).

Key contributor

Gert van Valkenhoef, University Medical Center Groningen