What is it?
Existing software for evidence synthesis and predictive modelling were reviewed by the GetReal group. The review was carried out to inform the development of the ADDIS software and aimed to identify comparable products, identify potential components for use in ADDIS and establish approaches and patterns in common.
What were the findings of the review?
The software review identified two main types of software for evidence synthesis and predictive modelling:
- User interfaces
- R packages.
Table 1. User interfaces identified for evidence synthesis and predictive modelling
|ADDIS and GeMTC||Open source (web-based)||Advanced methods such as network meta-analysis and network meta-regression in the Bayesian framework.|
|Cochrane RevMan||Open source||Pair-wise meta-analysis with subgroups using frequentist methods (also a tool for writing reviews according to Cochrane standards).|
|Comprehensive Meta-analysis||Commercial||Complex pair-wise meta-analysis and meta-regression using frequentist methods.|
|MetaEasy||Commercial (add-in for Microsoft Excel)||Converting differently reported outcome data (for example, medians and confidence intervals versus means and standard errors) to the scale needed for analysis.|
|EPPI-Reviewer||Commercial||Systematic reviewing and meta-analysis, including network meta-analysis using frequentist methods. (this software uses the ‘netmeta’ and ‘metafor’ packages for R – see details below)|
|Microsoft Excel||Commercial||Economic modelling, including the implementation of multi-state models.|
|TreeAge||Commercial||Modelling geared towards healthcare. Modelling techniques include decision trees, Markov models, patient-level simulation and discrete event simulation.|
|ARENA||Commercial||Patient-level simulation and discrete event simulation.|
|SIMUL8||Commercial||Patient-level simulation and discrete event simulation.|
Table 2. R packages identified for evidence synthesis and predictive modelling
|General purpose Markov chain Monte Carlo (MCMC) software such as BUGS, JAGS, or Stan||Open source||Estimating Bayesian models and conducting individual participant data (IPD) meta-analysis. Basic model structure is described in Dias et al 2013.|
|gemtc||Open source||Network meta-analysis and network meta-regression in the Bayesian framework.|
|netmeta||Open source||Network meta-analysis using frequentist methods.|
|metafor and mvmeta||Open source||Multi-variate meta-analysis and pair-wise meta-regression using frequentist methods (both can be used for network meta-analysis but this requires significant expertise to set up).|
|lme4 (linear models), MASS (generalized linear models), hglm (hierarchical generalized linear models), and nlme (non-linear mixed models)||Open source||Frequentist packages for IPD meta-analysis.|
|Predictive modelling (using multi-state models)|
|msm||Open source||Continuous time Markov models|
|gems, Epi, and simMSM||Open source||Estimating models with non-linear hazard functions, which need not be Markov models, using patient-level simulation.|
What were the limitations of the review?
- Not all user interfaces were reviewed in depth.
- The review focused on R software statistical packages, although alternatives may be available for other platforms such as SAS or Stata.
- For predictive modelling, only software for multi-state models and patient-level simulation were identified. Although these are useful for predicting longer term effects, they do not directly address the efficacy‑effectiveness gap (for a definition, see here).
Gert van Valkenhoef, University Medical Center Groningen