Using RWE to estimate relative effectiveness and inform trial design: a case study in multiple sclerosis


A key objective of Work Package 1 of GetReal was to develop a framework for incorporating real-world data (RWD) into decision-making. Case studies were constructed to explore the different ways that RWD may be used to help demonstrate the relative effectiveness of new medicines.

In the absence of a cure for multiple sclerosis (MS), many patients are administered disease-modifying therapies early in life to attenuate relapse frequency and severity and mitigate disease progression – a regimen that is often continued for the remainder of the patients’ lifetimes. Such long-term treatment regimens necessitate medicines that maintain effectiveness for the duration of the administration period, confer few side-effects and are conveniently administered to encourage patient adherence to treatment. Some headway is being made in these respects with the addition of new oral treatments for MS eliminating the need for sub-cutaneous injections or intravenous infusions of older therapies, but severe side-effects continue to be an unmet challenge in MS therapy.

Several challenges are particularly pertinent to MS therapies from the health technology assessment (HTA) perspective. As the number of new MS therapies grows, medicine developers need to demonstrate comparative efficacy of new treatments compared to long-available therapies and they will need to be shown as safer and better tolerated than existing medicines with more favourable ways of administration. However, randomised controlled trials (RCTs) of new medicines compared to older treatments have not always been available for MS. External real-world evidence (RWE) may help to address these challenges, when used in evidence synthesis where RCTs have not been available.

The overall aim of the case study was to identify development options that might reduce stakeholder decision-making uncertainty by including RWE earlier (before marketing authorisation) in the clinical development of MS therapies.

Data were made available to GetReal by Novartis (GetReal partner). Although MS was chosen as a suitable disease for using these methods, the same methods can be applied generally to other disease areas. More background information to the disease area and on the history of the evidence and previous assessments can be found here.

What was examined in this case study?

The following methods were examined in this case study:

  1. Supplementing trial results with RWE in network meta-analysis (NMA) to generate relative effectiveness estimates (for more information about NMA see here; for more information about the benefits and limitations of using RWE in NMA, see here)
  2. Incorporating RWE in NMA to support simulations informing trial designs
  3. Using risk equations derived from RWE to inform risk stratified trial designs

The use of RWE in NMA in the first method aimed to provide indirect comparisons of relative effectiveness to be made between therapies when the clinical trial network is incomplete (i.e. RWD are used to broaden the evidence base). In this approach, all available sources of information including RWE were synthesised. Analytical weighting techniques were applied to the RWE in the network to examine the impact this may have on power and uncertainty.

The second method using RWE to inform trial design aimed to enable a better understanding for pharmaceutical companies in the power of alternative designs (with potentially different costs and durations) required to detect effect differences. In this approach, trial-based modelling techniques were informed by effectiveness estimates generated from NMAs that include RCT data and RWE. Analyses at key stages of the clinical development programme of a medicine were performed to identify the most efficient designs possible given the available body of evidence.

The third method was used to understand the uncertainty of treatment effects in subgroups based on pivotal trial populations. The approach used RWE to inform the design of a risk-equation (risk score) that is used to stratify patients with multiple sclerosis based on their risk of relapse in simulated trials. The power and uncertainty of the simulated trials could be used to inform future study design.

Outputs of analyses were discussed at a GetReal workshop during which stakeholder views of the utility and applicability of the methods tested were obtained. Workshop outputs are reported here. The following questions were discussed:

  • Could you envisage using these approaches in your decision-making process?
  • Are there situations where this approach is particularly useful (or not at all useful)?
  • What issues might stand in the way of adopting this approach?
  • Are there situations where these approaches would be particularly useful (or not at all useful)?
  • How can we communicate the implications of these approaches to engage a broad range of stakeholders?

What were the findings and conclusions?

Using RWE in network meta-analysis (NMA) to generate relative effectiveness estimates

  • Simulations showed that assigning increasing weight to available RWE in a NMA had relatively little impact on point estimates of effectiveness of medicines, but increased corresponding levels of uncertainty. Although the inclusion of RWE in NMA might generally be expected to reduce the uncertainty of treatment effect estimates, it can however also increase uncertainty, as was shown.

Incorporating RWE in NMA to support simulations informing trial designs 

  • Simulations showed that inclusion of RWE with a NMA in planning a clinical development strategy could result in a more efficient development programme (such as, smaller Phase III studies).

Using risk equations derived from RWE to inform risk stratified trial designs

  • Simulations showed that when low, medium and high risk patients from the TRANSFORMS pivotal trial for fingolimod were analysed separately (rather than as a single cohort) the power of the trial was highest in higher risk patients. This could have significant implications for a drug development programme where a high risk population would require a smaller or quicker trial that does not sacrifice uncertainty, while lower risk population could be studied subsequently or in parallel.

What do stakeholders say?

In the workshops, stakeholders welcomed the proposals as additional options for reducing decision-making uncertainty. However, concerns were raised around potential biases that could be introduced by including RWE in these ways. The inclusion of RWE would most likely be considered as supportive of or adding context to regulatory submissions, but could be more central in HTA decision-making and early medicine development if appropriate quality control measures are put in place to mitigate biases commonly associated with non-interventional data.

For the use of RWE to become more acceptable by decision makers, standard methods for data synthesis should be developed, as well as guidelines to ensure transparency in selection of data sources and data synthesis.

Key contributors

Alexandre Joyeux, Novartis
Prof. Keith Abrams, University of Leicester