A key objective of Work Package 1 of GetReal was to develop a framework for incorporating real-world data (RWD) into decision-making. Case studies were constructed to explore the different ways that RWD may be used to help demonstrate the relative effectiveness of new medicines.
Rheumatoid arthritis (RA) is a chronic autoimmune disease that causes pain, stiffness, swelling and inflammation of the synovial joints (such as those in the hands and feet). It is characterised by periods when symptoms become worse (called flares or flare ups). There is no cure for RA, but there are multiple treatment options available to reduce disease activity, improve physical function and enhance quality of life. Medication is often used, in addition to lifestyle changes and supportive therapies, to limit the impact of the condition.
Three classes of drugs are widely used to treat RA:
- Non-steroidal anti-inflammatory drugs (NSAIDs), which have a rapid onset and are used to reduce acute inflammation and pain.
- Corticosteroids, which are anti-inflammatories used for patients with early onset or severe disease.
- Disease modifying anti-rheumatic drugs (DMARDs), categorised as conventional or biological DMARDS (also called biologics), which are the most common treatments used to improve RA symptoms. DMARDs have a slower onset but have been shown to alter the course of RA and improve radiographic outcomes (NICE technology appraisal guidance TA375 2016).
Effectiveness assessments for RA treatments have highlighted the following issues:
- a lack of RCTs directly comparing relevant treatments or evidence providing indirect treatment comparisons (resulting in a ‘disconnected network’)
- a lack of evidence on second-line treatment (after another medicine was found to be ineffective or result in intolerable side-effects).
What was examined in this case study?
This case study aimed to identify and evaluate methods to incorporate real-world evidence (RWE) in the evidence synthesis of second-line biological DMARDs in RA.
Data from two European patient registries (the Swiss Clinical Quality Management in Rheumatic Diseases and the British Society for Rheumatology Biologics Register) were used to supplement randomised evidence in network meta-analyses (NMA) (for more about NMA, see here).
This was used to address the following key issues:
- How to connect disconnected networks of evidence to conduct an NMA.
- How to optimise an evidence base using first-line evidence to inform second-line effectiveness estimates.
The network of RCTs for second-line biologics in RA is very sparse and there are differential levels of evidence for first- and second-line therapies. Simpler ‘naïve’ methods of pooling are compared to more sophisticated NMA models to try and maximise the use of all available evidence (both observational data from the patient registries and randomised evidence from first-line biologic trials). These methods were used to inform treatment comparisons otherwise not possible and to reduce the uncertainty around estimates of effect.
Outputs of analyses were discussed at a GetReal workshop held in London (16 March 2016) during which stakeholder views of the utility and applicability of the methods tested were obtained. Workshop outputs are reported here. The following questions were discussed:
- Under what circumstances would these methods have an impact on decision-making?
- Could these methods be used to address other ‘effectiveness’ issues and/or applied to other disease areas?
- How could these methods be improved and developed further to be useful and acceptable to stakeholders?
- What issues might stand in the way of adopting such methods by various stakeholders?
- Are there situations where these methods would be particularly useful (or not useful)?
- How can we best communicate the implications of these methods to engage a broader range of stakeholders?
What ‘effectiveness challenge(s)’ was addressed in this case study?
The evidence network for biological DMARDs as second-line therapy was only formed of four relevant RCTs, which were ‘disconnected’ and did not allow an NMA to be performed. However, using RWE allowed all of the treatment options to be connected, and NMA models were extended to include both randomised and registry data to inform treatment effects. In addition, univariate and bivariate NMAs were performed to link the randomised evidence available for first-line treatments to predict second-line effectiveness using ‘prior’ information from the real-world setting to estimate the correlation of effects between both lines of treatment.
What were the findings and conclusions?
Connecting the second-line network of evidence for biological DMARDs with data from registries led to higher confidence in the treatment effect. This could imply that using the registry data provided closer estimates to the ‘true’ effect of biological DMARDs. However, the reduction in uncertainty was arguably found to be ‘not enough’ to better inform decision-making for RA, and methods were subject to a number of caveats.
Using the first-line data to inform second-line effectiveness estimates (especially using the bivariate approach) allowed for predictions of treatment effects that had not been evaluated in second-line trials. The uncertainty around predicted effect sizes varied considerably, as linking first- and second-line evidence bases did allow the borrowing of strengths across treatments and networks, but also increased heterogeneity.
What do stakeholders say?
All stakeholders agreed that robustness of methods was a key acceptability factor for HTA bodies and pharmaceutical R&D and that analyses adding further decision uncertainty would not be deemed helpful at any stage of the product development cycle.
Large uncertainty in the results presented and the sometimes large variation in the point estimates raised concerns about the methodology suggesting underlying data limitations (i.e. inconsistencies within the evidence base and across networks); consequently, participants were in agreement that further demonstration of the robustness of the approaches was required.
More details from the stakeholder workshop are found here.
Pascale Dequen, Reynaldo Martina, Keith Abrams, David Jenkins, and Sylwia Bujkiewicz, University of Leicester
Mats Erikson, Amgen