Case study: predicting effectiveness based on RCT efficacy data and RWE – rheumatoid arthritis


Rheumatoid arthritis (RA) is the most common type of autoimmune arthritis, an inflammatory disease of the joints triggered by a faulty immune system. Its worldwide incidence rate is 0.5–1.0% and is about twice as high in women as in men. RA patients mainly suffer from pain and swelling of the wrists and ankles as well as of the smaller hand and foot joints. The earlier RA is diagnosed and appropriately treated, the more effectively joint damage and limitations in mobility can be prevented.

Available treatments for RA include non-steroidal anti-inflammatory agents, corticosteroids and disease modifying anti-rheumatic drugs (DMARDs). Since 1998, a special class of DMARDs, known as biological DMARDs (bDMARDs) has been available. Whether a person with RA receives a biological or conventional DMARD (cDMARDs) depends on a number of factors, including patient and disease characteristics. These decision criteria are not taken into account when treatment effect is assessed in randomised controlled trials (RCTs) on pre-selected patient populations.

What was examined in this case study?

The effect of combination treatment with a new biological and a conventional DMARD was predicted for patients who are likely to receive this treatment in a real-world healthcare system when the new biological DMARD is launched. RCT data on the efficacy of the new medicine were available, but real-world evidence (RWE) on its effectiveness was not yet available. The case study addressed questions about the population and treatment effect, as follows:

Which patients are likely to receive the new biological DMARD after its launch?

  • Ask for RWE on an existing similar biological DMARD, assuming that patients receiving the existing treatment are likely to receive the new biological DMARD once it is launched.
  • Identify the most important drivers of treatment decision.
  • Predict treatment decision (new biological DMARD + conventional DMARD vs. conventional DMARD).

What is the expected treatment effect in these patients?

  • Estimate the relative treatment effect from the RCT efficacy data.
  • Identify the most important effect modifiers based on expert opinion and the RCT efficacy data. Estimate their impact on treatment outcome.
  • Identify the most important prognostic factors based on expert opinion, the RCT efficacy data, and RWE. Estimate their impact on treatment outcome.
  • Predict treatment outcome.

The availability of individual participant data (IPD) on the efficacy of the new treatment and the effectiveness of the similar treatment was essential to this study. In addition, a strong collaboration with clinical experts was of key importance.

What ‘effectiveness challenge(s)’ was addressed in this case study?

RCT is the most reliable study design to assess drug efficacy and safety. However, treatment effects estimated from RCT data may differ from treatment effects observed in usual clinical practice because of the strict inclusion criteria for RCTs (see figure).

Figure: Comparison of treatment outcomes and effects for conventional DMARD (cDMARD) vs. biological DMARD (bDMARD) + cDMARD

Comparison of treatment outcomes and effects

It is important to be aware of this potential efficacy-effectiveness gap when predicting the real-world effect of a new treatment in patients who are likely to receive the treatment after it is launched. This case study combined expert knowledge, RCT outcomes and RWE to bridge the efficacy-effectiveness gap and predict drug effectiveness.

What were the findings and conclusions?

  • Real-world treatment effectiveness can be predicted before launch: expert knowledge, RCT results and RWE can be combined to predict real-world treatment effectiveness before market launch.
  • Accuracy can only be evaluated retrospectively: the accuracy of the predictions can only be evaluated in retrospective analyses; in general, external validation is an open issue.
  • Sensitivity analyses should be carried out: predicted treatment outcome depends on the weighting of RCT evidence vs. RWE when estimating the effects of the prognostic factors; sensitivity analyses should be conducted.
  • A sophisticated search strategy is needed: the search strategy needs to identify the most important effect modifiers and prognostic factors; different strategies yield different results.

What do stakeholders say?

  • RWE on the potential side effects of the similar biological DMARDs should be considered when predicting treatment decisions.
  • Statistical findings may differ from expert advice. This may be due to factors such as the limited amount of data, reporting issues and inconsistencies. In this approach, expert opinion had priority over pure statistics.
  • Adherence issues should be investigated. In this case study, information on adherence and non-adherence was not available.
  • It would be interesting to see how the model performs if the trial and real-world populations differ greatly.
  • Results from this case study should be used to inform the specification of more real-world oriented RCT inclusion criteria or to design clinical trials in a more pragmatic way.

Key contributor

Eva-Maria Didden, University of Bern