An earlier GetReal case study on drivers of effectiveness (for a definition, see here) identified a number of patient populations that are routinely excluded from phase 3 randomised controlled trials (RCTs) but have characteristics considered to be important treatment effect modifiers. As a result, these phase 3 RCTs may not be generalisable to the real-world population and, importantly, the studies can lead to biased estimates of a medicine’s effect.
What was examined in this case study?
A modelling and simulation study was performed to explore how the combination of an innovative design parameter (RCT population enrichment) and predictive modelling may improve the ability to predict the effectiveness of a medicine without increasing the RCT sample size or compromising the ability to detect the medicine’s effect. The population enrichment involves including some participants who would otherwise be excluded from the RCT while maintaining the same number of participants as for the RCT. The simulation explored the optimal proportion of the enrichment subset of participants to ensure an adequate prediction of effectiveness without jeopardising the success of the trial. See here for more information about the method used for this type of study.
This study used data from the European Schizophrenia Outpatient Health Outcome (SOHO) cohort study of 10,281 outpatients taking antipsychotic drugs. This cohort was considered to reflect a real-world population and was used to simulate an RCT.
From this cohort, a subset of patients (the ‘RCT population’) who had started on or switched to an antipsychotic drug were selected if they would have been eligible for a typical RCT because they had none of the following typical exclusion criteria:
- treated in private practices (because RCTs usually take place in hospitals)
- age over 65 years
- disease chronicity less than 3 years
- previous suicide attempt
- history of alcohol misuse/dependence
- history of substance misuse/dependence
- body mass index (BMI) under 17 or over 40.
Two of the most frequently prescribed antipsychotic drugs were selected (referred to as ‘drug A’ and ‘drug B’).
The effect estimates of the drugs were measured as the evolution in symptom severity 3 months from baseline. The effectiveness of drugs was defined as the antipsychotic effect estimates when using the whole SOHO population. The efficacy of drugs was defined as the antipsychotic effect estimates when using the ‘RCT population’ only.
Through simulation, patient populations typically excluded from schizophrenia phase 3 trials (the ‘enriched population’) were re-introduced into the trial incrementally, starting at 0% up to the maximum proportion of these patients represented in the SOHO study).
The real-life effect of the two antipsychotic drugs was predicted using only data from the ‘enriched’ phase 3 population (and using ordered probit regression models), and this was compared with the real-life effect of these medicines as demonstrated in the whole SOHO population.
The probability of success of the enriched trial (at accurately reflecting the results in the whole SOHO population) was evaluated for different levels of enrichment (in other words, different proportions of the trial population with these exclusion factors).
What ‘effectiveness challenge(s)’ was addressed in this case study?
Enriching typical phase 3 trials with patients characterised by selected factors can improve their generalisability because the trial population can better reflect routine practice. Consequently, the trials may improve the prediction of the real-life effects of the investigated medicine.
What were the findings and conclusions?
- Accuracy of prediction improved: prediction accuracy of the real-life effect (as measured in the SOHO cohort study) was improved when the ‘RCT population’ was enriched by partly relaxing specific eligibility criteria while keeping the total number of patients constant.
- The impact of enrichment was not the same across the criteria: the exclusion criteria had the greatest impact when:
- a large proportion of real-life patients had the factor: re-including patients with either a past suicide attempt or a recent history of schizophrenia.
- the factor was considered to be a treatment effect modifier: factors that had the biggest impact on the clinical symptoms (in other words, had the largest difference in results between observational and phase 3 studies) included recent history of schizophrenia (less than 3 years), a BMI under 17 or over 40, or receiving treatment in a private practice.
- Optimal enrichment corresponds to the real-life population: For each factor, the optimal enrichment proportion coincided with the percentage of this patient type in real life. Moreover, relaxing two factors at the same time achieved better results than using the factor separately.
In summary, enriching typical phase 3 trials with selected factors can improve their generalizability, and as a result, the predictions of the real-life effects of the investigated medicine.
Helene Karcher and Clementine Nordon, LASER