What is it?
First described by Schwartz and Lellouch pragmatic trials aim to measure the relative effectiveness of treatment strategies in real-world clinical practice. They provide evidence of the added value of a treatment strategy in routine clinical practice, while maintaining the strength of comparisons based on randomised controlled trials.
In a pragmatic trial a comparison is made between randomised groups of patients that are similar to the target group in the characteristics that influence outcome event rates and potentially modify drug response, in a study setting that is similar to what would be experienced in real life. The treatment strategies being compared, and the outcome measures used for the comparison should be relevant to routine clinical practice. The term ‘pragmatic trial’ is commonly used for trials that assess the difference between treatment strategies, to include the potential impact of extraneous factors other than the pharmacological effect of the medicine (such as co-medication, non-adherence and placebo effects). The aim is to maximise generalisability of the results to a broader setting or patient population, for example the ‘decision-making’ population being considered by HTA for a reimbursement agency.
For most treatments newly receiving marketing authorisation, the clinical evidence (mainly from phase 3 clinical trials) is still insufficient to fully guide clinicians and policy makers in choosing the optimal treatment for their patients. Pragmatic trials can help improve decision-making by supplementing these data with real-world evidence (RWE).
Pragmatic and explanatory trials: a continuum
Pragmatic and explanatory trials (which measure efficacy under ideal conditions, such as typical phase 3 RCTs) represent the ends of a continuum rather than distinct entities (Thorpe et al, 2009). A study may contain elements from both approaches.
The design choices that can be made towards a more pragmatic trial design can be related to four domains: the study population; the setting of the trial; operationalisation of the intervention and choice of comparator treatment; and the outcome measure. General issues of data management and monitoring also need to be considered, because these can influence routine clinical practice and therefore the generalisability of the trial results.
A choice between a more pragmatic or more explanatory trial design can be made in the following examples:
| More explanatory | More pragmatic | |
| Study population | Study participants are either highly selected patients likely to reveal a treatment effect | The target population that is encountered in clinical practice |
| Trial setting | Trials take place in specialised centres with extensive experience of regulatory and other formal requirements. Trials have access to sufficient numbers of patients for recruitment of study subjects | Trials take place in routine care settings, where there is minimal disturbance to routine patient care |
| Operationalisation of the intervention | Extraneous effects, such as adherence and co-medication, are either equalised in the comparison between the groups in order to study the true pharmacological effects of the medicine | Trials are included in the full set of components of an overall treatment strategy (more pragmatic). |
| Outcome measures | Biologically meaningful outcomes that demonstrate pharmacological effects are chosen | Meaningful outcomes for decision-making in routine clinical practice are chosen |
Importantly, a more pragmatic trial will aim to minimise the level of interference with clinical practice as far as possible, to maintain usual care throughout the trial.
Key elements of the continuum between pragmatism and explanation in trials are illustrated in the figure below
Figure. Continuum between explanatory and pragmatic trials
Specific trial designs may be considered for introducing more pragmatic elements into trials.
For example, if randomisation of patients in a medical setting changes the routine care process (for example, if introducing the intervention changes how care in general is provided at the site), cluster randomisation may be preferred (see Cluster RCTs).
Another way to minimise the level of interference with clinical practice might be through a ‘Cohort Multiple RCT’ (cmRCT) design. Both options come with their own advantages and disadvantages.
Why is it useful?
- Randomised study design: randomisation ensures that the comparison of benefits and harms between treatment groups is not confounded by differences of prognosis at baseline (‘prognostic incomparability’).
- Trial results relevant to real-world populations: a more pragmatic trial enrols participants who are more similar to the target population than those in an explanatory trial, particularly for characteristics that may have an impact on outcome event rates and the treatment response or ‘effect modifiers’, such as age or disease severity. They also use comparators and outcome measures relevant to clinical practice and aim to minimise changes in routine clinical practice during the trial. This should lead to study results that are more generalisable to patients in routine clinical practice than results from a more explanatory trial.
- More realistic reflection of the likely treatment effects: the treatment response in a pragmatic trial is the total difference between treatment strategies, including the associated impact of adherence, placebo and other extraneous effects. Compared with an explanatory trial, this could result in a more realistic reflection of the likely treatment effects in patients treated in routine care.
- Earlier evidence of effectiveness: use of a pragmatic trial may enable evidence of effectiveness to be provided at an earlier stage in the medicine development process, compared with an explanatory trial which only provides evidence of efficacy.
When is it suitable?
- To guide decision-making on prescribing in routine clinical practice: pragmatic trials are suitable when the results of a trial are meant to directly guide decision-making on prescribing in routine clinical practice (optimising the use of new medicines).
- To determine effectiveness once safety (and efficacy) is established: if safety (and efficacy) of treatment has been established in phase 2 and phase 3 clinical trials comparison with usual care in routine clinical practice is a good next step. In some cases, safety and efficacy cannot be assessed further due to there being an acute unmet need, for example the search for a vaccination for a life-threatening and fast-spreading infection. This next step is particularly important when a difference between the effect of the medicine in the phase 2 and phase 3 trials and the effect of the medicine in the real world is expected. This efficacy–effectiveness gap (see Clarify the Issues) may be due to a number of factors, such as a difference in the population studied and the levels of concomitant care provided in study centres, or how the treatments being compared are used in routine practice, including extraneous factors such as co-medication and non-adherence.
What are the limitations?
- Operational challenges: pragmatic trial designs may lead to different and unanticipated operational challenges compared with typical phase 3 (explanatory) trials. For example, involving real-world prescribers of a medicine, may require study sites to be moved from specialised trial centres to medical departments or primary care settings, which may not be fully equipped or experienced in conducting a clinical trial.
- Complex interplay between study design and operational challenges: designers of pragmatic trials need to be aware of the consequences of their study design choices and consider the implications of these choices on generalisability of results to routine practice, validity and precision of results, their acceptability to stakeholders and operational feasibility of the study.
How are they designed?
Defining the research question and developing the trial protocol accordingly are crucial to ensuring that the purpose of a trial is clear, and that reporting is transparent.
The design support tool GetReal Initiative Trial Tool captures the most relevant design choices, their implications and the operational challenges that may be encountered when executing the trial. Use of this tool facilitates a more explicit and transparent trial design process.
For more information about specific study design challenges, see Pragmatic Trial: Study Design Considerations.
What do stakeholders say?
Stakeholders’ views on the ethical challenges and implementation of pragmatic trials in medicines development were elicited as part of the GetReal project through in-depth interviews (Kalkman et al, 2016). Stakeholders interviewed included academia and independent research institutions, the pharmaceutical industry, regulators, health technology assessment (HTA) agencies and patient organisations.
Stakeholder views on the use of pragmatic trials were also elicited through a GetReal case study. For more information, see Early Pragmatic Trials: A Case Study in Chronic Obstructive Pulmonary Disease).
Key contributors
Mira Zuidgeest, University Medical Center Utrecht
Iris Goetz, Lilly
Rick Grobbee, University Medical Center Utrecht