The aim of a pragmatic trial is to capture the true effect of a treatment strategy in the real world. Selecting the setting, study population, mode of intervention, comparator and outcome are crucial in designing pragmatic trials. Combining these design elements with a monitoring and data collection that does not alter the routine care received by patients will support generalisation of the results to the appropriate target patient group in clinical practice. To benefit from the full potential of pragmatic trials, there is a need for guidance and tools for designing these studies in a way that ensures operational feasibility.
The GetReal Trial Tool is a decision support tool for pragmatic trial design. As the generalisability of trial results to routine clinical practice is considered a key aspect of a pragmatic trial, generalisability plays a central role in this tool. The GetReal Trial Tool aims to support trial design teams to maximise generalisability of trial findings to the routine care setting of interest while ensuring validity and operational feasibility of the trial. It provides insights into the possible consequences of more pragmatic trial design choices on generalisability, validity, precision, stakeholder and ethical acceptability and operational challenges of executing the trial.
Below the different study design aspects on which choices have to be made are described:
Selection and inclusion of usual care sites
The sites studied in a trial, with their, specific expertise (of clinicians), care pattern and population, may influence the observed treatment effect and generalisability of the study results to the target population. Actual participation of sites and subsequent patient enrolment also determine successful execution of a trial. A balance is needed between studying representative sites, regarding factors that could modify the treatment effect, and conducting the trial efficiently.
To learn more about possible modifiers of treatment effect (or ‘effect modifier’) related to sites and selecting sites for pragmatic trials (assessment of feasibility and capacity, training and supporting sites) explore the GetReal Trial Tool.
Participant selection, recruitment and retention
The population studied in a pragmatic trial can influence the observed treatment effect in the trial. For example, if drug metabolism changes with age, the observed effects or side-effects could be different when studying a population including children or elderly people. Pragmatic trials aim to include all patients eligible to receive the experimental treatment in real life. However, the population studied is not only determined by the eligibility criteria: some selection will occur when inviting patients for the trial. This may reduce the generalisability of the results to the target population, as well as the efficiency of running the trial.
A balance is needed between including a representative population for modifiers of treatment effect (or ‘effect modifier’) in the target population and the feasibility of running the trial. To learn more about how participant selection influences generalisability, and which factors affect participant recruitment and retention in pragmatic trials explore the GetReal Trial Tool.
Informed consent
A pragmatic trial aims to study the effect of a drug under real-world conditions. However, concerns have been raised that current ethical regulations compromise the real-world nature of a pragmatic trial. To address the challenges of traditional informed consent and explore the possibilities of alternative consent models, the following assumptions were made:
- Not all requirements for regulatory informed consent for randomised trials are always necessary to protect the rights and interest of participants in pragmatic trials.
- The requirements for regulatory consent impede the scientific and societal goals of pragmatic trials.
To learn more about the challenges of informed consent, proposed alternative consent models and their implications explore the GetReal Trial Tool.
Defining questions, choosing comparators, allocating treatments
To adequately guide clinicians’ treatment decisions, pragmatic trials should deliver evidence on the added value of new medicine, compared to usual care for those patients eligible to receive the new treatment in day-to-day clinical practice. However, variations in usual care and suboptimal care may complicate the operationalisation of such a comparator ‘usual care’ arm. Treatment-related study procedures, including allocation and implementation of treatment strategies, should resemble clinical practice as closely as possible, and decisions on drug dosage, co-interventions, and the management of adverse effects should be left to the treating clinician. The specific objectives of the trial should be carefully considered because this determines patient eligibility, allows (or prohibits) treatment switches between arms and determines the suitability of a superiority or non-inferiority trial (used to test a hypothesis that the new treatment is superior or non-inferior to usual care).
To learn more about challenges regarding comparator choice, mode of intervention and treatment allocation for pragmatic trials explore the GetReal Trial Tool.
Outcome selection and measurement
The selection of a primary outcome is a key design choice for any trial. The set of outcomes (primary and secondary) in pragmatic trials should be relevant for patients, clinicians and for decision-making regarding the value of treatment in practice. In contrast to explanatory trials, treatment effect estimates should include the impact of extraneous effects of co-medication, non-adherence and placebo effects as in clinical practice. For this reason patients and clinicians are usually not blinded (open-label) in pragmatic trials, and measurement of outcomes should be planned not to interfere with usual clinical practice.
To learn more about the selection and definition of outcome measures in pragmatic trials, including surrogate outcomes and patient-reported outcomes, and how to measure them while satisfying stakeholders and ensuring valid and precise estimates of treatment effect explore the GetReal Trial Tool.
Collecting and reporting safety data and monitoring trial conduct
Pragmatic trials fall under the category of ‘interventional trial’ in the European Clinical Trial Directive. This sets obligations for monitoring of safety and trial conduct that may affect routine clinical care and have an impact on patient and clinician behaviour. Current international guidelines state that in an interventional trial all serious adverse events must be collected and reported to the trial sponsor within 24 hours, unless specified otherwise in the study protocol. The collection and reporting of non-serious adverse events may be tailored to the study, and depends on the added risks of the trial intervention relative to standard care.
To learn more about possible approaches to ensure the timely capture of sufficiently detailed data on serious adverse events without influencing the generalisability of trial results, and about proportionate and risk-based methodologies for monitoring of trial conduct explore the GetReal Trial Tool.
Data collection
The collection of high quality data is essential to any trial. Collection of such detailed data typically requires on-site staff training in the methods agreed for data collection, including quality checking. These requirements can present a burden to study sites. They may prevent research-naive clinicians from participating in trials, and they may considerably interfere with usual care and the real-world nature of the trial, thus limiting the generalisability of study results. Therefore, pragmatic trials should use existing routine systems, such as electronic health records [EHRs] and processes as much as possible. Typically, pragmatic trials take a hybrid approach between the highly controlled, dedicated case report form (CRF)-based approach and using data routinely collected in, for example, EHRs.
To learn more about challenges with the use of existing routine systems, technical options for data collection, and international initiatives dedicated to improving the standardisation and quality of routinely collected data explore the GetReal Trial Tool.
| In addition to study design considerations, there may be issues with the analysis of pragmatic trials. To see simulation studies completed as part of GetReal that aimed to control for bias with pragmatic trial, see Analysing Pragmatic Trials. |