Is the comparator of interest different to that in registration randomised controlled trials?
This page takes you through the differences between efficacy and relative effectiveness with respect to the comparator (table 1 below) and reasons why they may be different. It then outlines possible planning questions which may help you to anticipate potential effectiveness issues that may arise when undergoing health technology assessment (HTA). The page concludes with links to a number of methods that can be used to further explore potential effectiveness issues for a medicine related to the comparator.
Here we define the comparator as the use of alternative medicine(s) given under the usual care provided by the healthcare system, subject to free and variable clinician and patient behaviour.
Table 1. Difference between efficacy and relative effectiveness with a focus on the comparator
| Efficacy
The change in a key clinical measure compared to placeboa in a cohort of well-defined patients under a standardised care protocol. |
Relative effectiveness
The change in a variety of endpoints of interest to patients and providers compared to the usual care providedb in the population of patients identified as eligible for treatment by clinicians under the normal care provided by the healthcare system, subject to free and variable clinician and patient behaviour. |
| a There are good reasons for choice of comparator in an RCT (following rules set by regulatory agencies): ethical considerations, lack of global standards, lack of licensed medicine. b There is a likely to be range of opinion of what constitutes the ‘standard of care’ or the ‘usual care’ comparator for a population according to different decision makers. |
|
Why is the comparator of interest different?
Clinicians decide which treatment to offer taking into account the clinical characteristics, prognosis, circumstances and preferences of the individual patient. They operate in a health system offering guidelines and constraints on care, and they may be incentivised to treat certain patient groups with particular treatments. They may be required to follow guidelines regarding eligibility of patient groups for specific treatments.. Clinicians will need to decide on the best treatment to offer to patients with a condition where there is a licence to administer the medicine but with certain characteristics (for example old age or co-medication) for which subjects were excluded from or under-represented in clinical trials. Clinical experience and peer reviewed guidelines may lead them to use medicines outside their approved indications. They will be comparing new medicines against a variety of alternative options. This real-world clinical practice is likely to vary by healthcare system.
Patients engage with the physician on treatment choice, and patient preference influences the treatments used. Patient adherence to the comparator treatment may be different in real-world practice compared to clinical trials, requiring adjustment in relative effectiveness assessment.
Healthcare systems set priorities and write clinical guidelines, for example based on health technology assessments and expert opinion. There may be specific restrictions on patient eligibility for medicines, and reimbursement may only be available for sub-groups of patients with conditions that have a license to receive the medicine due to considerations of value or budget impact. There may be specific rules for comparisons required for HTA or reimbursement based on cost, license or perceived best practice. These are likely to vary by healthcare system.
Potential assessment issues and related planning questions
The table below includes potential issues that may arise in the assessment of a medicine and questions that may help to understand these issues.
Table 2. Issues and planning questions related to the comparator
| Issue | Planning questions |
| There are no direct comparative trials between the new medicine and alternatives of interest
There are only placebo controlled trials Active comparator in the RCT is not usual care in the population Active comparator in the RCT is not the standard comparator of interest to a decision maker |
• Have the key decision makers identified the comparator of interest to them?
• What is the usual care for the target population for this decision maker? • Is the comparator of interest licensed for the indication of the RCT? |
| Although the comparator included in the RCT is appropriate, the study protocol dose and administration does not reflect real-world clinical practice (which may not be as licensed) | • What are the dose and administration choices (for example, route, combinations, titration schedules, medicine holidays) used in actual clinical practice?
• How does this compare to the licensed use? • What is the real-world patient adherence to treatment and what drives this? |
| The comparator included in the RCT is appropriate for the RCT population but is not appropriate for the population of interest (see Is the population of interest different to that in registration randomised controlled trials?). For example, comparator X is not appropriate for a population defined as ‘not responding to X’ | • What are the comparators and dose and administration options that are used in different populations of interest (for example, at different points in the care pathway)? |
| Although the comparator included in the RCT was appropriate at the start of the trial, a different comparator may be of interest at launch. This may be because of changing clinical practice, new launches or new safety concerns. The best practice use of any comparator in actual clinical practice may change over time | • Are new clinical guidelines, major new trials or medicine launches expected between the start of the RCT and marketing authorisation?
• Has actual clinical practice (medicine choice, dose and administration) evolved between designing the RCT and marketing authorisation? |
| Study subjects in either arm of the RCT may be switched to a new follow-up treatment (or to the other arm) following a primary endpoint event. This confounds the assignment of longer-term outcomes to each study treatment | • Can feasible treatment pathways be identified that maintains assignment of outcomes to study treatment options, beyond the primary endpoint event? |
| The network of indirect comparisons from all RCTs is not viable, or it is not robust | • What network of indirect comparisons can be created from trials prior to the RCT? (see Overview of evidence synthesis and network meta-analysis for more information about indirect treatment comparisons and network meta-analysis)
• Which new comparative trials would enhance the existing network? • Can new techniques including real-world data in the network be applied? |
Methods to explore potential issues
To consider how a change or choice in the comparator may impact the success of a trial, there are a number of methods you may use. See Methods to Explore and Identify Drivers of Effectiveness.
To explore the comparator that is most likely considered to be relevant to decision-makers, see Understanding the Care Pathway or the Patient Journey.
To start to find potential options using RWE to address the challenges you have identified, see
OR