Intervention: effectiveness vs. efficacy

Is the intervention of interest different to that in the registration randomised controlled trials?

This page takes you through the difference between efficacy and relative effectiveness with respect to the intervention (Table 1 below) and reasons why they may be different. It then outlines possible planning questions which may help you to anticipate potential effectiveness issues that may arise when undergoing health technology assessment (HTA). The page concludes with links to a number of methods that can be used to further explore potential effectiveness issues for a medicine related to the intervention.

Here the intervention is defined as the use of the new medicine given under the usual care provided by the healthcare system, subject to free and variable clinician and patient behaviour.

Table 1. Difference between efficacy and relative effectiveness with a focus on the intervention

Efficacy

The change in a key clinical measure compared to placebo in a cohort of well-defined patients under a standardised care protocol.a

 Relative effectiveness

The change in a variety of endpoints of interest to patients and providers compared to the usual care provided in the population of patients identified as eligible for treatment by clinicians under the normal care provided by the healthcare system, subject to free and variable clinician and patient behaviour.
aThere are good reasons for RCT care protocols (following rules set by regulatory agencies): ethical considerations, need to ensure best practice and standardisation across all patients.

Why is the intervention of interest different?

Clinicians decide which treatment to offer, taking into account the clinical characteristics, prognosis, circumstances and preferences of each individual patient. They operate in a health system offering guidelines and constraints on care. They may be required to follow guidelines regarding use of new treatments. Clinical experience and peer-reviewed guidelines may lead them to use medicines outside their approved indications and in ways not tested within RCTs. There may be particular interest in establishing guidance on appropriate use of medicines, such as stopping rules and use beyond the duration tested in the RCTs. Prescribers will have a different range of experience and resources compared to investigators in RCT centres, and the actual clinical care context will therefore be more variable. This real-world clinical practice is likely to vary by healthcare system.

Patients engage with the clinician on treatment choice, and patient preference influences the treatments used. Patient adherence to a new treatment may be different in real-world practice compared to clinical trials, requiring adjustment to the relative effectiveness assessment.

Healthcare systems set priorities and write clinical guidelines, for example, based on health technology assessment and expert opinion. There may be specific restrictions on patient eligibility for new medicines, and reimbursement may only be available for sub-groups of patients with conditions that have a license to receive the medicine due to considerations of value or budget impact. There may be particular interest in establishing guidance on appropriate use of medicines, such as stopping rules and use beyond the duration tested in the RCTs.

Potential assessment issues and related planning questions

The table below includes potential issues that may arise in the assessment of a medicine and questions that may help to understand these issues.

Table 2. Issues and planning questions related to the intervention

Issue Planning questions
The use of a new medicine in the RCT is appropriate, but there is a risk that the study protocol dose and administration does not reflect the post-launch dose and administration in real-world clinical practice

NB the real world dose and administration may diverge from licensed use

 •      Is there reason to believe that clinicians will start to use the new medicine in different ways?

•      Is there a reason to believe that patients will not adhere to prescribed treatment?
There is no information on how to use the new medicine beyond the period tested in the RCT •      Is there a clinical rationale for indefinite use, or for periodic reassessments or for stopping and restarting?

•      Is it possible to assess this in an extended trial programme?
There is no information on use in combination with other medicines •      Is combination therapy anticipated?

•      Is this possible to assess within the trial programme?
The reported efficacy is dependent on the quality of supportive clinical care; but local clinical practice is not the same as RCT protocol care at specialist RCT centres. There is doubt that the same effect size will be achieved locally •      How does clinical care (guidelines, resources, technology, clinical choice) differ between the real world and the RCT protocol?

•      What is the diversity of experience and expertise of prescribing clinicians?

•      How likely is it that any differences would influence effect size?

Methods to explore potential issues

To start to find potential options using RWE to address the challenges you have identified, see

OR