Outcome: understand effectiveness vs. efficacy

Is the outcome of interest different to that in registration randomised controlled trials?

This section will take you through the difference between efficacy and relative effectiveness with respect to the outcome (table 1 below) and reasons why they may be different. It will then outline possible planning questions which may help you to anticipate potential effectiveness issues that may arise when undergoing health technology assessment (HTA). The page will conclude with links to a number of methods that can be used to further explore potential effectiveness issues for a medicine related to the outcome.

Table 1. Difference between efficacy and relative effectiveness with a focus on the outcome

Efficacy

The change in a key clinical measurea compared to placebo in a cohort of well-defined patients under a standardised care protocol.

Relative effectiveness

The change in a variety of endpoints of interest to patients and providers compared to the usual care provided in the population of patients identified as eligible for treatment by clinicians under the normal care provided by the healthcare system, subject to free and variable clinician and patient behaviour.

aThere are good reasons for choice of outcome measure in the RCT (following rules set by regulatory agencies): feasibility/time required, some outcomes may be confounded by RCT design, there may be no single global acceptable measure

Why is the outcome of interest different?

Clinicians decide which treatment to offer taking into account the clinical characteristics, prognosis, circumstances and preferences of the individual patient. They operate in a health system offering guidelines and constraints on care, and may be incentivised to maximise certain outcome metrics. They may not have access to state-of-the-art technology to measure all clinical markers.

Patients engage with the clinician on treatment choice. Patient-reported outcome measures are constructed to capture the elements that are most important to patients. Patients may prioritise certain outcomes not prioritised by the payers, and may accept more risk.

Healthcare systems set priorities and write clinical guidelines, for example based on health technology assessments and expert opinion. There may be specific outcomes of interest from a health policy or economic perspective.

Potential assessment issues and related planning questions

The table below includes potential issues that may arise in the assessment of a medicine and questions that may help to understand these issues.

Table 2. Issues and planning questions related to the outcome

Issue Planning questions
There are no data on the outcome of interest in the RCTs •      What are the outcomes of interest to each decision maker?

•      Can they be included in the RCTs?

The outcome of interest requires extrapolation from a biomarker or over an extended period of time. The method of extrapolation is not acceptable or the results of extrapolation are too uncertain •      Do the trials use surrogate or intermediate measures requiring extrapolation to a final outcome?

•      Has the method of extrapolation been shown and validated quantitatively?

•      Is the modelling methodology acceptable to decision makers?

The outcome of interest has been included as a secondary endpoint, but is underpowered or is inconsistent with primary endpoint •      Are secondary outcome measures adequately powered?

•      Is there reason to believe that results for the outcome of interest will vary (e.g. by subgroup) in ways that the primary endpoint does not?

The primary outcome of the RCT is a composite endpoint. The individual components of the composite endpoint may not show consistent effect sizes, and are not of equal importance •      What is the importance of each component to different stakeholders?

•      Are secondary analyses of individual components adequately powered?

Outcomes measured in the RCT are not captured in usual clinical practice, or are captured in a significantly different way. So there is no baseline real-world data, and no practical metric for monitoring future performance •      What are the key outcomes in clinical practice?

•      How are these measured and captured?

•      Is this different to the RCT protocol?

•      Will use of outcome measure in real-world practice require change in clinical practice or extra resources?

The definition of outcome in the RCT differs from previous trials in this disease area, making comparison with alternative treatments more difficult •      How have outcomes been defined?

•      Can past results be re-analysed according to new definitions?

There is no intermediate measure of response. Patients that do not benefit cannot be identified, potentially leading to poor value for the new medicine •      What response measure could give early indication of benefit?

Methods to explore potential issues

To explore and incorporate patient perspectives in medicines development, see here.

To explore what outcomes are relevant to decision-makers, consider reviewing existing health technology assessments.

OR