Is the outcome of interest different to that in registration randomised controlled trials?
This page takes you through the difference between efficacy and relative effectiveness with respect to the outcome (table 1 below), and reasons why they may be different. It then outlines possible planning questions which may help you to anticipate potential effectiveness issues that may arise when undergoing health technology assessment (HTA). The page concludes with links to different methods that can be used to further explore potential effectiveness issues for a medicine related to the outcome.
Table 1. Difference between efficacy and relative effectiveness with a focus on the outcome
| Efficacy
The change in a key clinical measurea compared to placebo in a cohort of well-defined patients under a standardised care protocol. |
Relative effectiveness
The change in a variety of endpoints of interest to patients and providers compared to the usual care provided in the population of patients identified as eligible for treatment by clinicians, under the conditions for normal care in the healthcare system, subject to free and variable clinician and patient behaviour. |
| aThere are good reasons for choice of outcome measure in the RCT (following rules set by regulatory agencies): feasibility/time required, some outcomes may be confounded by RCT design, there may be no single global acceptable measure | |
Why is the outcome of interest different?
Clinicians decide which treatment to offer, taking into account the clinical characteristics, prognosis, circumstances and preferences of the individual patient. They operate in a health system offering guidelines and constraints on care and may be incentivised to maximise certain outcome metrics. They may not have access to state-of-the-art technology to measure all clinical markers.
Patients engage with the clinician on treatment choice. Patient-reported outcome measures are constructed to capture the elements that are most important to them. Patients may prioritise certain outcomes that differ from priorities of payers. They may be willing to accept more risk of adverse events or harms, to balance against the potential benefit.
Healthcare systems set priorities and develop clinical guidelines, for example based on health technology assessments and expert opinion. There may be specific outcomes of interest from a health policy or economic perspective.
Potential assessment issues and related planning questions
The table below includes potential issues that may arise in the assessment of a medicine and questions that may help to understand these issues.
Table 2. Issues and planning questions related to the outcome
| Issue | Planning questions |
| There are no data on the outcome of interest in the RCTs | • What are the outcomes of interest to each decision maker?
• Can they be included in the RCTs? |
| The outcome of interest requires extrapolation from a biomarker or over an extended period of time. The method of extrapolation is not acceptable, or the results of extrapolation are too uncertain | • Do the trials use surrogate or intermediate measures requiring extrapolation to a final outcome?
• Has the method of extrapolation been shown and validated quantitatively? • Is the modelling methodology acceptable to decision makers? |
| The outcome of interest has been included as a secondary endpoint in the trial, but is underpowered (for drawing conclusions about comparisons) or is inconsistent with the primary endpoint | • Are secondary outcome measures adequately powered?
• Is there a reason to believe that results for the outcome of interest will vary (e.g. by subgroup) in ways that the primary endpoint does not? |
| The primary outcome of the RCT is a composite endpoint. The individual components of the composite endpoint may have differing effect sizes. The components may not be of equal importance clinically or economically | • What is the importance of each component of the composite endpoint to different stakeholders?
• Are secondary analyses of individual components adequately powered? |
| Outcomes measured in the RCT are not captured in usual clinical practice, or they are captured in a significantly different way. So there is no baseline real-world data, and no practical metric for monitoring future performance | • What are the key outcomes used in usual clinical practice?
• How are these measured and captured? • Is this different to what is implemented in the RCT protocol? • Will implementation of (RCT) outcome measures in real-world practice require change in clinical practice or extra resources? |
| The definitions of outcome in the RCT differs from those in previous trials in this disease area, making comparison with alternative treatments more difficult | • How have outcomes been defined?
• Can past results be re-analysed according to new definitions? |
| There is no intermediate measure of response. Patients that do not benefit cannot be identified early, potentially leading to poor effectiveness results for the new medicine | • What response measure could give early indication of benefit? |
Methods to explore potential issues
For patient perspectives in medicines development, see Exploring and Incorporating Patient Perspectives in Medicines Development.
For exploring what outcomes are relevant to decision-makers, consider Reviewing Existing Health Technology Assessments.
To start to find potential options using RWE to address the challenges you have identified, see
OR