Step 1: Clarify the issues

Clarify and understand the potential issues (or ‘effectiveness issues’) you may have in establishing the relative effectiveness of a medicine (for a definition of effectiveness issues see here).

This section explains the main relevant terminology and then takes you through a series of considerations, questions and examples relevant to the population or patient group, intervention, comparator, and outcome (PICO framework).

  • Consider the questions for your medicine of interest.
  • Explore and identify the effectiveness issues, which may be addressed using real-world evidence (RWE).

When you have clarified the issues, go to Step 2: Find RWE options.

How does effectiveness differ from efficacy?

See the GetReal glossary for definitions of efficacy and effectiveness from the High Level Pharmaceutical Forum. The definitions in the box below have been developed to highlight the differences between key aspects of efficacy and effectiveness in the population, intervention, comparator and outcomes (PICO).

Box. Definitions of efficacy, relative efficacy and relative effectiveness

Efficacy – The change in a key clinical measure compared to placebo in a cohort of well-defined patients under a standardised care protocol.

Relative efficacy – The change in a key clinical measure compared to a standard alternative in a cohort of well-defined patients under a standardised care protocol.

Relative effectiveness – The change in a variety of endpoints of interest to patients and providers compared to the usual care provided in the population of patients identified as eligible for treatment by clinicians under the normal care provided by the healthcare system, subject to free and variable clinician and patient behaviour.

What is the efficacy-effectiveness gap?

The observed discrepancy between the effects of a health intervention in routine clinical practice (effectiveness) compared with the effects demonstrated in randomised controlled trials (efficacy) is increasingly known as the efficacy‑effectiveness gap (adapted from Eichler et al 2011).

For the GetReal narrative review on the background and conceptualisation of the efficacy‑effectiveness gap, see here.

What are the drivers of effectiveness?

The concept of ‘drivers of effectiveness’ has been developed to help understand the difference between efficacy and effectiveness and the efficacy-effectiveness gap. Drivers of effectiveness are contextual factors that interact with the medicine’s pharmacological effect in the real world; if they not properly accounted for, they may have an impact on the effect of the medicine reported in trial. For more information, see here.

Clarify and understand the issues

The differences between efficacy and effectiveness for each aspect of the PICO and the reasons for these differences are described in more detail on the subsequent pages. These pages also take you through a serious of planning questions to anticipate any effectiveness issues that may arise when undergoing health technology assessment (HTA) and specific methods to explore these issues. The questions may be helpful for pharmaceutical companies to consider when developing a medicine’s development plan, for HTA organisations when giving scientific advice or at the HTA scoping process, or for patients to understand what considerations should be made when developing an evidence development plan.

While the questions are presented separately for each aspect of the PICO framework, each aspect is interconnected. Consider all of the aspects and note how these may be relevant to your medicine of interest. Use this when considering different options to address the issues (in step 2).

Select an aspect of the PICO framework below:

Explore issues related to POPULATION Explore issues related to INTERVENTION  Explore issues related to COMPARATOR Explore issues related to OUTCOME

Note: In addition to the methods and questions related to each aspect of the PICO, there are some general methods that should be used to explore what issues may be relevant for a particular medicine. See the box below.

Box. General methods to explore potential effectiveness issues

· Explore patient perspectives (read more)

· Use scientific advice processes (read more)

· Examine existing health technology assessment reports (read more)

· Understand care pathways (read more)