Effectiveness issues: challenges in demonstrating effectiveness

What are effectiveness issues?

When evidence of (relative) effectiveness of a new medicine is presented to health technology assessment (HTA) agencies a variety of issues or challenges to the evidence may be raised. These are referred to here as effectiveness issues.

Effectiveness issues may be classified using the PICOS acronym, which defines five categories of issues: population, intervention, comparator, outcomes, and study design or study setting.

Choose a PICOS category from the left-hand menu to find out more information about the different types of issues.

Understanding effectiveness issues

Sponsors of new medicines aim to understand likely effectiveness issues, through:

  • review of recent assessments for medicines with similar characteristics
  • interpretation of guidance for making submissions
  • direct consultation with experts and the agencies themselves (‘scientific advice’).

Using this information together with a thorough understanding of the new medicine, sponsors will design and execute programmes of studies and analyses that generate evidence of relative effectiveness in a way that is likely to be acceptable to different decision makers. In this way effectiveness issues will be anticipated and prevented.

Effectiveness issues in decision-making

For decision-making the main focus is on the pivotal studies (Phase 2b and Phase 3 clinical trials) that are powered to detect superiority or non-inferiority in an efficacy outcome of importance. In addition to evidence from these studies, the evidence submitted to HTA agencies is likely to contain further information from systematic reviews and meta-analyses, indirect comparisons (especially valuable in the absence of head-to-head data), statistical or other predictive models and supporting observational studies.

HTA agencies consider the extent to which the evidence presented is relevant to clinical practice and the population in their country who would be likely to be eligible to receive the new medicine. They assess whether the evidence presented can form the basis of a decision, and the risks of making an incorrect decision.

Agencies may place a different emphasis on particular aspects of the evidence presented, depending on their views of the strength of different types of evidence and what type of evidence is optimal to support their decision-making in the particular disease area. It is quite likely that perfect packages of evidence cannot be generated for all decision makers, especially given these differences and time/budget constraints for pivotal studies.


P. Population

Is the study population well-defined and relevant? Does it represent the population for whom the decision is being made?

Read More

I. Intervention

Is the study intervention (diagnosis, medicine, administration, monitoring) applicable in routine practice?

Read More

C. Comparator

Does the study comparator represent the current usual care (or Standard of Care), which would be replaced by, or added to, the new intervention?

Read More

O. Outcome

Are the outcomes that are presented (or predicted) relevant and sufficient to form the basis of a decision?

Read More

S. Study Design / Setting

Does the study design or study setting (including timing of the study) support the demonstration of effectiveness?

Read More