Effectiveness issues – Population

1. Trial population mix differs from usual practice

Application of trial inclusion/exclusion criteria, together with other factors affecting recruitment, is likely to mean that the characteristics of the trial population differ from those of a population likely to be treated locally in usual practice. There may be differences in the distribution by age, gender, ethnicity, socio-economic factors, co-morbidities or other factors. Firstly, this is of concern if the intervention of interest has differential efficacy across any of these factors. Secondly, even in the absence of such efficacy differentials, the risk profile of the study population for the outcome of interest may differ from that of the population likely to be treated in usual practice, in which case there may be different levels of absolute effect (events averted etc.) in these populations, associated with the efficacy reported in the trial.

2. Trial participants are at a different position in treatment pathway

The ‘Treatment pathway’ refers to the sequence of previous treatments received by patients in the disease area of interest. This is based on patient selection criteria and response to previous therapies, which may be articulated in clinical guidelines for that disease area. The position of patients in the treatment pathway will determine the choice of appropriate comparator or comparators for HTA of the new medicine.

The position in the treatment pathway occupied by trial participants may differ from that which would be occupied in usual practice in the healthcare system of interest. For example, in usual practice a new medicine may be considered for use (only) after the disease has progressed (twice) in patients receiving two types of previous therapy, whereas an inclusion criterion for participants in the trial may be experience of disease progression when receiving (just) one previous therapy. It is possible that the treatment pathway changes during the course of a clinical  trial, as new medicines become available.

3. Trial population (usual care) differs from populations in other studies

Effectiveness results from previous or concurrent studies of therapies in the same disease area (which may be considered ‘comparator’ therapies, including usual care or standard of care) are potentially useful for planning new studies, as well as for inclusion in meta-analyses or indirect comparisons that include results for the new medicine of interest. However, the characteristics of the populations included in these studies may differ from those in the target population for the new medicine. These differences may be related to demographics, risk profile, place in the treatment pathway, concomitant care received or referral practices.

Differences may also be a result of changes in patient management since the data for the comparator therapies (including usual care) were reported. These variations in effectiveness reported for comparator therapies (including usual care) in such trials or non-randomised studies may present problems when using the results for trial design (for example, using inaccurate expected event rates to calculate adequate sample size) or may introduce bias into meta-analyses synthesising results from all relevant trials.

4. Disease area is not well defined

The disease/indication may have an imprecise definition, and so there may be no well-established criteria for defining the population likely to receive the new intervention. A clinical code (ICD or other) may not yet have been established for the disease entity. Correspondingly there may not be any clinical guidelines, and usual care (or standard of care) may not be well defined for the target population within an agreed treatment pathway. This may lead to uncertainty in applying results from clinical trials to the local decision-making context, as well as presenting difficulties in subsequently setting up studies to monitor the usage and effectiveness of the medicine post-launch.