Effectiveness issues – Comparator

1. Trial comparators do not include current usual care or
standard of care

Current usual care, or standard of care, for the healthcare system of interest is not included as a comparator in the clinical trial. This may be because there is wide variation in usual care across healthcare systems, so that not all options can be included in a single study. A ’usual care’ medicine in the country of interest may not be licensed, reimbursed or recommended for use (for example, in clinical guidelines) in some study countries, preventing its inclusion in the trial. In many situations a placebo-controlled trial may have been required to support marketing authorisation by regulatory agencies, especially where there are concerns about the safety of a new medicine. This might preclude the inclusion of usual care as a comparator in the trial. It is possible that more than one ‘usual care’ comparator is relevant for different segments of the target population, for example if a new diagnostic paradigm, such as a genetic marker, has been used to identify patents suitable for an alternative therapy.

In the absence of a direct head-to-head trial, any comparison with usual care based on clinical trial data will need to rely on an indirect comparison, for example a network meta-analysis based on an evidence network of all trials in populations with the disease of interest. Such analyses depend on statistical modelling assumptions, for example concerning the homogeneity of the results across the source trials, as well as similarity in the design of the source trials, for example, study durations, study populations and definitions of health outcomes. Results of such meta-analyses may be associated with high levels of uncertainty. They are viewed with caution by some decision-makers because the methods used are quite new (not yet fully ‘tried and tested’), are quite complex (loss of transparency) and are not yet widely understood.

2. A trial-based indirect comparison is not possible

In a situation where there is no direct (head-to-head) trial comparison of the new medicine with usual care, or standard of care, for the healthcare system of interest, an indirect comparison (with usual care) may be required by some decision-makers. However, it may not be possible to construct a connected evidence network based on available trials to support such an indirect comparison. This is likely to be the case if there are no trials comparing the new medicine and usual care with a common alternative therapy option, such as placebo. It is also possible that patient groups in studies directly comparing the new medicine or usual care with alternative therapies are sufficiently different to preclude these studies being pooled as part of a network meta-analysis.

3. Trial participants withdraw from therapy or cross over
between treatment groups

Differential withdrawal rates between study arms in a trial may complicate interpretation of the findings, especially if this results in a divergence between intention-to-treat and on-treatment results. If trial participants cross over between treatment groups after reaching a study endpoint (for example, disease progression in cancer) the ability of the trial to report unbiased comparisons for longer-term ‘effectiveness’ outcomes such as overall survival is compromised.

4. Administration of trial comparator differs from usual practice

Although usual care, or standard of care, for the healthcare system of interest is included as a comparator in the trial, its administration (for example dose, dose titration/escalation, frequency, route of administration, monitoring) in the trial setting may differ from its use in routine clinical practice. This may raise concerns about the transferability of study results to routine clinical practice. In some situations, the clinical background and skill level of the administering clinicians may be a relevant factor.