1. Administration of therapy is inconsistent with usual practice
The administration of the study medicine in trials (for example, dose, dose titration/escalation, frequency, route of administration, monitoring) may differ from the schedule that is likely to be used in clinical practice. This is perhaps more likely if the new therapy is added to current usual care, or if the treatment itself may be intentionally misused by patients if its administration is not under strict control (for example, opioids in pain relief). For some medicines, the clinical background and skill level of the administering clinicians may be important.
2. Stopping rules for therapy are unclear
Health technology assessment (HTA) agencies (and healthcare payers) are interested in knowing when new therapies should cease to be used by patients, for example due to lack of response or another measure of efficacy, and potentially when there is no likely patient benefit even in the presence of continuing efficacy. ‘Stopping rules’ may not be well defined in clinical practice (especially clinical guidelines), and further evidence for such ‘rules’ may not available from the clinical trials. Stopping rules used in the trials may not match those used in local clinical practice.
3. Adherence in study differs from usual practice
Outcomes reported in trials are usually for study participants at a high level of adherence to the study medicine and comparators. However, in usual practice lower levels of adherence are expected, potentially with different levels of adherence for different therapy options resulting from differences in side effects or methods of administration. In the absence of real-world evidence, an understanding of the relationship between effectiveness and adherence is required to project estimates of effectiveness in usual practice (with sub-optimal adherence) from efficacy reported in trials. However, the relationship (often non-linear) may be difficult to predict. Having real-world data on adherence alone is insufficient to estimate effectiveness.