Conducting a literature review

A literature review should be used to systematically identify characteristics related to patients, the disease or the healthcare system that may interact with the medicine’s pharmacological effect (drivers of effectiveness).

Two types of literature review can be conducted:

  • Systematic literature review (SLR): based on a clearly formulated question using explicit and systematic methods to identify, select, critically appraise, qualitatively analyse and interpret exhaustive research relevant to the question; the studies are identified, selected and critically appraised by two independent reviewers. Where appropriate and feasible (i.e. allowed by homogeniety of studies), a quantitative analysis and a statistical summary, such as a meta-analysis, is performed. SLR is the most comprehensive and preferred type of review for HTA submission.
  • Focused literature review (FLR): (also known as a ‘targeted literature review’) is also based on a clearly formulated question using explicit methods to identify, select, critically appraise, qualitatively analyse and interpret key relevant research. However, this approach is less comprehensive than a SLR since the focus is on key research relevant to the question, and is performed by a single reviewer. It may be used for initial work to understand the issues in the area better and may precede a SLR.

For a more detailed description of the differences between a SLR and FLR, see [INSERT LINK TO D2.2].

Structure of the literature review

Both types of review should be structured using the PICOS framework (see below). Considerations for planning a literature reviews include:

  • Population: if the disease is rare, it might be worth exploring diseases likely to have similar drivers of effectiveness (for example, all haematological malignancies rather than Hodgkin’s lymphoma alone, such as the approach used in the GetReal case study).
  • Intervention: the medicine of interest, or class of medicine of interest with the same mode of action (for example,antipsychotic drugs). In rare diseases or diseases with very few treatment options, the search may encompass all available treatments.
  • Comparator: if an active comparator will be obviously used in randomised controlled trials (RCTs), it should be included in the search (for example, the use of insulin in diabetes). If a broad search is carried out, any comparator should be included.
  • Outcome: one or several outcomes of interest may be defined, such as those used as a measure of effectiveness (for example, overall survival in haematology or oncology) and/or those potentially impacted by drivers of effectiveness (for example, HbA1c in diabetes).
  • Study type: RCTs and observational studies may be included depending on the objective. Previously published systematic literature reviews or meta-analyses may be a helpful source of information. The approach chosen will depend on the aims of the review.

Approaches to the literature review

There are three different approaches that may be considered, depending on the aim of the literature review:

  • Focus on studies that explicitly explore effect-modification and interaction (RCT or observational study): to understand if response to a medicine may differ in general for certain subgroups of patients (i.e. not in relation to the impact this might have in terms of an efficacy-effectiveness gap).
  • Focus on exploring a gap in evidence between different study designs: examining study protocols of RCTs and observational studies for different characteristics between study designs, such as inclusion/exclusion criteria or decisions on treatments (for example, treatment withdrawals). The objective is to identify studies that explore this gap (for example, studies comparing data in RCTs and observational studies), and/or characteristics that are potential drivers of effectiveness or may account for an efficacy-effectiveness gap (for example, older patients excluded from RCTs when age may be a driver of effectiveness).
  • Directly investigate a potential efficacy-effectiveness gap by reviewing RCTs and observational studies: this may be combined with a meta-analysis of the effect estimates from the identified RCTs and observational studies. Efficacy results from RCTs are compared with effectiveness results of observational studies using the same outcome; a gap may be identified and patient-related or disease-related characteristics potentially affecting the outcome can be compared across study designs (RCTs and observational studies).

The following table gives more information on the strengths and limitations of each of these approaches.

Table. Strengths and limitations of approaches to literature reviews

Approach Strengths Limitations
Focus on studies that explicitly explore effect-modification and interaction The search will target the interaction regardless of the study type or the medicine used

A lot of information can be retrieved if the disease has been explored for enough time

Uses scientifically recognized (SLR) or robust (FLR) methodology. A SLR can be published

This approach is not possible for rare diseases or diseases where subgroups were not explored

Only characteristics that were explored can be identified

There is a potential for publication bias

Focus on exploring a gap in evidence between different study designs Directly retrieves the characteristics known or suspected to play a role in efficacy-effectiveness gap and/or drivers of effectiveness and allows those factors to be taken into account in subsequent data analysis

Relatively easy and quick way (especially using a FLR) to obtain information since it is already published

Uses scientifically recognized (SLR) or robust (FLR) methodology. A SLR can be published

It is generally a good starting point in order to evaluate existing knowledge in terms of efficacy-effectiveness gap and drivers of effectiveness, and the data gaps before embarking in further data generation

Relies on available evidence and its quality, which depends on the objectives and data presented by the authors

Interactions or correlations between factors can only be explored if they are included in the publications

Explorations on unknown factors are usually limited

There is a potential for publication bias

Directly investigate a potential efficacy-effectiveness gap by reviewing RCTs and observational studies Uses scientifically recognized (SLR) or robust (FLR) methodology. A SLR can be published

The investigator has full control over what to investigate

Relies on available evidence and its quality, which depends on the objectives and data presented by the authors

Only characteristics that were explored can be identified

Explorations on unknown factors are usually limited

There is a potential for publication bias

Abbreviations: FLR, focused literature review; RCT, randomised controlled trial; SLR, systematic literature review.

GetReal case studies using literature reviews to identify drivers of effectiveness

Key contributors

Clementine Nordon, LASER 
Robert Olivares, Sanofi