Analysing pragmatic trials

While it is crucial to pay special attention to the design of pragmatic trials (see here), it is critical to ensure the study results are analysed appropriately to ensure incorrect conclusions are not drawn from these results. While a well-considered analytical strategy is important with all studies, the additional design considerations with pragmatic trials can add additional complexity for analysis.

For example, despite the use of randomisation, pragmatic trials may still be subject to confounding bias, where there are systematic differences between arms that is independently related to the outcome. For example, a pragmatic trial attempts to reflect current practice as much as possible, such using ‘usual care’ in the control arm. However, ‘usual care’ will often not be one treatment and so there may be a heterogenous control arm in a pragmatic trial and patients may be selected for treatments in the control arm on the basis of their individual characteristics (to reflect the ‘real world’), with possible influence on the treatment effect. This heterogeneity needs to be accounted for in the analysis.

See below simulation studies conducted as part of GetReal examining ways to adjust analyses in pragmatic trials:

  • Simulation to adjust for confounding in a pragmatic trial with a heterogeneous comparator arm – see here
  • Simulation to adjust for confounding in a pragmatic study with time-to-event outcome – see here
  • Comparison of methods for competing risks in pragmatic trials – see here.
Some of the methods examined in the above simulation studies have also been used to adjust for confounding in non-randomised or observational studies (see here).