Healthcare setting-level drivers of effectiveness: a case study in schizophrenia


Characteristics of healthcare settings and of healthcare providers are usually more varied and heterogeneous in real life than in randomised controlled trials (RCTs). If key characteristics of healthcare settings and of healthcare providers that may have an impact on the effectiveness of drugs are not well captured in trials, there is a risk of an efficacy-effectiveness gap (for a definition, see here).

Identifying the drivers of effectiveness (for a definition, see here) related to setting is important to minimise this gap in evidence and help decision makers understand the effectiveness of these medicines in a real-world population.

A separate GetReal case study examined potential drivers of effectiveness at a patient-level in schizophrenia .

What was examined in this case study?

This case study aimed to explore the impact of several healthcare setting characteristics on the effectiveness of antipsychotic drugs in patients with schizophrenia. The Schizophrenia Outpatient Health Outcomes (SOHO) prospective observational cohort study was used. Between 2001 and 2002, 899 psychiatrists from 10 European countries recruited into the study 10,218 patients with schizophrenia. Patients were treated as outpatients and had started or switched their antipsychotic drug.

The effectiveness of ’drug A’ was measured by assessing the evolution of schizophrenia symptoms 3 months after baseline and comparing the results with those for other antipsychotic drugs.

The impact of different healthcare settings on the effect of the antipsychotic drug, measured by the severity of schizophrenia symptoms, was assessed at different levels of setting:

  • Country-level: this included the density of psychiatrists and the efficiency of the healthcare system (Health Service Efficiency [HSE] score).
  • Psychiatrist-level: this included the gender of the psychiatrist, whether the psychiatrist worked in the private or public sector, and the level of experience (duration of practice).

The effect was explored using a mixed multivariate linear regression model, which was adjusted for patient-related confounders and clustering data at the psychiatrist-level (using a random intercept model). The interaction with key healthcare setting characteristics was also explored using interaction terms.

What were the findings and conclusions?

Patients who started drug A had the same level of improvement as patients starting all other antipsychotic drugs.

Three healthcare setting characteristics were found to be independently associated with better outcomes for patients (independent from patient- or disease-related characteristics):

  • less experience of psychiatrists (i.e. shorter duration of practice)
  • higher HSE score
  • greater density of psychiatrists.

Psychiatrists’ experience or duration of practice was found to be a modifier of the effect of drug A: higher levels of symptom improvement were found in patients cared for by psychiatrists with less experience.

23% of the overall uncertainty in the effect (or variance) could be explained by unmeasured confounders at a psychiatrist-level.

A practical implication of these findings is that before designing a trial or when analysing data from a trial, the impact of a settings characteristics on the medicine’s effect should be assessed.

Key contributor

Clementine Nordon, LASER