Skip to Main Content

Editorial

Taking health systems research syntheses to the next level: overviews of systematic reviews

M Kent Ranson, David B Evans
Editorial Article

The vision of universal health coverage is that all people obtain the health services they need, while not being pushed into poverty by paying out-of-pocket costs for health care.[1] To achieve this, health systems must be strengthened, based on the best available information. Four new Cochrane Overviews summarize evidence from systematic reviews about the effects of delivery, financial, and governance arrangements, and implementation strategies for health systems.[2-5] The overviews, published to coincide with the Global Evidence Summit (www.globalevidencesummit.org), are accompanied by more than 100 SUPPORT Summaries – short, structured summaries of the systematic reviews selected for inclusion in the overviews. In addition to summarizing the main findings of systematic reviews, SUPPORT summaries also assess the quality of the evidence for the main comparisons and reflect on the relevance of the results for low-income countries. Together, the overviews and SUPPORT Summaries provide a valuable tool set for both researchers and policy-makers.

The overviews summarize the findings of 124 systematic reviews, covering delivery (51), implementation (39), governance (19) and financing (15). The number of primary research studies captured by the four overviews, a total of 2630, is impressive. One can imagine interesting and informative graphics that might be produced based on the included primary research. For example, over time, is the relative proportion of evidence that is high or moderate certainty, or that was conducted in low- or middle-income countries, on the rise?

These overviews highlight that for the majority of interventions that have been investigated in primary research, little can be said about the overall impact because the only evidence is of low or very low certainty. This is likely to be linked to the great heterogeneity not only in the health systems of the countries in which the interventions were implemented, but also in the way the interventions were implemented. Nonetheless, policy makers regularly have to make decisions on issues around which the overviews offer no direction – everything from decentralization of decision-making and control of corruption to financial incentives for healthcare workers.[3,4] We now need research that will help us to understand the heterogeneity in the existing evidence base. This will likely involve turning to study designs other than experimental or quasi-experimental studies. There clearly remains a role for reviews that include studies that have useful information for policy makers, including qualitative studies, which would be a valuable additional form of evidence to feature in Cochrane Overviews.

The authors point out that these overviews can also help to identify needs and priorities for new primary research on health system arrangements and for systematic reviews. In addition to identifying topical gaps, other important considerations are the likelihood that the research can have desirable impacts – for example, addressing an important source of burden of disease, or equity, or financial risk protection.[6]

These overviews may also help to identify ways in which that quality of primary research or subsequent systematic reviews can be improved. It is striking how few studies looked at equity outcomes, or examined the costs and cost-effectiveness of interventions, suggesting that these may warrant additional attention. Very few studies reported adverse or unexpected effects, which calls into question how carefully they were monitored. Understanding the nature of the heterogeneity in the way interventions were implemented and how this might have effected outcomes is an important issue for systematic reviews.

Conducting systematic reviews, and overviews, requires many people and person-hours. By the time an overview is published, some of the primary data included may be outdated, and even newly published overviews may exclude relevant, recently published primary data. Given these inefficiencies, perhaps this is an area ripe for innovation? Cochrane has published its first living systematic reviews, which will be updated continually.[7] Furthermore, software such as EPPI Reviewer (http://eppi.ioe.ac.uk/cms/er4; available free to authors working on Cochrane Reviews) facilitates collaborative work of reviewers from across countries, languages, disciplines, and levels of experience. Perhaps such technologies will come together in the development of an online platform ("Wik-epi-pedia"?) where primary research (upon publication) would immediately be screened for inclusion in real-time systematic reviews, and these synthesized upwards into overviews.

Related Cochrane Reviews