New data initiative helps Morocco lift lid on country’s risk factors for noncommunicable diseases

American author Mark Twain was fond of saying that there are 3 kinds of lies: lies, damned lies and statistics. To which WHO Director-General Dr Margaret Chan would respond, “What gets measured gets done.”

Reliable statistics are vital in many areas of life – none more so than in public health, when knowing how many people die of or are made sick by particular diseases is essential for policymakers and planners to make the best decisions about how to prevent suffering and save lives. But how do they know the data on which they base those decisions are reliable?

Nothing beats counting individual cases and deaths. But many countries lack the resources and infrastructure to gather good quality data, and data are often missing for certain populations and time periods. Countries sometimes count in different ways, making comparisons problematic. To fill those gaps, WHO and other researchers calculate estimates of health data based on complex methods, sometimes using indicators such as gross domestic product and education that are known to be drivers of health outcomes, as well as comparable data from other countries in the same region or with similar characteristics.

Here’s the problem: around the world, different statisticians use different ways of calculating estimates, with (unsurprisingly) different results. And they haven’t always shared all the details on how they arrive at their estimates; there has been no agreement on how to disclose methodologies.

18 best practices for health estimates

All that is about to change. The Guidelines for Accurate and Transparent Health Estimates Reporting, or GATHER, is a checklist of 18 best practices that sets the standard for disclosing how health estimates are developed. The GATHER checklist was developed by WHO and researchers from around the world including the Institute for Health Metrics and Evaluation (IHME) at the University of Washington in Seattle, and was published today in the Lancet and PLoS Medicine.

“When the health and well-being of millions of people is at stake you need to be sure you have the best possible information to make the best possible decisions,” said Dr Ties Boerma, Director of WHO’s Department of Information, Evidence and Research. “GATHER is a crucial step towards making sure health estimates can stand up to scrutiny.”

GATHER includes requirements for disclosing which data are used to calculate estimates, and for making them available to others. It also includes a requirement to disclose how the computer code used to crunch the numbers can be accessed, making it possible for others to reproduce estimates, making them more robust.

Both WHO and IHME have agreed to comply with GATHER when they publish new global health estimates, and hope that medical journals will also start requiring authors to comply. Several estimates compiled by WHO already follow GATHER principles, including child mortality, childhood causes of death, maternal mortality, and HIV estimates.

Better transparency improves credibility

GATHER will also help researchers to be more efficient and make better use of research funds; greater transparency will enable researchers to build on the work done by others, instead of wasting months or even years of work trying to reproduce it.

“Transparency gets to the essence of credibility in health science,” said Dr Christopher Murray, Director of IHME. “If researchers are not willing to be completely open about their sources of information and methods used for analysis, the credibility of their findings may be questioned. Those who adhere to the guidelines will raise the bar in terms of research excellence. And all of us will benefit from that higher bar.”

The production of global health estimates has increased tremendously in the past years, driven mainly by an increasing global demand for data on key indicators, such as those used to measure progress towards the Millennium Development Goals (MDGs).

The Sustainable Development Goals, which are far broader in scope than the MDGs, will only increase the demand for more, better data. Through initiatives such as the Health Data Collaborative, WHO is working with countries and partners to meet that demand.