http://bmj.com/cgi/content/full/323/7308/295

 

BMJ 2001;323:295-296 ( 11 August )

Editorials

Measuring the efficiency of health systems

The World Health Report sets the agenda, but there's still a long way to go

Papers p 307

In June 2000 the World Health Organization provided a long awaited answer to the question beloved of politicians and journalists: "How does the health system in country X compare with that in country Y?" The results, published in the World Health Report 2000,¹ delighted some governments, such as that of France, which came first, but infuriated others, such as Brazil, at 125. The rankings are based on measures of achievement of five health system goals. The achievement of health is seen as a core objective of a health system, so goals are a high level of health and a fair distribution. A health system should also be responsive to popular expectations. This includes respect for individuals (autonomy and confidentiality) and client orientation (prompt service and quality of facilities). As with health, the resulting goals relate to the absolute level of responsiveness and its distribution. The fifth goal is fair financing, with expenditure reflecting ability to pay rather than risk of illness.

In this week's BMJ some of the authors of the WHO report describe the methods they used to assess one of these goals, the attainment of health (p 307).² They relate expenditure on health, adjusted for local prices, to attainment of health. After adjusting for the level of education in the population, itself an important determinant of health, they rank the health systems of the world according to their efficiency in turning expenditure into health.

Inevitably, a few problems exist in an undertaking of this magnitude. The first is how one defines the health system. As set out in the World Health Report this encompasses "all the activities whose primary purpose is to promote, restore, or maintain health." This is welcome as it emphasises the importance of intersectoral action in promoting health, but unfortunately it also provides a problem since a figure for "all the activities" is nowhere to be found in any national health accounts. Instead, the report argues, as the health care system accounts for most of what is incorporated in the broader health system, "little is lost in concentrating on a narrower definition that fits existing data."¹ Consequently, we must compare inputs to the health care system with outcomes of the wider health system.

A subsidiary question is whether health outcomes can even be attributed to the activities included within the broader definition of a health system. As the report notes, there is growing evidence of the health gains that can be achieved both from health care interventions and from policies in other sectors, such as vehicle safety. But there are many other determinants of health. For example, in industrialised countries the health of populations reflects long established dietary patterns that owe more to climate, and thus the nature of agricultural produce, than to any contemporary policy. Thus, it is unsurprising that many of the countries performing best are characterised by "Mediterranean" diets. The growing evidence of how events throughout life influence health creates a further difficulty.³ Health system inputs that affect infant and child health may have consequences many years later.

A second problem is the availability of data.⁴ Many governments have only the vaguest idea of how many people live in their territory. Some have not undertaken censuses for many years,⁵ in some cases because large areas are outside their effective control. In many parts of the world population registration systems are fragmentary, and even in some industrialised countries significant gaps exist in coverage of some groupsfor example, native Americans.⁶ Equally, there are substantial problems with comparability of data on the other measures used, health expenditure and education. The authors recognise this problem and have constructed an elaborate set of procedures to address it, so generating figures for disability adjusted life expectancy⁷itself a highly controversial measure.⁸ Fundamentally, however, one cannot create data where none exist, so each step requires a series of often heroic assumptions and extrapolations.⁹ Unfortunately, though the World Health Report and its associated working papers note that many figures are estimates, it is not easy to discover just how extensive this process has been. Using complex models to generate estimates of uncertainty fails to tackle the underlying problem.

Other criticisms of this exercise have been aired elsewhere and include concern about the ideological values underpinning it and the intrinsic limitations of performance ranking.¹⁰ But some of these difficulties are insuperable, and a fairer question to ask is whether the report has achieved anything.

Despite its many limitations, arguably it has. Firstly, the WHO has stated clearly that governments have a responsibility for their health systems. It has invoked the concept of stewardship,¹¹ which implies a much more active involvement in promoting health than most governments have previously assumed.¹² Secondly, it has provided a useful conceptual framework that begins to tease out the goals of health systems. Thirdly, it has emphasised the need for a much better understanding of the undoubted impact that health systems have on health.¹³ It has not, however, provided a valid answer the question of whether one system is better than another.

Martin McKee, professor of European public health. 

London School of Hygiene and Tropical Medicine, London WC1E 7HT

Footnotes

   Competing interests: MM directs a WHO Collaborating Centre and was a member of the regional reference group for the 2000 World Health Report.

1.	World Health Organization. World Health Report 2000. Health systems: improving performance. Geneva: WHO, 2000.
2.	Evans DB, Tandon A, Murray CJL, Lauer JA. Comparative efficiency of national health systems: cross national econometric analysis. BMJ 2001; 323: 307-310[Abstract/Full Text].
3.	Kuh D, Ben-Shlomo Y. A life course approach to chronic disease epidemiology. Oxford: Oxford University Press, 1997.
4.	Almeida C, Braveman P, Gold MR, Schwarcwald CL, Riberio JM, Miglionico A, et al. Methodological concerns and recommendations on policy consequences of the World Health Report 2000. Lancet 2001; 357: 1692-1697[Medline].
5.	Sibai AM, Fletcher A, Hills M, Campbell O. Non-communicable disease mortality rates using the verbal autopsy in a cohort of middle aged and older populations in Beirut during wartime, 1983-93. J Epidemiol Community Health 2001; 55: 271-276[Abstract/Full Text].
6.	Heck KE, Schoendorf KC, Parker J. Are very low birthweight births among American Indians and Alaska Natives underregistered? Int J Epidemiol 1999; 28: 1096-1101[Abstract].
7.	Murray CJL, Lopez AD. Mortality by cause for eight regions of the world: global burden of disease study. Lancet 1997; 349: 1269[Medline].
8.	Anand S, Hanson K. Disability-adjusted life years: a critical review. J Health Econ 1997; 16: 685-702[Medline].
9.	Cooper RS, Osotimehin B, Kaufman JS, Forrester T. Disease burden in sub-Saharan Africa: what should we conclude in the absence of data. Lancet 1998; 351: 208-210[Medline].
10.	Mulligan J, Appleby J, Harrison A. Measuring the performance of health systems. BMJ 2000; 321: 191-192[Full Text].
11.	Saltman RB, Ferroussier-Davis O. The concept of stewardship in health policy. Bull WHO 2000; 78: 732-739[Medline].
12.	McKee M, Janson S. The challenge of stewardship. Eur J Publ Health (in press).
13.	McKee M. For debatedoes health care save lives? Croatian Med J 1999; 40: 123-128[Medline].

© BMJ 2001
ALL INFORMATION, DATA, AND MATERIAL CONTAINED, PRESENTED, OR PROVIDED HERE IS FOR GENERAL INFORMATION PURPOSES ONLY AND IS NOT TO BE CONSTRUED AS REFLECTING THE KNOWLEDGE OR OPINIONS OF THE PUBLISHER, AND IS NOT TO BE CONSTRUED OR INTENDED AS PROVIDING MEDICAL OR LEGAL ADVICE.  THE DECISION WHETHER OR NOT TO VACCINATE IS AN IMPORTANT AND COMPLEX ISSUE AND SHOULD BE MADE BY YOU, AND YOU ALONE, IN CONSULTATION WITH YOUR HEALTH CARE PROVIDER.