http://bmj.com/cgi/content/full/323/7308/307
BMJ 2001;323:307-310 ( 11 August )
Papers
Comparative efficiency of national health systems: cross national
econometric analysis
David B Evans
a Global Programme on Evidence for Health Policy, World Health
Organization, 1211 Geneva 27, Switzerland, b Evidence
and Information for Policy, World Health Organization
Correspondence to: D B Evans evansd@who.int
|
|
Abstract |
|
|
Objective: To improve the evidence base for health policy by
devising a method to measure and monitor the performance of health
systems.
Design: Estimation of the relation between levels of
population health and the inputs used to produce health.
Setting: 191 countries.
Main outcome measure: Health system efficiency (performance).
Results: Estimated efficiency varied from nearly fully efficient
to nearly fully inefficient. Countries with a history of civil
conflict or high prevalence of HIV and AIDS were less efficient.
Performance increased with health expenditure per capita.
Conclusions: Increasing the resources for health systems is
critical to improving health in poor countries, but important gains
can be made in most countries by using existing resources more efficiently.
|
What is already known on this topic Studies have not used a consistent framework for
specifying goals or measuring outcomes What this study adds Efficiency is related to expenditure on health per capita,
especially at low expenditure The methods of measuring performance provide a basis for
identifying policies that improve health and for monitoring reforms |
|
|
Introduction |
|
|
Policymakers have long been concerned with improving the performance of
health systems. 1
2 Reforms
have targeted financing (for example, social health insurance and
user charges), provision (for example, managed care, autonomous
hospitals), stewardship (for example, regulation of the private
sector, health legislation), and resource development (for example,
retraining of staff). 1
3-5 The
impact of these reforms is increasingly being studied, 6 7 but for
the results to be useful to policymakers across different settings,
studies need a consistent framework for assessing performance and a
measurable indicator.8
The World Health Report 2000 defined three intrinsic goals of
health systems
improving
health, increasing responsiveness to the legitimate demands of the
population, and ensuring that financial burdens are distributed
fairly.9
For health and responsiveness, systems should improve levels and
reduce inequalities. The report published first attempts to measure
the attainment of these goals by 191 countries and considered
how well countries were performing given their available resources.9 This paper
describes the methods used for measuring and monitoring performance
of health systems. Since improving health is the defining goal of
the health system, we report performance in terms of that goal. Data
sources have been given elsewhere.10
|
|
Methods |
|
|
Theory
Efficiency is defined as the ratio of the observed level of
attainment of a goal to the maximum that could have been achieved
with the observed resources. Normally, outputs are zero when inputs
are zero. In health, however, health levels would not be zero if
there were no health expendituresthat
is, no health systems. So to measure the contribution of the health
system we have to determine what it achieves in excess of what would
be achieved in its absence (the minimum). Accordingly, we define
performance as the current level of population health, in excess of
the estimated minimum, compared with the maximum achievable level of
health given the inputs. Because of the similarity between
performance and efficiency, we use the terms interchangeably.
Neither the maximum (frontier) nor the minimum levels of
health are observable, so they have to be estimated. Two strategies could
be used for estimating the maximum. One involves defining feasible
interventions, identifying their costs and outcomes, and choosing
those that maximise health for the available resources. This
approach has not been widely used 11 12 because
of data limitations but is currently being pursued by the World
Health Organization.13
The second approach, which we have used here, estimates the
maximum from a sample of observed inputs and outcomes. This approach requires
the relation between outcomes (population health) and inputs to be
specified. We estimated this relation with a form of regression
analysis that shows how health levels vary with inputs. The country
with the highest health level, after controlling for inputs, is the
most efficient. The maximum is the level of health the most
efficient country would have produced at each observed combination
of inputs. Efficiency of other countries is measured with respect to
the maximum. Inefficiencies might be from wastage or because the
most cost effective set of programmes or interventions are not used.
Further details of the method are given on the BMJ 's website.
Data
We estimated the efficiency of 191 countries from data for
1993-7. Population health was measured as healthy life
expectancy (box). The health system input was health expenditure per
capita measured in 1997 US dollars (adjusted for the cost of a
generic basket of goods in different settings).
|
Healthy life
expectancy Healthy life expectancy builds on the concept of life expectancy. Life
expectancy is adjusted to allow for the fact that people live part of their
lives in less than full health. These states are given weights between
0 and 1 to reflect their severity compared with full health (valued
at 1). In rich countries, between 7 and 10 years are typically
spent living in less than full health. Partly because of a longer life span,
women spend more time in poor health than men do. In poor countries, people
may spend over 20 years of their expected life span in poor health.
Taking into account these weights, ill health and its consequences reduce
healthy life expectancy by between 5 and 11 years across
191 countries. |
Levels of health are not solely affected by
health systems.9
The most widely accepted other determinant is education, which is
strongly associated with the health of both children and adults in
developed and developing countries. 14 15 Educated
people translate information and health services into health more
effectively than uneducated people do. We used a summary indicator
of educational attainmentaverage
years of schooling in the adult population.10
We did not include income per capita because income is
highly correlated with both health expenditure and education and complicates
statistical estimation. Moreover, income does not directly contribute
to health but acts through factors such as education, housing, and
food intake. Inclusion of the part of income acting through mechanisms
other than health expenditure and education made little difference
to our resultsthe
rank order correlation of efficiency scores was >0.99.
We estimated the minimum achievable health in the absence of
a health system from observations on 25 countries before the existence
of a modern health system (average year, 1908). Health levels were
correlated mainly with literacy. We estimated the minimum health
level for 1997 on the basis of current literacy rates as though
the 1908 relation still applied.16
We generated an uncertainty interval as well as a point
estimate of healthy life expectancy for each country.9 For all
countries, we randomly drew observations from the uncertainty distributions
and estimated efficiency and rank, repeating the procedure
1000 times with slightly different results. The reported efficiency
estimate is the mean score for that country, and the uncertainty
interval represents the range in which estimates fell, omitting the
bottom and top 10%. Rank was based on mean efficiency, and rank
uncertainty intervals were generated in a similar manner.
|
|
Results |
|
|
Table 1 gives the
coefficient estimates used in the regression equation to determine efficiency.
We investigated numerous specifications of the regression equation,
but they gave stable estimates of efficiency and rank.
|
|
Table 2 shows the
efficiency and ranks for the highest and lowest 10 performers and the
United Kingdom. Estimated efficiency varies from 0.08 to nearly
1, implying that although some countries may be close to their
potential, others are not reaching anywhere near maximum levels of
health. Figure 1
depicts the efficiency for all countries; the full results are
available on the BMJ 's website.9
Figure 2 shows
that efficiency is positively related to health expenditure per capita,
especially at low expenditure. Performance sharply increases with
expenditure up to about $80 (£53) per capita a year.
|
|
Discussion |
|
|
Perceptions about the relative performance of health systems in different
countries have been based on anecdote or case studies. For example,
Sri Lanka and China are believed to have been efficient in producing
health, 17
18 but
our results show that both perform less well than other countries at
similar levels of development. On the other hand, Oman performs
extremely wellperhaps
because it has reduced child mortality from 310 to 18 per
1000 live births over the past 40 years.19
|
|
Our efficiency scores compare current population health
levels with the maximum possible for observed levels of health expenditure
and education in a country. This does not mean that 100% efficiency can
be reached immediately. There will be time lags between some actions
and their outcomes, and efficiency in many low performing countries
is hampered by civil unrest or a high prevalence of HIV and AIDS
(fig 1). Healthy
life expectancy is reduced by up to 15 years in African
countries with the highest prevalence of HIV, clearly restricting
the ability of these systems to reach full efficiency in the short
term.
Validity of findings
Although other non-health variables affect health (housing quality,
environmental conditions, etc), relevant indicators are difficult to
find or estimate for many countries. In addition, many are highly
correlated with educational attainment, which we used because it
functions as a broad measure of non-health inputs.
Omission of non-health variables reduces the estimates of
efficiency. On the other hand, the measurement strategy biases estimates upward.
The fact that five countries have efficiency scores >0.97 does
not mean they can improve performance by nearly 3%. It means they
could improve by 3% compared with the most efficient country, but we
have no way of estimating the potential of the highest performer to
become more efficient. Microlevel studies suggest the potential is
there nevertheless.
Reasons for inefficiency
We found that efficiency is positively related to health expenditure
per capita. Performance increased greatly with expenditure up to
about $80 per capita a year, suggesting it is difficult for systems
to be efficient at low expenditure. There seems to be a minimum
level of health expenditure below which the system simply cannot
work well. We estimate it would cost just over $6bn a year (<0.3%
of global annual health expenditure) to increase health spending to
this threshold in the 41 countries with lowest expenditures.
Despite the need to increase funds in poor countries, there
is enough variation in efficiency at all levels of expenditure to
suggest that using current resources better could improve health considerably.
Reducing wastage is one way, but the studies of Tengs and Murray et
al show that allocation of resources is also important. 12 20 They
argued that health in the United States and sub-Saharan Africa could
be greatly improved by reallocating available resources from
interventions that are not cost effective to those that are more
cost effective but not fully implemented.
Another possible reason for inefficiency is that goals other
than health may be deemed important. The World Health Report 2000 recognised
that countries may also wish to reduce inequalities or increase the
responsiveness of the system.9 The
efficiency of health systems in achieving all defined goals has been
explored elsewhere.21 The
analysis produced some changes in rankfor
example, France was estimated to have had the most efficient system overallbut,
in general, countries efficient in producing health are also
efficient in producing other goals.21
Future research
Our conclusions are, of course, tentative. The quality of data
across countries varies greatly, and only some of this is accounted
for in our uncertainty analysis. Our main objective was to show that
the attainment and efficiency of health systems can be measured and
compared across countries and over time. Much can be done to improve
the data and methods, and WHO is currently working on this with
member countries and academic experts. We believe this is critical
work for health policymakers considering reforms. Without the
ability to measure the inputs and outputs of health systems, they
cannot know if the reforms achieve their objectives.
|
|
Acknowledgments |
The views expressed are solely those of the
authors and do not necessarily represent those of WHO.
|
|
Footnotes |
Contributors: DBE conceived the idea of applying the frontier production
function approach to measuring the performance of health systems,
supervised the performance research team, and wrote and revised the
manuscript. AT, CJLM, JAL, and DBE developed, performed, and
interpreted the econometric analysis. AT and CJLM developed the
methods for uncertainty analysis, and AT and DBE put together the
data required for the educational attainment variable. CJLM
coordinated the World Health Report 2000 research teams,
conceptualised the framework for analysing and measuring attainment
and performance, and contributed to the development of the health
system assessment framework. JAL estimated historical levels of
health system attainment. All four authors revised the manuscript
and approved the final version. Raymond Hutubessy, Yukiko Asada, and
JAL researched historical income and education levels. Alan Lopez,
Colin Mathers, Ritu Sadana, Josh Salomon, Omar Ahmad, and Doris
Mafat estimated life expectancy and healthy life expectancy.
Jean-Pierre Pouillier, Patricia Hernandez, and Chandika Indikadahena
estimated health expenditure. Julio Frenk had a major input to the
health system assessment framework. DBE is guarantor.
Funding: None.
Competing interests: None declared.
Further details of the methods and
full results are available on the BMJ's website
|
|
References |
|
|
|
1. |
Maynard A, Bloor K. Health care reform: informing
difficult choices. Int J Health Plann Manage 1995; 10: 247-264 |
|
2. |
Collins C, Green A, Hunter D. Health sector reform and the
interpretation of policy context. Health Policy 1999; 47: 69-83 |
|
3. |
Hussein AK, Mujinja PG. Impact of user charges on
government health facilities in Tanzania. East Afr Med J 1997; 74:
751-757 |
|
4. |
Feldman R. The ability of managed care to control health
care costs: how much is enough? J Health Care Finance 2000; 26: 15-25 |
|
5. |
Moore M. Public sector reform; downsizing,
restructuring, improving performance. Geneva: World Health Organization,
1996 (Forum on health sector reform discussion paper 7). |
|
6. |
Durham G, Kill B. Public health funding mechanisms in New
Zealand. Aust Health Rev 1999; 22: 100-112 |
|
7. |
Ron A. NGOs in community health insurance schemes: examples
from Guatemala and the Philippines. Soc Sci Med 1999; 48: 939-950 |
|
8. |
DeRosario JM. Healthcare system performance indicators: a
new beginning for a reformed Canadian healthcare system. J Health Qual
1999; 21: 37-41 |
|
9. |
World Health Organization. World Health Report 2000.
Geneva: WHO, 2000. |
|
10. |
Evans DB, Bendib L, Tandon A, Lauer JA, Ebener S,
Hutubessy R, et al. Estimates of income per capita, literacy, educational
attainment, absolute poverty, and income Gini coefficients for the World
Health Report 2000. Geneva: World Health Organization, 2000. (Global
programme on evidence for health policy discussion paper No 7.) |
|
11. |
Tengs TO, Adams ME, Pliskin JS, Safran DG, Siegel JE,
Weinstein MC, et al. Five-hundred life-saving interventions and their
cost-effectiveness. Risk Anal 1995; 15: 369-390 |
|
12. |
Murray CJL, Kreuser J, Whang W. Cost-effectiveness
analysis and policy choices: investing in health systems. Bull World
Health Org 1994; 74: 663-674 |
|
13. |
Murray CJL, Evans DB, Acharya A, Baltussen RM. Development
of WHO guidelines on generalized cost-effectiveness analysis. Health Econ
2000; 9: 235-251 |
|
14. |
Caldwell JC. Education as a factor in mortality decline:
an examination of Nigerian data. Pop Stud 1979; 33: 395-413 |
|
15. |
Caldwell JC, Caldwell P. Education and literacy as factors
in health. In: Halstead SB, Walsh JL, Warren KS, eds. Good health at low
cost. New York: Rockefeller Foundation, 1985:181-185. |
|
16. |
Evans D, Tandon A, Murray CJL, Lauer J. The comparative
efficiency of national health systems in producing health: an analysis of
191 countries. Geneva: World Health Organization, 2000. (Global
programme on evidence for health policy discussion paper No 29.) |
|
17. |
Halstead SB, Walsh JA, Warren KS, eds. Good health at
low cost. New York: Rockefeller Foundation, 1985. |
|
18. |
Hsiao W. What should macroeconomists know about health
care policy? A primer. Washington, DC: International Monetary Fund, 2000.
(IMF working paper.) |
|
19. |
Ahmad OB, Lopez AD, Inoue M. The decline in child
mortality: a reappraisal. Bull World Health Org 2000; 78: 1175-1179 |
|
20. |
Tengs TO. Dying too soon: how cost-effectiveness
analysis can save lives. Dallas, TX: National Center for Policy Analysis,
1997. (NCPA policy report No 204.) |
|
21. |
Tandon A, Murray CJL, Lauer J, Evans D. Measuring
overall health system performance for 191 countries. Geneva: World
Health Organization, 2000 (Global programme on evidence for health policy
discussion paper No 30.) |
(Accepted 17 April 2001)
© BMJ 2001
ALL
INFORMATION, DATA, AND MATERIAL CONTAINED, PRESENTED, OR PROVIDED HERE IS FOR
GENERAL INFORMATION PURPOSES ONLY AND IS NOT TO BE CONSTRUED AS REFLECTING THE
KNOWLEDGE OR OPINIONS OF THE PUBLISHER, AND IS NOT TO BE CONSTRUED OR INTENDED
AS PROVIDING MEDICAL OR LEGAL ADVICE. THE DECISION WHETHER OR NOT TO
VACCINATE IS AN IMPORTANT AND COMPLEX ISSUE AND SHOULD BE MADE BY YOU, AND YOU
ALONE, IN CONSULTATION WITH YOUR HEALTH CARE PROVIDER.