Weighing the homeopathic evidence

E Ernst

Peninsula Medical School, Complementary Medicine, Universities of Plymouth & Exeter, Exeter, UK

Available online 6 May 2003.

This issue of `Homeopathy' contains an article[1] which is in several ways remarkable and deserves a comment. Essentially it is an attempt to take a fresh look at the homeopathic trial evidence.

A remarkable feature is the methodological approach of the analysis. Robert Mathie evaluated the `weight' of the evidence, a term not usually used in this context. Having said this, we have used it extensively in our recent assessment of the entire field of complementary medicine.[2] We employed the term as a composite measure of three independent variables associated with any trial evidence (i) methodological quality, (ii) volume (ie total sample size), (iii) level (ie uncontrolled, controlled or randomised trial, systematic review or meta-analysis). Mathie's used the term `weight' to describe a simple `vote count', ie negative vs positive trials, and he attributed a positive weight if the latter outnumbered the former by at least two.

The most remarkable feature of the article is its overall result. It concludes that `the present weight of evidence favours homoeopathic treatment efficacy in 8 conditions'. This overtly contradicts most of the 17 systematic reviews that have evaluated (some of) this evidence since 1997³ and contradicts my review of these reviews, which concluded that `the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice'.[3]

How can this be? Surely systematic reviews are supposed to minimise bias! Should they therefore not reach identical conclusions? Linde and Willich have recently suggested that this is not necessarily the case and that `systematic reviews often differ considerably'.[4] They demonstrate that different methodologies are the cause of such discrepancies.

We therefore need to ask how good Mathie's methodology was and how sound his conclusions are. The eight conditions for which he finds positive evidence are listed in Table 1. In five cases (influenza, childhood diarrhoea, fibrositis, sprains and radio/chemotherapy), there are only two or three trials available; in some cases these studies are small. Thus, the volume of the evidence is clearly not substantial. In other instances there is a clear need for independent replication of the trials, as the data originate exclusively from only one or two research groups. In yet other cases, the methodological quality of some of the studies is such that it seriously limits the conclusiveness of the result.

Table 1. Critique of positive trials
(13K)

But there are other problems as well. For instance, Mathie included studies that were `randomised and/or double-blind'. In other words, non-randomised trials were included, which are known to be wide open to selection bias. Mathie also categorised a trial result as positive if `at least one outcome measure was statistically significantly improved'. Consider a trial with 20 outcome measures and a statistical cutoff at P = 0.05. This is by no means a rare scenario––a study with five endpoints measured at four different timepoints would already fulfil this criterion. In such a case, one outcome measure would normally be significant purely by chance rather than through anything remotely associated with therapeutic efficacy. The more rigorous criterion for a `positive' trial is thus that the endpoint, which had been predefined by the original authors as the primary outcome measure, yielded significant results.

Thus, I fear, Mathie's methodology was not as strong as it could have been, and this deficit has led to conclusions that may be falsely positive. People who are less enthusiastic about science and more emotionally dedicated to homoeopathy than I am will emphatically dispute this point––at least I hope they will. It is this dispute that will drive us to conduct more and better trials; and it is only through such tests that, one day perhaps, we will be able to document the truth about homoeopathy's value beyond placebo.

Your use of this service is governed by Terms and Conditions. Please review our Privacy Policy for details on how we protect information that you supply.

	This Document
		SummaryPlus
		Full Text + Links
		PDF (61 K)

	Actions
		E-mail Article

Article Outline

References