13 June 2002 14:32 GMT

by Bea Perks, BioMedNet News

Hands up, all biologists who really understand statistics. There aren't many, judging from a quick survey of biologists and statisticians. What to do about a problem that may be leading to erroneous conclusions in countless studies? Fingers point variously at software providers, inadequate teaching, and, heaven forfend, biologists themselves.

"I am extremely aware of my lack of statistical knowledge," admitted Jacqui Matthews, lecturer in equine medicine at the University of Liverpool's Faculty of Veterinary Science, who has recently come in for some public criticism of her analysis. She says she gets around this deficiency by seeking advice from her "statistical colleagues."

Matthews finds herself on the defensive about the statistical methods she and her coauthors used in a paper published last year in the International Journal for Parasitology.

In that study, Matthews' team treated four groups of worm-infested calves in one of four different ways. They compared data on worm burden following each treatment, using a series of non-parametric Mann-Whitney U tests.

But they may have reached the wrong conclusion. The Mann-Whitney test is only suitable for comparison of two groups, where there is one degree of freedom, says David Morrison, senior lecturer in the Department of Environmental Sciences at the University of Technology, Sydney.

Degrees of freedom represent the number of independent pieces of information in a statistical analysis, notes Morrison; there is always one less degree of freedom than entities being analyzed. For the four independent data groups in Matthew's study, there are three degrees of freedom, he notes, making Mann-Whitney U tests inappropriate.

"The appropriate statistical test for analyzing more than two groups is analysis of variance, if you want a parametric test," said Morrison, "or the Kruskal-Wallis test, if you want a non-parametric test."

Matthews and coauthor Nigel French, also at Liverpool, now concede that their analysis was "outdated." But Matthews adds that she has used the same test in several similar papers, and that independent referees approved all of them. "Perhaps the so-called problem is universal!" she said.

Morrison raises the general issue in a letter in press in the International Journal for Parasitology.

"It would be nice to explain it," he told BioMedNet News, "but I suspect that there are many interacting reasons."

He places much of the blame on introductory stats textbooks and courses, which often stop at t-tests. People use what they know, he says with a sigh, while "almost all worthwhile biological experiments are more complex than a t-test can handle."

Morrison is also dismayed that many commercially available stats programs fail to provide a comprehensive range of tests. In particular, few programs offer non-parametric multiple comparison tests. (For those a little shaky on the terminology, classical parametric methods such as the t-test assume a normal distribution of data; many non-parametric tests are based on ranks of data rather than actual values, which is useful when the data do not satisfy certain assumptions.)

Ted Gaten, senior experimental officer at the University of Leicester biology department, lays the blame for the problem largely at the door of software manufacturers - particularly Microsoft, whose ubiquitous spreadsheet Excel offers a limited number of parametric statistical tests and analytical tools.

"Those to whom statistics remains a foreign language will all too often resort to Excel, in spite of its undeniable inadequacies," said Gaten, who devises computer-based learning projects for biology students. "I think Excel should carry a warning that it should not be used for statistical testing without consideration of its deficiencies and weaknesses."

Judging from Microsoft's considered response, this is unlikely to happen. A Microsoft spokesperson replied to Gaten's criticism with the following: "For many individuals, teams, and organizations, Microsoft Excel 2002 provides the technologies to manage critical business data, while giving everyday users the tools they need to get the most out of their information."

Perhaps a warning wouldn't work, anyway, if an old adage about biologists and numbers is valid. "I always tell my second-year students ... that they chose biology because they hate mathematics," Morrison said. "To an Australian audience, my analogy is that they chose biology because they want to cuddle koalas, not because they love mathematics."

This is a serious problem, because scientists can do little to avoid generating and analyzing numbers. But a British statistician and statistical-software writer argues against doing too much analysis.

"A lot of the misuse comes from the overuse of significance tests," said Roger Stern, principal biometrician at the University of Reading's Statistical Services Centre. Coauthor of the online statistical package INSTAT, Stern feels overuse of non-parametic methods is a particlar problem, and says he has no intention of adding them to his software.

"[Significance tests] are often taught extensively, but often play only a small part in the objectives of a practical study," he told BioMedNet News. He puts a lot of the problem down to poor teaching.

The real root cause of "messy-looking" data may be the researcher's own inadequate understanding of the system that generated it, he says: Inadequate care in figuring out how to make measurements in the first place can lead to unreliable data.

Think harder about the structure of the data, Stern suggests, rather than suppressing the complications and using an analysis that ignores them. Reasonable estimates can usually generate more meaningful and useful results than most non-parametric methods which often assume the measurements themselves are flawed or at least weak, he adds.

It is telling that statistical analysis finds its harshest critics among statisticians themselves. Stern's warning is echoed by Rob Kass, professor and head of the Department of Statistics at the Carnegie Mellon University in Pittsburgh.

"Computers have only exacerbated a problem that has been recognized for a long time," Kass told BioMedNet News. "As the famous quote of Benjamin Disraeli indicates: there are lies, damned lies, and statistics!"

Send us your comments for publication.

Sign up for BioMedNet News weekly email alerts.