World still waiting for Ontario flu data

By Terry Murray

TORONTO – The effectiveness of Ontario's universal flu immunization program, which is now entering its third year, may never be known.
   That's because the Ontario program is not being adequately evaluated, according to information obtained by the Medical Post.
   And the first public presentation of the evaluations that exist will be made in late October—not in Ontario, not even in Canada, but at the First European Influenza Conference in Malta, where the province's chief medical officer, Dr. Colin D'Cunha, is slated to attend.
   Assessment of the multi- million-dollar program's success has been sought by health departments in Canada and around the world, including the U.S. Centres for Disease Control and Prevention (CDC) and the World Health Organization, because the Ontario project is the world's only universal flu immunization program.
   The Malta conference appears to have been regarded by the ministry as an opportunity to spotlight the program because, until recently, Health Minister Tony Clement was scheduled to attend it along with Dr. D'Cunha, chief medical officer.
   Vaccine coverage data were collected in the 2000/01 flu season, the first year of the program, as part of the agreement between the Ontario Ministry of Health and Long-Term Care and the manufacturer supplying the vaccine to the province.
   Despite six requests for information by the Medical Post over the last year, the ministry has failed to provide the data or interpretation of them.
   However, the Medical Post uncovered the figures buried in the Web site of the Massachusetts Peer Review Organization Inc. The site contains slides prepared for presentation earlier this year by a physician at the CDC.
   The CDC slides, indicating the Ontario ministry as the source of the numbers, showed that flu shots reached 44.4% of the total population during the 2000/01 season, and 34.9% of those with no risk factors (see chart).
   The highest coverage rates were found in people over age 65 (78.9%) and those under 65 with a chronic medical condition (52.4%), both groups that are considered to be at high risk of flu complications.
   Those are self-reported figures, based on a survey of more than 22,000 Ontario residents.
   But there was no vaccine coverage survey done last year, and it remains to be seen whether one will be included with the 2002/03 program, which is due to be launched Oct. 17.
   The lack of data have been particularly vexing for the research scientist who has been asked by the ministry to assess the program's impact on Ontario's health-care system.
   "I've been quite frustrated because we have a program that's over $30 million and we don't have enough information to evaluate it at all," said Dr. Doug Manuel, research scientist at the Institute for Clinical Evaluative Sciences in Toronto.
   "Clearly this needs to be done, and clearly we have insufficient data to do it," he said in an interview. A coverage survey would cost about $500,000, he estimated, but the data would be better if it came from an immunization registry, which has been under discussion by all levels of government for several years.
   Data from the Ontario Health Insurance Plan (OHIP) could yield the needed information, but the relevant billing codes are "quite poor" and need to be validated, Dr. Manuel said.
   OHIP data would also give information on only those immunizations given in physicians' offices, which according to the first year information provided to the CDC accounted for 64% of flu shots that were given.
   "It seems to me that these requirements are quite clear. We need to know who is and is not vaccinated. It seems to me that the ministry would play an important role in getting that information."
   Despite a budget of $38 million the first year and $44 million last season to put the vaccine in Ontarians' arms, Dr. Manuel has been told that funding for evaluation is hard to sell to the provincial cabinet.
   "This is a new program, one of the first programs in the world. The whole world is looking at us to evaluate it, and we really don't have sufficient data to do that."
   Dr. Manuel does have other ideas about how to get the analysis done, but all of the study designs are weaker than he'd like, or will not provide all the needed information.
   "The research hasn't been generated yet by our OHIP offices because they're dealing with a lot of other things right now and that's lower on their priority list than other things," according to Anne Matthews, media relations manager for the ministry.
   "(Dr.) Colin (D'Cunha) all along has said that at the end of year three, we'll be able to have numbers," Matthews said. "That was the timeline OHIP was given. The numbers will start coming in after this year."
   She didn't respond to a Medical Post request for the "success indicators" of the Ontario program, which is part of the title of a poster presentation to be made in Malta by Dr. D'Cunha.