http://www.citeline.com/C1SE/view?target_url=http%3A%2F%2Fwww.cdc.gov%2Fnip%2Fvacsafe%2Fresearch%2Fconfound.htm&return_url=%2FC1SE%2Fresults%3Fsummaries%3Don%26enhance%3Don%26kw%3Dvaccination%2Bnot%2BCDC&title=Confounding%20in%20Studies%20of%20Adverse%20Reactions%20to%20Vaccines&new_s=search%3F

Confounding in Studies of
Adverse Reactions to Vaccines 

As published in American Journal of Epidemiology, July 15, 1992; 136(2): 121-135

Authors:

Paul E M Fine, VMD, PhD
Robert T Chen, MD, MA

Centers for Disease Control and Prevention
National Immunization Program
1600 Clifton Road, MS E-61
Atlanta, Georgia 30333

ABSTRACT

Several social and medical attributes are associated both with avoidance or delay of vaccination and an increased risk of adverse events such as sudden infant death syndrome (SIDS) or childhood encephalopathy. Studies that fail to control adequately for such confounding factors are likely to underestimate the risks of adverse events attributable to vaccination. This paper reviews the literature on studies of severe adverse events after the administration of pertussis antigen-containing vaccines, with particular attention to the measures taken by different investigators to avoid this problem. Most published studies have reported a deficit of SIDS among vaccinees, which may reflect confounding in their study designs. An expression is derived to explore the extent of underestimation that may be introduced in such studies, under different sets of conditions. Confounding of this sort is a general problem for studies of adverse reactions to prophylactic interventions, as they may be withheld from some individuals precisely because they are already at high risk of the adverse event.

INTRODUCTION

Immunization programs are undeniably among the most effective public health interventions. Reductions over recent decades in the morbidity and mortality attributable to smallpox, measles, polio, diphtheria, whooping cough, and tetanus are eloquent reminders of this fact (1). However, the very success of these programs brings new problems. No intervention is entirely without risk, and even very rare adverse reactions to a vaccination increase in importance as the target disease itself disappears.

Changes in the perception of risks attributable to vaccination, compared with those attributable to natural disease, are of immense importance to vaccination programs. Recognition of these changes within the scientific community led to termination of smallpox vaccination in many countries prior the global elimination of disease. Such recognition also is now the basis for  reconsideration of polio vaccination strategies (2,3). The public perception of such changes led to dramatic declines in the uptake of pertussis vaccination during the 1970's in the United Kingdom and Japan (4,5). Similar concerns in the United States have led to a large number of lawsuits, a substantial rise in vaccine prices (6), and new legislation governing reporting and compensation of adverse events (7). Given such issues, one sees an obvious need for continued monitoring of vaccine safety to assist policymakers in assessing needs for improvements in vaccine preparations or for changes in vaccination strategy.

The monitoring of vaccine safety may be based on either active or passive ascertainment of adverse events (8). To assess whether such events are in fact attributable to vaccination, the investigator may use two sorts of approach. The first involves cohort logic, i.e., the comparison of incidence rates of the event in question between cohorts of vaccinated and unvaccinated individuals. If there are very few unvaccinated individuals in the population, then the comparison may be between (age-specific) rates of events before and at successive intervals after vaccination. The alternative approach involves the application of case-control logic, i.e., comparisons of the frequency of a history of recent vaccination between individuals experiencing adverse events and appropriate controls.

Regardless of the approach used, such studies face several methodological difficulties (9-10). Many potential sources of bias have been identified. Prominent among these is the problem of ensuring that adverse events are ascertained independently of vaccination history. Failure to control for this factor may lead to creation, or overestimation, of an association between a vaccine and an adverse event. Another problem is that of confounding between the risk factor (vaccination) and outcome measure (adverse event) of interest. Many factors known to be associated with either avoidance or delay of vaccination may themselves be associated with an increased risk of adverse event-type medical outcomes. As an illustration, Table 1 presents reported risk factors for sudden infant death syndrome (SIDS) and for childhood encephalopathy, on the one hand, and for failure to receive diphtheria-tetanus-pertussis (DPT) vaccination on the other (11-22). The close correspondence between these sets of factors, which include medical contraindications and social correlates of low vaccine coverage, suggests that individuals predisposed either to SIDS or to encephalopathy are relatively unlikely to receive DPT vaccination. Studies that do not control adequately for this form of "confounding by indication" (23) will tend to underestimate any real risks associated with vaccination.

This paper examines the influence of such confounding on vaccine adverse event studies by reviewing the literature to illustrate its presence and by modeling to demonstrate its impact under different sets of conditions.

REVIEW OF THE LITERATURE

Published studies that are relevant to the problem of confounding between risk factors for DPT vaccination and for potential adverse events are summarized in Table 2. This review does not cover reports of cases or clusters of time-associated adverse events (24-25), as these are not likely to be representative and they provide no means to evaluate the confounding problem that is the focus of this paper.

Studies of DPT and SIDS

The first published controlled investigation of the relation between DPT and SIDS was a case-control study by Taylor and Emery in 1982 (26), who reported that 8 (31%) of 26 SIDS cases had ever received DPT or DT vaccine compared with 27 (52%) of 52 age- and area- matched controls (odds ratio= 0.41). Except for the matching of controls, no attempt was made to overcome confounding by factors predisposing to vaccination or to SIDS in this investigation.

The following year, Baraff et al. (27) reported data on the time interval between DPT vaccination and death of 27 SIDS cases who had received DPT vaccine within 28 days prior to death. A significant excess of deaths was noted within 24 hours (observed = 6; expected = 0.96; p < 0.005) and within 7 days (observed = 17; expected = 7.72; p < 0.05) of vaccination. Subsequent correspondence discussed the potential for selection, recall, and observer bias in this study and raised the possibility that the association might have been due in part to the similarity in age trends between SIDS incidence and DPT vaccination (28). The paper also included an analysis of intervals between visits to physicians and death for 40 SIDS cases reported to have sought medical care (but not received vaccination) within 28 days prior to death. There was an excess of visits within 7 days, which may have reflected prodromal symptoms associated with the subsequent deaths of these children. Given that some of these symptoms may have been interpreted as contraindications to vaccination, we again see evidence of the concordance of risk factors summarized in Table 1.

Results of the largest investigation of the relation between DPT and SIDS were reported by Hoffman et al. in 1987 (14). These were based upon a multicenter case-control study comparing risk factors in 757 SIDS cases with those in randomly selected living controls matched for birthplace and age (control group A) or for birthplace, age, race and birth weight (control group B). Overall, SIDS cases were less likely to have received DPT (or any vaccine) than were their matched controls (odds ratios= 0.54 [control group A] and 0.58 [control group B]). The significant negative association between prior DPT vaccination and SIDS was maintained in multiple logistic analysis controlling for 11 other factors: birthweight, sex, race, parity, maternal age, maternal education, smoking during pregnancy, alcohol consumption during pregnancy, use of prenatal care, prepregnancy weigh, and pregnancy weight. Case children were less likely than controls to have had postnatal outpatient visits, but more likely to have had sick visits; but no attempt was made, however, to control for these factors in the analyses. The negative association between vaccination and SIDS was strongest when analyses were restricted to vaccination within 24 hours of death (crude odds ratios= 0.19 [control group A] or 0.46 [control group B]). The authors concluded,  "DPT immunization does not appear to be a significant factor in the occurrence of SIDS." (14, p. 610).

A smaller case-control study based on linked data was reported by Walker et al. (29). These authors compared 29 SIDS cases with 262 age-matched controls drawn from linked vaccination and mortality records of 26,500 children registered between 1972 and 1983 with the Group Health Cooperative of Puget Sound. SIDS was defined as "...any death for which no cause could be discerned among infants of normal birthweight (> 2500 grams) and without predisposing medical conditions...." (29, p. 245). The criteria for "predisposing medical conditions" were not stipulated in detail, but led to the exclusion of two children with "life-threatening medical conditions" (29, p.950). Such exclusions represent an effort to control for confounding in the design of this study, and should have compensated to some degree for the concordance of risk factors illustrated in Table 1. These authors found a negative association between SIDS and a history of having ever received DPT (odds ratio = 0.15). On the other hand, when nonimmunized children are excluded from analysis, detailed breakdown by successive intervals between DPT vaccination and death suggested that the daily mortality risk in the period 0 to 3 days after vaccination (4 deaths observed) was 7.5 (95% CI: 1.7 - 31) times greater than that during the period more than 30 days after vaccination (9 deaths observed).

Griffin et al. (30) linked birth, death and immunization records in Tennessee in order to follow up 129,834 infants who were born over the years 1974 - 1984 and recorded as having received at least one dose of DPT vaccine. Sudden infant death was reported in 109 of these children between the ages of 29 days and 1 year of life. Cohort logic was used in order to calculate the relative risks of SIDS in successive intervals after receipt of DPT vaccine, compared to the risk of SIDS occurring more than 30 days after vaccination. A clear gradient in relative risk was observed, from a low of 0.2 during the first 72 hours after vaccination to unity for the period 2 weeks or more after vaccination. The trend remained when controlled for age, sex, race, year, birth weight, and Medicaid enrollment. The authors interpreted the finding as follows: "The most plausible explanation for the decreased rate of SIDS in the period immediately after immunization is that children may be immunized when they are in better health and that this healthier state is associated with a lower risk of SIDS." (30, p.621). The authors then attempted to evaluate the potential impact of such confounding on their investigation, noting that other studies had shown that "nearly half of all children who die of SIDS have either no symptoms or very minor ones before death. Therefore these studies suggest that selective immunization of asymptomatic cohort children could at most account for a 50% decrease in the rate of SIDS after immunization in this study, but that the decrease could not be of sufficient magnitude to mask a true increase in the incidence of SIDS after immunization." (30, p. 622).

This statement implies two things.  First, even if more than half of the children who died of SIDS had prior symptoms that might have rendered them ineligible for vaccination shortly before death, such selection "could" still only have reduced the observed relative risk by a maximum of 50%, at least under the conditions of their study.  Second, their finding of a relative risk of 0.18 (see Table 2) was therefore incompatible with a true relative risk greater than unity.  We will return to the logic of this argument below.

In summary, we see that all investigators have found that SIDS cases are less likely to have ever been vaccinated than are living age-matched controls. On the other hand, analyses of time intervals between DPT vaccination and SIDS have shown a deficit of deaths shortly after vaccination in some studies (14,26,30) and an excess of such deaths in others (27, 29) The two positive short-interval associations were based upon small numbers (27 and 29 total cases), and may have been due in part to the fact that the peak age distribution of SIDS coincides with the recommended onset of DPT vaccination. This was exacerbated by the use of time more than 30 days after vaccination as the reference period, as this extends into ages of low background risk.

All in all, the negative associations between DPT vaccination and SIDS are impressive. None of the investigators cited above has suggested that these findings might be due to DPT being protective against SIDS, and several have noted that the findings are probably attributable to the fact that risk factors for SIDS are similar to factors known to be associated with either avoidance or delay of vaccination (e.g. Table 1). The negative associations between SIDS and having ever been vaccinated reflect avoidance of vaccination. On the other hand, the negative associations between SIDS and having recently been vaccinated could reflect either avoidance or delay of vaccination by those predisposed, for one reason or another, to die of SIDS.

Studies of DPT and encephalopathy:

The British National Childhood Encephalopathy Study (NCES) represents the largest controlled study of encephalopathy and DPT vaccination thus far carried out. It also includes the most thoughtful discussion in the literature on the issue of confounding between factors predisposing both to avoidance of vaccination and the adverse reaction under study (16, 31).

The NCES was designed as a case-control study comparing detailed vaccination histories of more than 1000 encephalopathy cases with those of controls (2 per case) matched for sex, date and area of birth. Significant associations were revealed between encephalopathy and receipt of DPT vaccine less than 7 days before onset of illness or between encephalopathy and receipt of measles vaccine within 7-14 days prior to onset of illness, but no association was detected with prior DT vaccination.  Many aspects of this study, in particular, biases that may have been introduced by the method of case ascertainment, have been discussed extensively in the literature (e.g., 10).

The authors explored the potential for confounding in four ways. First, they restricted their most rigorous analyses to those cases who had no evidence of neurological abnormality prior to onset of the encephalopathy. This should have controlled for most neurological factors (except for febrile convulsions, which were not treated as prior neurological abnormalities) that may have served as contraindications for vaccination. Second, they carried out separate analyses excluding all cases and controls with previous history of fits (again in an effort to control for factors that might have influenced both the risk of encephalopathy and the propensity to be vaccinated). Third, they carried out a separate analysis, matching for social class (manual v nonmanual occupation of the head of the family). The significant association remained, leading the authors to comment,  "There is, therefore, no evidence that correcting for the effect of social class eliminates or diminishes the significant association demonstrated between serious neurological disorder and immunization against pertussis, or that social class is a significant confounding variable." (16, p. 132)  Finally, the authors considered "other possible confounding variables...such...as past family and personal medical history, and other environmental conditions. For these, or any other factor, to cause significant bias in the calculations of relative risk they would need to operate powerfully and consistently in one direction, to be specific for one vaccine (DPT) and not another (DT), and to concentrate their influence on the observed associations over relatively short time intervals before onset which differed between vaccines (DPT and measles). It seems highly improbable that all of these criteria would be satisfied by any of the confounding variables postulated in this Study." (16, p. 132).

The authors of the NCES study were concerned whether confounding factors might have been responsible for creating the observed significant association between DPT vaccination and encephalopathy. Given that all of the factors listed in Table 1 would be expected to reduce rather that to create such an association, the conclusion of the Study of a significant association between recent DPT vaccination and encephalopathy does not appear to be threatened by any failure to control for additional factors that relate both to the propensity for (avoidance of) vaccination and to the risk of encephalopathy. Indeed, as might have been predicted, controlling for previous neurological status, for prior history of fits, and social class led to increases in the estimated relative risks, the only exception being in a subanalysis of one social class group (manual), for which the estimated relative risks associated with DPT remained virtually unchanged.

Three other investigations of the relation between between DPT vaccination and encephalopathy or serious neurologic illness have now appeared. Both Walker et al (32) and Griffin et al (33) have extended their studies of DPT and SIDS to include encephalopathies. Neither found any evidence of an association with DPT vaccination, but the numbers of cases were small and none had recently received DPT, which may reflect avoidance of vaccination by children at risk. In addition, a preliminary report has appeared, describing a major case-control study of acute, serious, neurological diseases of children in Oregon and Washington states in the United States (34). Matched-set analysis of the first 100 severe cases revealed an odds ratio of 2.5 (95% confidence interval 0.7 - 9.3) with a history of DPT vaccination within the previous 7 days. Adjustment for several factors that might be related to vaccine avoidance (personal or family history of seizures, prior DPT reaction, and illness within 30 days) led to an increase in the odds ratio to 3.6 (95% confidence interval 0.8 - 15.2), although the relationship was still not statistically significant. Once again,

we see evidence of confounding and must ask whether the adjustment actually carried out has removed the effects entirely.

THEORETICAL ARGUMENT

The extent of bias introduced by confounding will be a function of several variables. In order to explore the quantitative implications of these variables, we begin with the following definitions, using DPT and SIDS as an example.

S = Risk of SIDS in unvaccinated children who lack the contraindication to vaccination. (It should be noted that we refer to "contraindication" here to exemplify any factor associated with avoidance or delay of vaccination.)

R = True relative risk of SIDS associated with vaccination.

D = Relative risk of SIDS associated with the contraindication.

C = Proportion of children with the contraindication.

V = Proportion vaccinated among children without the contraindication.

P = Proportion vaccinated among children with the contraindication.

Using these definitions, we can calculate the expected risk of SIDS in different segments of the child population, as shown in Table 4. It should be noted that these predictions assume that the risks of SIDS associated with vaccination and with the contraindication are independent and, thus, the risk of SIDS among children who are vaccinated despite having the contraindication is R*D times that in unvaccinated children who lack the contraindication. Given these expressions, we can estimate what would be the observed relative risk of SIDS associated with vaccination, if an investigation were to take no account of the potential confounding by contraindication (i.e., no appropriate matching or stratification). With cohort logic, the observed relative risk would be a(c+d)/c(a+b), using conventional definitions for the cells of Table 5. In a case-control study, the odds ratio (ad/bc) should give a close approximation of the relative risk, given that the adverse event is rare (i.e., a and c are small).

We explore the implications of these expressions under two general sets of circumstances. The first relates to probabilities of vaccination and of SIDS such as would accumulate over a year (analogous to studies that have used a history of having ever been vaccinated as the risk factor). In this long term case, the (annual) risk of SIDS may be on the order of S = 0.001 (35), and the overall proportion vaccinated at least once may be on the order of V = 0.7 to 0.9. The second uses parameter levels such as might arise in short-term studies which examine the risk of SIDS within one day or one week of vaccination. In this case, the risk of SIDS will be small, on the order of S = 3*10^-6 per day or 2*10^-5 per week, and the probability of vaccination also small, on the order of V = 0.01 per day or 0.7 per week.

Figures 1 and 2 present the ratios between the observed and the "true" relative risks of SIDS, associated with vaccination, under each of these circumstances, and given different sets of assumptions as to the values of the several parameters. Although risk factors such as those listed in Table 1 are unlikely to be associated with relative risks (D) greater than 10, D=30 is included for sensitivity analysis to examine the impact of extreme values.

An interesting feature of this relation between observed and true relative risks is its independence of R (the true relative risk) and of S (the background risk of SIDS in the population). The magnitude of the bias is a function of the degree to which the contraindications are observed (i.e., the ratio V/P), as this determines the proportions with contraindications and, hence, the risks of adverse events in the vaccinated and unvaccinated populations. The lower the proportion (P) vaccinated among those with "contraindications" (i.e., the greater the extent to which contraindications are observed by those responsible for vaccination), the greater will be the bias in a study that does not control for these factors. Under both the long- or short-term assumptions, we see that a substantial bias in estimating R can occur, given levels of D greater than 10 and prevalences of the contraindication (C) greater than 1 %.

Table 6 illustrates the implications of various combinations of variables for the observed relative risks of SIDS associated with the vaccination, as a function of the true relative risks and the observed proportion of SIDS cases who have the contraindication. We see that it is possible for the observed relative risk of SIDS associated with vaccination to be less than half the true relative risk, even if half the children with SIDS have contraindications.

DISCUSSION

Review of the literatures on SIDS, encephalopathies and DPT suggests that a large number of factors are associated both with a tendency to avoid or to delay vaccination and an increased risk of SIDS and other serious neurological events (Table 1). That failure to control for such factors may lead to spurious negative associations between vaccination and adverse events is evident in several published investigations (Table 2). Examination of the logic underlying this relation reveals that failure to control for such factors in analyses may mask true associations between vaccinations and certain adverse outcomes under certain conditions (Tables 3-6; Figures 1,2).  In particular, we note that the extent of relative risk underestimation will be related directly to the proportion of individuals with contraindications to vaccination that are also risk factors for the adverse outcome, the relative risk of the adverse outcome associated with these contraindications, and the extent to which these contraindications to vaccination are observed (i.e., the difference in vaccination coverage between individuals with and without the contraindications).

The magnitude of such confounding effects may be considerable. The five studies of DPT and SIDS summarized in Table 2 reported relative risk estimates ranging from 0.15 to 5.4 using various methods; however, most of the estimates were below 1.0, and four of the studies have reported at least some relative risk measures below 0.2 (Table 2). It seems unlikely to us, though, on biological grounds that the true relative risk in this situation could be less than unity (as this would imply that such vaccines provide some immediate non-specific protection against sudden infant death). Although the underestimation may have been due in part to biased case ascertainment, inappropriate control selection, or chance effects, its most obvious source is the confounding problem discussed in this paper. Major reductions are seen when the prevalence of contraindications exceeds 1%, and the effect approaches its maximum when their prevalence reaches 5% (Figures 1 and 2). It may not be unreasonable to suppose that 5% of infants in many populations will have at least one of the confounding risk factors cited in Table 1 (36,37).

In contrast to the conclusion of Griffin et al (30), our simulations demonstrate that it is at least possible for the observed relative risk to be less than half the true value even if more than half of the cases (e.g. of SIDS) have risk factors for avoidance of vaccination (Table 6).  On the other hand, our exploration of parameter values, such as might arise in "recent vaccination history" studies exemplified by Griffen et al., does not easily explain the very low relative risks of SIDS associated with DPT vaccination observed by some investigators (14, 30).  Sampling errors aside, observed relative risks on the order of 0.2 could arise even if the true relative risk were greater than 1.0, if one assumes that the contraindications were highly prevalent (high C) and associated with a very high relative risk of the adverse outcome (high D) (e.g., if V=0.01, C=0.2, D=50, and P=0.01, then a true relative risk of R=1.2 would be observed as 0.24).  Such a high prevalence of so strong a contraindication/risk factor, however, seems implausible.  Risk factors such as those listed in Table 1 are unlikely to be associated with relative risks (D) greater than 10, let alone 30 or 50.  Thus, whether the low observed relative risks of SIDS associated with vaccination reflect sampling error, interactions among several contraindications/risk factors, or other sorts of biases, or, indeed, whether they do reflect some "protective" effect of vaccination remains unclear to us and awaits elucidation.   We note, however, that reanalysis of the British National Childhood Encephalopathy Study of all cases and controls with any potential contraindications to vaccination has led to a fourfold increase, from 3.3 to 12.6 (95% confidence interval 2.8-114.7), in the estimated relative risk of encephalopathy subsequent to DPT vaccination (D. Miller, St. Mary's Hospital Medical School, London, personal communication, 1990).

In theory, it might be possible to estimate the extent of this bias in a particular situation, but this would require knowledge of the nature, frequency, and implication of each of the six factors that may influence both propensity to be vaccinated and the risk of adverse event (Table 3). The difficulty of obtaining such information on all six factors makes it extremely hard to assess whether an observed relative risk of, for example, 0.2 is consistent with a true relative risk greater than 1.0. This inference is made even more problematic by the fact that many other sorts of bias, for example, relating to case ascertainment, may influence the observed relative risk.

In reviewing the literature for this paper, we have been impressed that much more is known about factors associated with a failure to receive adequate vaccination in different societies than about the nature and the frequency of factors that lead to postponement of vaccination. It may be expected that a number of situations (ill health on the part of the child or other family member, domestic crises in the family) will lead parents to delay taking their child to be vaccinated and that some of these situations will themselves be risk factors for severe neurological episodes or SIDS. For example, Stanton et al  found that parents reported prior symptoms classified as "major", i.e., "...usually needing a medical opinion on the same day and continuing close supervision..." (38, p. 1250) in 48% of 145 SIDS cases as compared to 12% of age-matched controls (odds ratio = 7). It is likely that most parents and health care providers would postpone vaccination of children with such symptoms. Given that studies of associations between vaccination and severe adverse reactions typically focus on narrow time intervals between prior vaccination and onset of the "reaction", it becomes important to understand the nature and frequency of vaccination-postponing factors in study populations. This is an area of research that has attracted inadequate attention in the past.

We have focused in this paper on just one of many methodological problems confronting studies of adverse reactions to vaccinations. Most published discussions of the subject have concentrated upon biases that act to overestimate the relative risk of adverse events following vaccination (10). Biases that underestimate the risk, as discussed here, have received less attention. The fact that such biases do exist makes it difficult to demonstrate convincingly that a vaccine is not responsible for rare, severe, adverse reactions. The avoidance of so many potential confounding factors presents a difficult challenge to epidemiologists who would study the problem of rare, severe, adverse reactions to vaccines. If such studies are to prove useful, they must include strenuous efforts to control for such factors in their design, analysis and interpretation. Whether this is possible at all may be open to discussion. The difficulty of doing so is indisputable.

TABLES AND FIGURES

REFERENCES

1. Plotkin SA, Mortimer EA, ed. Vaccines. Philadelphia: WB Saunders 1988.

2. Lane JM, Millar JD. Routine childhood vaccination against smallpox reconsidered. N Engl J Med 1969; 281:1220-1224.

3. Institute of Medicine. An evaluation of poliomyelitis vaccine policy options. Washington DC: National Academy of Medicine Pub No. IOM 88-04 1988.

4. Miller DL, Alderslade R, Ross EM. Whooping cough and whooping cough vaccine: The risks and benefits debate. Epidemiol Rev 1982; 4: 1-24.

5. Kimura M, Kuno-Sakai H. Epidemiology of pertussis in Japan. Tokai J Exp Clin Med 1988; 13 (suppl):1-7.

6. Hinman AR. DPT vaccine litigation. Am J Dis Child 1986;140:528-530.

7. Smith MH. National Childhood Vaccine Injury Compensation Act. Pediatrics 1988; 82:264-269.

8. Strom BL, ed. Pharmacoepidemiology. New York:Churchill Livingstone 1989.

9. Ray WA, Griffin MR. Use of Medicaid data for pharmacoepidemiology. Am J Epidemiol 1989;129:837-849.

10. Wentz KR, Marcuse EK. DPT and serious neurologic illness: an updated review of the epidemiologic evidence. Pediatrics (in press).

11. Marks JS, et al. Risk factors associated with failure to receive vaccinations. Pediatrics 1979; 64: 304-309.

12. Hutchins SS, et al. Measles outbreak among unvaccinated preschool-aged children: opportunities missed by health care providers to administer measles vaccine. Pediatrics 1989; 83: 369-374.

13. Markland RE, Durand RE.  An investigation of sociopsychological factors affecting infant immunization.  Am J Public Health 1976; 66: 168-70.

14. Hoffman HJ, et al. Diphtheria-tetanus-pertussis immunization and sudden infant death: results of the National Institute of Child Health and Human Development Cooperative epidemiological study of sudden infant death syndrome risk factors. Pediatrics 1987; 79: 598-611.

15. Golding J. Immunizations. In: Butler NR, Golding J, eds. From Birth to Five: a study of the Health and Behaviour of Britain's 5 year-olds. Oxford:Pergamon Press 1986:295-319.

16. Alderslade R, Bellman M, Rawson NSB, Ross EM, Miller DL. The National Childhood Encephalopathy Study. In: Whooping Cough. London: HMSO 1981:79-169.

17. Woodward HB. Some factors associated with non immunization. In: Proc 21st Immunization Conference. Atlanta: Centers for Disease Control 1987:67-69.

18. Gergen PJ, et al. DPT immunization status and tetanus antitoxin titers of Mexican American children ages six months through eleven years. Am J Pub Hlth 1988; 78:1446-1450.

19. Roper J, Day S. Uptake of immunisations in low birthweight infants. Archives of Disease in Childhood 1988; 63: 518-521.

20. Committee on Infectious Diseases. American Academy of Pediatrics. Report of the Committee on Infectious Diseases. 21st ed. Elk Grove, IL: American Academy of Pediatrics 1988:273.

21. Immunization Practices Advisory Committee. Diphtheria, tetanus, and pertussis: guidelines for vaccine prophylaxis and other preventive measures. MMWR 1985;34:405-414, 419-426.

22. Orenstein WA, et al. Preschool immunization - 1988. In: Proc 22nd Immunization Conference. Atlanta: Centers for Disease Control 1988: 121-128.

23. Greenland S, Neutra R. Control of confounding in the assessment of medical technology. Int J Epidemiol 1980;9:361-367.

24. Bernier RH, Frank JA, Dondero TJ, Turner P. Diphtheria-tetanus toxoids-pertussis vaccination and sudden infant deaths in Tennessee. J Pediatr 1982; 101: 419-421.

25. Solberg LK. DPT vaccination, visit to child health center and sudden infant death syndrome (SIDS): evaluation of EPT vaccination. Report to the Oslo Health Council 1985. (Available from CR Manclark. Laboratory of Pertussis. Food and Drug Administration. Bethesda, Md. [NIH laboratory translation 85-152].

26. Taylor EM, Emery JL. Immunisation and cot deaths. Lancet 1982; 2:721.

27. Baraff LJ, Ablon WJ, Weiss RC. Possible temporal association between diphtheria-tetanus toxoid-pertussis vaccination and sudden infant death syndrome. Pediatric Infectious Disease 1983; 2: 7-11.

28. Mortimer EA, Jones PK, Adelson L. DPT and SIDS. Pediatr Infect Dis 1983; 2: 492.

29. Walker AM, Jick H, Perera DR, Thompson RS, Knauss TA. Diphtheria-tetanus-pertussis immunization and sudden infant death syndrome. Am J Pub Hlth 1987; 77: 945-951.

30. Griffin MR, Ray WA, Livengood JR, Schaffner W. Risk of sudden infant death syndrome after immunization with the diphtheria-tetanus- pertussis vaccine. New England Journal of Medicine 1988; 319: 618-623.

31. Miller DL, et al. Risk factors for acute neurological illness in children. In: Proceedings of the Fourth International Symposium on Pertussis, Geneva, 1984. Develop Biol Stand 1985; 61:389-394.

32. Walker AM, Jick H, Perera DR, Knauss TA, Thompson RS. Neurologic events following diphtheria-tetanus-pertussis immunization. Pediatrics 1988;81:345-349.

33. Griffin MR, Ray WA, Mortimer EA, Fenichel GM, Schaffner W. Risk of seizures and encephalopathy after immunization with the diphtheria-tetanus-pertussis vaccine. JAMA 1990;263:1641-1645.

34. Gale JL, Thapa PB, Bobo JK, Wassilak SGF, Mendelman PM, Foy HM. Acute neurological illness and DPT: report of a case-control study in Washington and Oregon. In: Manclark, CR ed. Sixth International Symposium on Pertussis Abstracts. Dept. Health and Human Services, United States Public Health Service, Bethesda, Maryland. DHHS Publication No. (FDA)90-1162 1990:228.

35. Kraus JF, Bulterys M. The epidemiology of sudden infant death syndrome. In: Kiely M, ed. Reproductive and Perinatal Epidemiology, CRC Press, 1989.

36. Jelley DM, Nicholl AG. Pertussis: what percentage of children can we immunize? Br Med J 1984;288:1582-3.

37. Hewitt M. Incidence of contraindications to immunisation. Arch Dis Child 1989;64:1052-3.

38. Stanton AN, Downham MAPS, Oakley JR, Emery JL, Knowelden J. Terminal symptoms in children dying suddenly and unexpectedly at home: preliminary report of the DHSS multicentre study of postneonatal mortality. Br Med J 1978; 2:1249-1251.

ALL INFORMATION, DATA, AND MATERIAL CONTAINED, PRESENTED, OR PROVIDED HERE IS FOR GENERAL INFORMATION PURPOSES ONLY AND IS NOT TO BE CONSTRUED AS REFLECTING THE KNOWLEDGE OR OPINIONS OF THE PUBLISHER, AND IS NOT TO BE CONSTRUED OR INTENDED AS PROVIDING MEDICAL OR LEGAL ADVICE.  THE DECISION WHETHER OR NOT TO VACCINATE IS AN IMPORTANT AND COMPLEX ISSUE AND SHOULD BE MADE BY YOU, AND YOU ALONE, IN CONSULTATION WITH YOUR HEALTH CARE PROVIDER.