follow CCP

Recent blog entries
« Q. Where do cultural predispositions come from in the cultural cognition theory? A. They are exogenous -- descriptivey & *normatively*! | Main | A nice empirical study of vaccine risk communication--and an unfortunate, empirically uninformed reaction to it »
Thursday
Mar062014

Developing effective vaccine-risk communication strategies: *Definitely* measure, but measure what *counts*

Now that the the important Nyhan et al. study on vaccine-risk communications has gotten people's attention on the hazards of empirically uninformed vaccine-risk communication, it's important to reflect on exactly what it means for risk-communication to be genuinely evidence based. Here's a contribution toward the discussion, excerpted from the "Recommendations" section of the CCP Vaccine Risk Perceptions and Ad Hoc Risk Communication study. 

5. Reject story-telling alternatives to valid empirical analysis of public perceptions of vaccine risk.
 

Decision science has established a rich stock of mechanisms, from “anchoring” to “availability,” from “probability neglect” to “hyperbolic discounting,” from “overconfidence bias” to “pluralistic ignorance.” Treating them as a collection of story-telling templates, a person of even modest intelligence can easily use these dynamics to fabricate plausible “scientific” “explanations” for any observed social phenomenon (e.g., Brooks 2012). But the narrative coherence of such syntheses furnishes no grounds for crediting them as true. They are at best conjectures—fuel for the empirical-testing engine that alone propels genuine insight (Kahan 2014)—and when not acknowledged as such suggest either the expositor’s ignorance of the difference between story-telling and science or his or her intention to exploit the lack of such understanding on the part of others (Rachlinski 2001).

The case of vaccine-risk perceptions supplies a compelling example of the dangers of treating this genre of writing as a source of reliable guidance for practical decisions. In a compelling proof of the utility of decision science as a grab-bag of prefabricated story-telling templates, numerous commentators, popular and even scholarly, have used the inventory of mechanisms it comprises to “explain” a nonexistent phenomenon—namely, a “growing public distrust” of the safety of vaccinations (e.g., MacDonald, Smith & Appleton 2012).

Risk communication is a critical element of public health policy. It is a mistake for public health officials and professionals to exempt it from their field’s norm of evidence-based practice.

The number of genuine and valid empirical examinations of the general public’s perceptions of childhood vaccines is regrettably smaller than it should be. But both to promote the enlargement of it and to protect public health policy from the potentially deleterious consequences of seeking guidance from faux-empirical substitutes, those committed to conserving the high existing level of public support for universal immunization should base their risk-communication strategies on empirically informed assessments of who fears what and why in the domain of childhood vaccines.

6. Use behavioral measures to assess behavior; use fine-grained, field research & not surveys/polls to understand dynamics of resistance.
 

This study combined an attitudinal survey and an experiment. When administered to a diverse and appropriately recruited sample, attitudinal surveys enable measurement of the impact of affective and group-affinities on societal risk perceptions and information processing. These dynamics are important, because they reflect the quality of the science communication environment in which individuals evaluate risk information relevant to personal and collective decisionmaking.

But as stressed at the outset, survey methods alone are not valid for assessing the impact of vaccine-risk perceptions on the actual decisions of parents to permit their children to be vaccinated. Parents’ self-reports are not a reliable or valid measure of their children’s vaccination status; only behavioral measures akin to those reflected in the NIS are. Accordingly, researchers who use observational methods to investigate variance in vaccination coverage should rely on the NIS or on other valid behavioral measures of vaccination status (Opel et al. 2011b, 2013b).

The study results also suggest two other important limitations on survey methods. First, survey measures are unlikely to support valid inferences about the proportion of the public that holds beliefs or opinions on specific issues relating to vaccines, including the likelihood that vaccines cause autism or other diseases.

Because members of the public often have not formed opinions on or given meaningful attention to specific public policy issues (e.g., stem cell research), it is a mistake in general to treat specifically worded survey items (“Based on what you have read or heard, do you think the federal government should or should not fund federal stem cell research?”) as genuinely measuring positions on those matters (Bishop 2005; Schuman 1998). If such items are reliably measuring anything, it is an expression of a more generally pro- or con-attitude that is evoked by the item (in the case of stem cells, positions on “government spending” or possibly “abortion” or even simple partisan affiliation). What that attitude consists in cannot be reliably analyzed unless responses to the item in question are compared with responses to other items that would help to pin down the latent disposition that they are measuring (Berinsky & Drukman 2007).

The coherence of the responses to the items that made up the PUBLIC_HEALTH scale—and in particular, the high, inverse correlation between the perceived risks of vaccines and the perceived benefits of them—suggest that what those items are measuring is an affective orientation (Slovic 2010) toward childhood vaccines. Under these circumstances, reliable inferences can be drawn from vaccine-risk/benefit items only about the valence of individuals’ affective orientation. But no single item can reliably be treated as revealing anything more specific—or more edifying—than that.

This point was illustrated by responses to the item on “postnatal isoerythrolysis.” Survey participants’ beliefs that childhood vaccination caused this fictional disease—one they necessarily had not heard of before—were highly correlated with their responses to every one of the other diverse risk-benefit items used to form the PUBLIC_HEALTH scale. Rather than reflecting a specific belief formed on the basis of exposure to information on vaccine risks, the affective orientation measured by “vaccine disease risk” items should be interpreted as an emotional predisposition to credit or dismiss propositions conditional on their perceived conformity to one’s orientation (Loewenstein et al. 2001; Slovic et al. 2004).

Researchers might well have good reason to assess public knowledge about specific issues such as the impact that vaccines have on the risk of contracting autism or other diseases. But to do so, they will need to follow the steps necessary to form valid measures of such knowledge. Or in other words, they will need to use the psychometric tools that distinguish scholarly opinion research from popular opinion polling (Bishop 2005).

Second, general-population survey measures cannot be expected to generate insight into the identity or motivations of that portion of the population that is genuinely hostile to childhood vaccination. As the analysis of sources of variation in the PUBLIC_HEALTH scale revealed, none of the familiar cultural styles divided over other societal risks (such as those associated with climate change or nuclear power) has a negative affective orientation toward vaccines. To the extent that they explain any variance at all, these styles are associated only with differences in the intensity of the positive affective orientation toward vaccines that prevails in all these groups. Accordingly, none of the demographic or attitudinal indicators used to identify members of those groups can be expected to identify the characteristics that indicate the presence of whatever group identity might be shared by members of the “anti-vaccine” fringe.

There are without question groups of individuals, some in geographically concentrated areas, who are hostile to childhood vaccines (Mnookin 2011). Who they are and why they feel the way they do are questions that merit serious study. But to answer these questions, researchers will need to use measures that are more fine-grained and discerning than the ones that can profitably be made use of in studying the small class of risk issues on which there is genuine cultural contestation.

Such research is now emerging. In a pair of studies, Opel and his collaborators (2011a, 2011b, 2013b) have devised a “vaccine hesitancy” scale for new parents that predicts delay or avoidance of vaccination behavior. Such a screening device would be comparable to ones used in diverse fields from credit assessment (e.g., Klinger, Khwaja & Lamonte 2013) to organizational staffing (e.g., Ones et al. 2007), not to mention to ones used to predict or diagnose disease vulnerability (e.g., Wilkins et al. 2013). If perfected, it could be used by researchers to guide their investigation of who fears vaccines and why and to focus their testing of risk communication materials.

Resources—financial, intellectual, and social—should be devoted to the extension and refinement of these methods rather than ones that focus on attitudinal correlates of vaccine risk perceptions in more diffuse elements of the general population. In order for vaccine-risk communication to be empirically informed, it is essential not only to measure but to measure what counts.

7. Empirical study should be used to develop appropriately targeted risk communication strategies that are themselves appropriately responsive to empirically identified risk-perception concerns.
 

Anyone who dismisses the existence or seriousness of unfounded fears of childhood vaccines would be behaving foolishly. Skilled journalists and others have vividly documented enclaves of concerted resistance to universal immunization programs. Experienced practitioners furnish credible reports of higher numbers of parents seeking counsel and assurance of vaccine safety. And valid measures of vaccination coverage and childhood disease outbreaks confirm that the incidence of such outbreaks is higher in the enclaves in which vaccine coverage falls dangerously short of the high rates of vaccination prevailing at the national level (Atwell et al. 2013; Glanz et al. 2013; Omer et al. 2008).

At the same time, only someone insufficiently attuned to the insights and methods of the science of science communication would infer that this threat to public health warrants a large-scale, sweeping “education” or “marketing” campaign aimed at parents generally or at the public at large. The potentially negative consequences of such a campaign would not be limited to the waste of furnishing assurances of safety to large numbers of people who are in no need of it. High-profile, emphatic assurances of safety themselves tend to generate concern (Kahan 2013a; Kasperson et al. 1988). A broad scale and indiscriminant campaign to communicate vaccine safety—particularly if understood to be motivated by a general decline in vaccination rates—could also furnish a cue that cooperation with universal immunizations programs is low, potentially undermining reciprocal motivations to contribute to the public good of herd immunity. Lastly, such a campaign would create an advocacy climate ripe for the introduction of cultural partisanship and recrimination of the sort known to disable citizens’ capacity to recognize valid decision-relevant science generally (Bolsen & Druckman 2013; Kahan 2012), and valid science relevant to vaccines in particular (Gollust, Dempsey, Lantz, Ubel & Fowler 2010; Kahan, Braman, Cohen, Gastil & Slovic 2010).

The right response to dynamics productive of excess concern over risk is empirically informed risk communication strategies tailored to those specific dynamics. Relevant dynamics in this setting include not only those that motivate enclaves of resistance to universal immunization but also those that figure in the concerns of individual parents seeking counsel, as they ought to, from their families’ pediatricians. Risk communication strategies specifically responsive to those dynamics should be formulated (e.g., NCIRS 2013)—and they should be tested, both in the course of their development and in their administration (Shourie et al. 2013), so that those engaged in carrying them out can be confident that they are taking steps that are likely to work and can calibrate their approach as they learn more (Sadaf et al. 2013; Opel et al. 2012).

Again, preliminary research of this sort has commended. Perfection of behavioral-prediction profiles of the sort featured in Opel et al. (2011a, 2011b, 2013b) would not only enable researchers to extend understanding of the sources and consequences of genuine vaccine hesitancy but also to test focused risk-communication strategies on appropriate message recipients.  If made sufficiently precise, screening protocols of this sort would also enable practitioners to accurately identify parents in need of counseling, and public health officials to identify regions where the extent of hesitancy warrants intervention.

The public health establishment should exercise leadership to make health professionals and other concerned individuals and groups appreciate the distinction between targeted strategies of this sort and the ad hoc forms of risk communication that were the focus of this study.  They should help such groups understand in addition that support for the former does not justify either encouragement or tolerance of the latter. 

Refs

Atwell, J.E., et al. Nonmedical Vaccine Exemptions and Pertussis in California, 2010. Pediatrics 132, 624-630 (2013).

Berinsky, A.J. & Druckman, J.N. The Polls—Review: Public Opinion Research and Support for the Iraq War. Public Opin Quart 71, 126-141 (2007).

Bishop, G.F. The Illusion of Public Opinion : Fact and Artifact in American Public Opinion Polls (Rowman & Littlefield, Lanham, MD, 2005).

Bolsen, T., Druckman, J. & Cook, F.L. The Effects of the Politicization of Science on Public Support for Emergent Technologies. Institute for Policy Research Northwestern University Working Paper Series (2013). Available at http://www.ipr.northwestern.edu/publications/papers/2013/ipr-wp-13-11.html

Bowles, S. & Gintis, H. A cooperative species : Human reciprocity and its evolution (Princeton University Press, Princeton, 2013).

Brooks, D. The Social Animal : The Hidden Sources of Love, Character, and Achievement (Random House Trade Paperbacks, New York, 2012).

Glanz, J.M., et al. A Population-Based Cohort Study of Undervaccination in 8 Managed Care Organizations across the United States Managed Care Organizations. JAMA pediatrics 167, 274-281 (2013).

Gollust, S.E., Dempsey, A.F., Lantz, P.M., Ubel, P.A. & Fowler, E.F. Controversy Undermines Support for State Mandates on the Human Papillomavirus Vaccine. Health Affair 29, 2041-2046 (2010).

Kahan, D. Making Climate-Science Communication Evidence Based—All the Way Down. In Culture, Politics and Climate Change, eds. M. Boykoff & D. Crow. (Routledge Press, 2014).

Kahan, D., Braman, D., Cohen, G., Gastil, J. & Slovic, P. Who Fears the HPV Vaccine, Who Doesn’t, and Why? An Experimental Study of the Mechanisms of Cultural Cognition. Law Human Behav 34, 501-516 (2010).

Kahan, D.M. A Risky Science Communication Environment for Vaccines. Science 342, 53-54 (2013a).

Kasperson, R.E., et al. The Social Amplification of Risk: A Conceptual Framework. Risk Analysis 8, 177-187 (1988).

Klinger, B., Khwaja, A. & LaMonte, J. Improving credit risk analysis with psychometrics in Peru. (Inter-American Development Bank, 2013). Available a0074

Loewenstein, G.F., Weber, E.U., Hsee, C.K. & Welch, N. Risk as Feelings. Psychological Bulletin 127, 267-287 (2001).

MacDonald, N.E., Smith, J. & Appleton, M. Risk Perception, Risk Management and Safety Assessment: What Can Governments Do to Increase Public Confidence in Their Vaccine System? Biologicals 40, 384-388 (2012).

Mnookin, S. The Panic Virus : A True Story of Medicine, Science, and Fear (Simon & Schuster, New York, 2011).

NCIRS, MMR Decision Aid (2013). Available at http://www.ncirs.edu.au/immunisation/education/mmr-decision/index.php.

Omer, S.B., et al. Geographic Clustering of Nonmedical Exemptions to School Immunization Requirements and Associations with Geographic Clustering of Pertussis. American Journal of Epidemiology 168, 1389-1396 (2008).

Ones, D.S., Dilchert, S., Viswesvaran, C. & Judge, T.A. In support of personality assessment in organizational settings. Personnel Psychology 60, 995-1027 (2007)

Opel, D.J., et al. Characterizing Providers’ Immunization Communication Practices During Health Supervision Visits with Vaccine-Hesitant Parents: A Pilot Study. Vaccine 30, 1269-1275 (2012).

Opel, D.J., et al. Characterizing Providers’ Immunization Communication Practices During Health Supervision Visits with Vaccine-Hesitant Parents: A Pilot Study. Vaccine 30, 1269-1275 (2012).

Opel, D.J., et al. Development of a survey to identify vaccine-hesitant parents: The parent attitudes about childhood vaccines survey. Human Vaccines 7, 419-425 (2011a).

Opel, D.J., et al. The Relationship between Parent Attitudes About Childhood Vaccines Survey Scores and Future Child Immunization Status: A Validation Study. JAMA pediatrics 167, 1065-1071 (2013b).

Opel, D.J., et al. Validity and reliability of a survey to identify vaccine-hesitant parents. Vaccine 29, 6598-6605 (2011b).

Otto, S. Antiscience Beliefs Jeopardize U.S. Democracy. in Scientific American (Oct. 16, 2012a), available at http://www.scientificamerican.com/article.cfm?id=antiscience-beliefs-jeopardize-us-democracy.

Otto, S.L. One Way to Help Science: Become Republican. Nature Medicine 18, 17 (2012b).

Rachlinski, J.J. Comment: Is Evolutionary Analysis of Law Science or Storytelling. Jurimetrics 41, 365-370 (2001).

Sadaf, A., Richards, J.L., Glanz, J., Salmon, D.A. & Omer, S.B. A Systematic Review of Interventions for Reducing Parental Vaccine Refusal and Vaccine Hesitancy. Vaccine 31, 4293-4304 (2013).

Shourie, S., Jackson, C., Cheater, F., Bekker, H., Edlin, R., Tubeuf, S., Harrison, W., McAleese, E., Schweiger, M. & Bleasby, B. A cluster randomised controlled trial of a web based decision aid to support parents’ decisions about their child's Measles Mumps and Rubella (MMR) vaccination. Vaccine 31, 6003-6010 (2013).

Slovic, P. The feeling of risk : new perspectives on risk perception (Earthscan, London ; Washington, DC, 2010).

Slovic, P., Finucane, M.L., Peters, E. & MacGregor, D.G. Risk as Analysis and Risk as Feelings: Some Thoughts About Affect, Reason, Risk, and Rationality. Risk Analysis 24, 311-322 (2004).

Slovic, P., Peters, E., Finucane, M.L. & MacGregor, D.G. Affect, Risk, and Decision Making. Health Psychology 24, S35-S40 (2005).

Wilkins, C.H., Roe, C.M., Morris, J.C. & Galvin, J.E. Mild physical impairment predicts future diagnosis of dementia of the Alzheimer’s type. Journal of the American Geriatrics Society 61, 1055-1059 (2013).

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (2)

I tried to make this comment last night on your post from March 4th, but the CAPTCHA validation tool wasn't displayed and I couldn't submit the comment. I guess it fits just as well here.

I fail to see why you consider this to be an important paper. It appears to me the usefulness of the statistics is extremely questionable. I see little point in studying people who have children and have already made a decision whether to vaccinate or not. The study might have some minimal value if it was of couples with no children, but with intent to have some.

Giving groups already having children up to age 17 different information isn't likely to sway whatever choice they already made regarding their children. They likely have already discussed and thought about their decision. They also have accumulated over time some observational information concerning the result of their choice. If they haven't seen any negative consequences to their choice, I wouldn't expect them to change their mind. Would you change your mind if someone you didn't know solicited you for a study and provided only a small selection of the information known to be available?

March 6, 2014 | Unregistered CommenterBob Koss

@Bob--

Sorry about the arbitrary CAPTCHA (it definitely is mischievous...).

I think you are absolutely right that the design & sample do not permit confident inferences to be drawn from the study about how parents of vaccine-age childrent would behave when exposed to information of the sort featured in the experiment.

I think, too, that that point merits stressing, because studies that do enable such inferences are desperately needed, and there shouldn't be confusion among researchers or research funders about what sorts of methods to employ.

But I do think that the study helps to illustrate the impact of certain styles of "ad hoc" vaccine risk communication on the larger collection of influences that determine the affective stance or oroientation of members of the general public toward vaccines.

We should definitely pay attention to those, b/c even if they aren't the proximate drivers of parental behavior, they form the science communication environment in which everyone, including parents, use their reasoning capacity to figure out what the best evidence is. The style of communication featured in some of the experimental communications & displayed by much of the media coverage of the study itself can be shown -- was shown in that very study! -- to have effects that could damage that aspect of the science communication environment for vaccines.

I discuss these issues in the CCP Report, but I acknowledge that they are complicated & my own answers only the "best I can do for now." Definitely feel free to respond if you have additional thoughts!

March 8, 2014 | Registered CommenterDan Kahan

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>