"SCS_1.0": Measuring science curiosity - Cultural Cognition of Health

Yesterday, I discussed how evolution “believers” and “nonbelievers” reacted to a cool evolution-science documentary. The data I described came from Study No. 1 of the Annenberg Public Policy Center/CCP “Evidence-based Science Filmmaking Initiative” (ESFI).

That data suggested that “belief” in evolution wasn’t nearly as important to engagement with the documentary ( Your Inner Fish , an award-winning film produced by ESFI collaborator Tangled Bank Studios) as was science curiosity .

Today I’ll say a bit more about how we measured science curiosity.

Developing a valid and reliable science curiosity scale was one of the principal aims of Study No. 1.  As conceptualized here, science curiosity is not a simple transient state (Loewenstein 1999) but instead a general disposition, variable in intensity across persons, that reflects the motivation to seek out and consume scientific information for personal pleasure.

Obviously, a measure of this disposition would furnish science journalists, science filmmakers, and related science-communication professionals with a useful tool for perfecting the appeal of their work to those individuals who value it the most. But it could also make myriad other contributions to the advancement of knowledge.

A valid science curiosity measure could be used to improve science education, for example, by facilitating investigation of the forms of pedagogy most likely to promote its development and harness it to promote learning (Blalock, Lichtenstin, Owen & Pruski 2008). Those who study the science of science communication (Fischhoff & Scheufele 2014; Kahan 2015) could also use a science curiosity measure to deepen their understanding of how public interest in science shapes the responsiveness of democratically accountable institutions to policy-relevant evidence.

Indeed, the benefits of measuring science curiosity are so numerous and so substantial that it would be natural to assume researchers must have created such a measure long ago.  But the simple truth is that they have not.

“Science interest” measures abound. But every serious attempt to assess their performance has concluded that they are psychometrically weak and, more important, not genuinely predictive of what they are supposed to be assessing—namely, the disposition to seek out and consume scientific information for personal satisfaction (Blalock et al 2008; Osborne, Collins & Simons 2003).

ESFI’s “Science Curiosity Scale 1.0” (SCS_1.0) is an initial step toward filling this gap in the study of science  communication.  The items it comprises, and the process used to select (and combine) them, self-consciously address the defects in existing scales.

One of these is the excessive reliance on self-report measures. Existing scales relentlessly interrogate the respondents on the single topic of their own attraction to or aversion toward information on scientific discovery: “I am curious about the world in which we live,” “I find it boring to hear about new ideas,” “I get bored when watching science programs on TV,” etc.  Items like these are well-known to elicit responses that exaggerate respondents’ possession of desirable traits or attributes.

To counteract this dynamic, SCS_1.0 disguises its objectives by presenting itself as a general “marketing” survey.

Individual self-report items relating specifically to science were thus embedded in discrete blocks or modules, each consisting of ten or more items relating to an array of “topics” that “some people are interested in, and some people are not.” Items were presented in random order, each on a separate screen.

There was thus no reason for subjects to suspect that their motivation to learn about science was of particular interest, nor any opportunity for them to adjust the responses across items in a manner that overstated their interest in it.  A similar strategy was used to gather information on behavior reflecting such an interest, including visits to science museums, attendance at public science lectures, and the reading of books on scientific discovery.

SCS_1.0 also featured an objective performance measure.

Well into the survey, subjects were advised that we were interested in their reactions to a news story “of interest” to them.  In order to assure that the story was one that in fact matched their interests, they were furnished with discrete news story sets , the shared subject matter of which was be identified by a header and reinforced by the individual story headlines and graphics. One set consisted of science stories; the others ones on popular entertainment, on sports, and on financial news.

Subjects, we anticipated, were likely to find the prospect of reading a story and answering questions about it burdensome.  Accordingly, the selection of the science set rather than one of the others would be a valid indicator of genuine science interest . Responses to this task were then used to validate the self-reported interest items to help furnish assurance the genuineness of the latter.

When combined, the items displayed the requisite psychometric properties of a valid and reliable scale.  Their unidimensional covariance structure warranted the inference that they were measuring the same latent disposition.  Formed with item response theory, the composite scale weighted particular items in relation to the level of the disposition that responses to them evinced. The result was an index—SCS_1.0—that reflected a high degree of measurement precision along the entire population distribution of that trait (Embretson & Paul 2000).

Finally and most importantly, SCS_1.0 was behaviorally validated .

As detailed in ESFI Study Report No. 1 , subjects were instructed to watch a 10-minute clip from the science documentary Your Inner Fish .  SCS_1.0 strongly predicted engagement with the clip as reflected not only in self-reported interest but also in objective measures such as duration of viewing time and subjects’ election (or not) to be furnished free access to the documentary as a whole.

SCS_1.0 is by no means understood to be an ideal science curiosity measure .  Additional testing is necessary, both to assure the robustness of the scale and to refine its powers to discern the motivation to seek out and consume science information for pleasure.