follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« The conservation of perplexity . . . | Main | More from ongoing investigation of biased assimilation, disgust, & ideology »
Wednesday
Sep132017

WSMD? JA! Various summary stats on disgust and gm food risks

This is approximately the 9,999th episode in the insanely popular CCP series, "Wanna see more data? Just ask!," the game in which commentators compete for world-wide recognition and fame by proposing amazingly clever hypotheses that can be tested by re-analyzing data collected in one or another CCP study. For "WSMD?, JA!" rules and conditions (including the mandatory release from defamation claims), click here.

So . . . new CCP subscriber @Zach (membership # 14,000,000,041) has asked for some summary statistics on various of the relationships modeled in “Yesterday’s”™ post on “biased assimilation, disgust, & ideology”. The queries seem aimed at a distinctive interpretation (or in any case, an interpretation; no one else has offered any!) of the data presented in the previous post.

Therefore, I’ll supply the data he’s requested, as I understand the requests:

@Zach:  What does the Disgust (z-score) vs Left_right plot look like for GM foods for your sample? I don't see it in either your previous post on the subject or your working paper (from the left panel of Fig 4 in your paper I would guess it's flat). 

I’m understanding this to mean, What would does the distribution of z-scored responses look like for “how disgusted are you with GM foods?” (6-point: “not at all” to “extremely”). This is a simple one:

 

It should be obvious, then, that there’s no partisan influence on disgust toward GM foods. No surprise!

@Zach:  For interpreting this data, it might be useful to see the exact distribution of Disgust ratings ("absolute" Disgust) used to generate the Disgust (z-score). It looks like it's asymmetrical, but it would be good to see how much.

Here I think @Zach is asking to see the frequencies with which each of the “disgust” response categories were selected (I’m reading “asymmetrical” to mean skewed); also “absolute disgust” to mean “normally distributed.”)  Again, not too hard a request to satisfy:”

Next @Zach states,

Similar to [above], it might be interesting to see a version of these plots with an absolute x-scale (e.g. Disgust in units of the first figure in your previous post). Are there trends with "absolute" Disgust and how quickly the lines for the two study assessments deviate?

I’m not 100% sure what @Zach has in mind here. . . . Does he want to see the distributions featured in his first request after responses to “disgust” are transformed back from z-scores to raw scores?  There’s nothing interesting to see in that case: the distribution is the same whether “disgust” is presented in raw form or the z-score transformed one.

But @Zach might be suspicious of the “smoothness” of the regression analyses featured in “Yesterday’s”™ post. The linear regression constrains the variance to appear linear when maybe it really wasn’t in raw form—in which case the linear model of the impact of disgust on GM food concerns would be misspecified. So here is a locally weighted regression plot:

 What does this (on its own or in combination with the other bit of information presented here) signify?  I’m not sure! But @Zach apparently had a hypothesis here, albeit one not completely spelled out, about what this way of reporting the “raw data” would look like.  So I’ll leave it to him & interested others to spell out their interpretations here.

Oh -- @Zach gestures toward his answer—

To combine 2 & 3, if the distribution of "absolute" Disgust is asymmetrical and weighted towards neutral, does that help explain how close the two "safe" and "not safe" branches stick together at low Disgust (z-score)? I.e. the opinion may be extreme for the sample, but on average the person still isn't too disgusted with GM foods?

Does @Zach see this in the data prestened in this post? If so, what’s the upshot? Same if the distributions defy his surmise—what additional insight can we derive, if any, from these distributions?

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (19)

Hello Dan,

Thanks so much for putting in the time for a detailed response!

I apologize for not being clear about the purpose of my questions... All my questions were aimed at understanding why the two trends for "GM food safe" and "GM food dangerous" overlap strongly at low disgust z-score, compared to the other activities.

The first plot you provided was useful, because it showed that response to GM foods is unusual in another way compared to the other activities (in that it's less polarized). From just the data you sent just now, one might hypothesize that the shape of the Study Evaluation vs Disgust (z-score) trend for GM foods is the generic shape for mildly disgusting activities that are not strongly polarized. Potentially the mildly disgusting activities that *are* strongly polarized would then have this base shape, and maybe some modulation of the trend on top of that. I need to look around to see if this is supported by other research (e.g. do other things that are mildly disgusting, but not political, have the same shape as the GM foods trend), and I haven't had the chance yet. I suspect this is too straightforward of an interpretation and drawing too much from one case.

My other questions were based on hypothesis that may have been built around a misunderstanding of your original presented results. I'll comment again later today with a more detailed description of that hypothesis, what I thought your results meant, and maybe some clarifying questions.

In the meantime, thanks again!

Zach

September 13, 2017 | Unregistered CommenterZach Hafen

@Zach-- you might want to peruse this for more background.... Wait a sec; I think in last comment you implied you had already. WEll, in that case, others might want to.

September 13, 2017 | Registered CommenterDan Kahan

link drop:
https://phys.org/news/2017-09-journalists-biased-views-adjudicating-facts.html

Dr Lyons' study showed that journalists can provide one-sided evidence - information from experts supporting a particular view - that has a bigger impact over people's factual beliefs than their partisan or ideological attachments.

non-paywall paper:
https://www.researchgate.net/publication/318324801

September 13, 2017 | Unregistered CommenterJonathan

Hello Dan,

I'm going to try to reword my original hypothesis below, because I think it may still have merit to it.

First, a few clarifications:
1. In the following discussion, it will be useful to define H( disgust ) as the absolute value of the difference between the study evaluation when the study conclusion promotes the activity and the study evaluation when the study conclusion finds the activity to be harmful. In other words, the "distance" between branches on your Study evaluation vs Disgust plot.
2. z-score as I'm used to it is just the number of standard deviations from the mean. You are using that here, right? If you're using a different definition, most of what I say below isn't meaningful.

When I asked for data results, my hypothesis was that H was possibly a function of disgust rating more than disgust z-score.
To explain this further, you can imagine that if H is related to disgust, it could be a function of two ways of measuring disgust:
1. How disgusted a person is with an activity, relative to some standard activities the person uses to measure disgust.
2. How disgusted a person is with an activity, relative to how disgusted the average person is with that activity.
The former (1) should be measured by disgust rating, without regard to what the rest of the sample said about an activity. (As a caution, this assumes that the standards are set approximately evenly from person to person...)
The latter (2) should be measured by disgust z-score, without regard to disgust rating.

I suspected that H was more a function of disgust rating because if so I could imagine a scenario where (a) the Study evaluation vs Disgust (rating) plot for GM foods would fit nicely with the the plots for the other activities, but (b) the Study evaluation vs Disgust (z-score) plot would look as it does currently for GM foods.
To my understanding, this would be pretty simple achieve, requiring only two things:
1. That H be noticeably large for sufficiently low disgust rating, close to zero for medium disgust rating, and noticeably large again for sufficiently high disgust rating.
2. That there aren't very many sufficiently low disgust ratings in the disgust distribution for GM foods (that second plot you so helpfully provided).
In other words, even if people have a relatively extreme response relative to the rest of the sample, if they're still relatively neutral about the subject they won't have much bias towards the studies.
With the second plot you provided, the disgust distribution, I can see that this explanation for the Study evaluation vs Disgust (z-score) plot just doesn't work, because the second requirement isn't satisfied.
So thanks for that!

That said, I'm still a bit confused. You said
the distribution is the same whether "disgust" is presented in raw form or the z-score transformed one
and that's surprising to me. To me that suggests one of three things:
1. The disgust distribution is very similar for every activity, so it doesn't matter whether you present it in raw form or z-score transformed.
2. That how disgusted a person is with an activity, relative to some standard predefined activity the person uses to measure disgust, has no bearing on H.
3. I don't know what I'm talking about and am misunderstanding the math/analysis/social science.
The first would be surprising given the first plot in your post on 9/6/2017, where the means, at minimum, are different.
The second would be surprising because it suggests that it doesn't matter whether or not we're talking about (as an example) cockroaches or bunnies: it's my opinion relative to my peers on how disgusting it is that affects my ability to assimilate new information.
I'll let you judge for yourself whether or not the third possibility is what's happening.

Thoughts?

Thanks just for reading this far,
Zach

September 13, 2017 | Unregistered CommenterZach Hafen

@Zach-- will give all this some thought & respond presently (Th & Fri are my teaching days)

September 14, 2017 | Registered CommenterDan Kahan

link drop - republicans backfire on anti-business sugary drink tax messaging:
http://jhppl.dukejournals.org/content/early/2017/08/08/03616878-4193606.full.pdf

September 14, 2017 | Unregistered CommenterJonathan

@Zach-- as downpayment on response, consider the plot of the disgust measure when converted to z-scores. All this amounts to is substituting one metric for another in quantifying the responses on the 6-point measure; the distribution of the responses is undisturbed, as are the zero-order correlations between those reponses and anything else in the dataset. I think z-score is often more informative than using the arbitrary uits of the untransformed data; plus the centered-at-zero standardized measure makes multivariate analyses a bit easier to intepret if the standarized variable happens to be an element of a cross-product interaction.

Does this change anything in your analysis?

One more point: each study subject saw only one putative object of disgust. None compared or rated the disgustingness of multiple putative disgust objects. Does that matter to you?

Going back to your original point, plotting the distribution of disgust evaluations -- whether raw data or z-score-- does vindicate, I thinkj, your interpretation about why there is convergence of "safe" & "dangerous" at low disgust. That's where most of the observations are concentrated on the disgust measure-- on "none at all"--& their subsequent ratings of the studies were just noise. Most Americans don't have any opinion at all about GM foods-- they just eat loads of them. Right?

September 15, 2017 | Registered CommenterDan Kahan

"republicans backfire on anti-business sugary drink tax messaging"

Oh, yes. Their argument was: "These companies spend millions each year on sophisticated tactics to market products with no nutritional value to kids. Soda companies will say and do almost anything to protect their profits, and they do it at the expense of children’s health." Anyone with any experience of working in a business will know that's simply not true - and will classify the author of this argument as an anti-business nut-case. It just screams 'partisan!' And that discredits the rest of the argument.

In a world where information sources and experts can be wrong, it makes sense to include the credibility of the source as part of the hypothesis/model, add adjust it based on their accuracy on topics you do already know about. If an expert tells you that "Black is white, grass is purple, and skin cream X really works!", you would tend to use the first two statements to assess the credibility of the expert, and adjust the likelihood ratio on the last of these statements accordingly. If an expert tells you "The Democrats are absolutely and totally right about everything, and skin cream X really works!" then you'll get a differential response on the last statement, but from the same effect. Is it irrational?

Given that experts vary in credibility, how would you say a Bayesian *should* account for that? What's the correct calculation, and how do the experiment subjects differ from it?

(I also noted that while they did pro-tax, and both-sides arguments, they didn't test the effect of the pure anti-tax message. I wonder why not?)

I find it a bit difficult to judge, myself, since *none* of the arguments offered are scientifically valid. So from my point of view, the correct answer ought to be that none of them should have any effect on people's beliefs. I wonder how the authors would have interpreted that, if people had responded that way?

September 15, 2017 | Unregistered CommenterNiV

@Dan,

The disgust distribution (z-score) plot definitely changes some things about my analysis. I didn't expect it to look the exact same. I'll think more about why it doesn't look like my expectations, and get back to you.

Regarding not comparing between objects, that's what I thought was the case, so it doesn't change what I was thinking. The assumption I had there was that all measurements have to be relative to something, so when we rate disgust it has to be relative to our previous ideas and experiences of what is disgusting. The assumption that all measurements are relative is perhaps more of a real world thing, and maybe not something that is true in people's minds.

Yes, I think the low H at low z-score could be explained by the distribution as you said. However, I would think this is less the case if the other disgust distributions look similar to the gm food one. If they look roughly the same in shape and the study evaluation vs disgust plots look completely different then that's not a convincing explanation to me.

Apologies for any typos: on my phone.

September 15, 2017 | Unregistered CommenterZach Hafen

NiV,

Anyone with any experience of working in a business will know that's simply not true ...

I have worked in several businesses, and I don't agree with you. However, I do agree that the message will seem to Republicans as obviously coming from Democrats. Still, I would think a Bayesian should just underweight the message, possibly to zero, and not backfire in this case. A backfire here suggests that Republicans think that other relevant information that they had received that improved their opinion of the tax should be recast in a new light by the Democrats' anti-business message. Why?

September 15, 2017 | Unregistered CommenterJonathan

This is chock fulla good stuff:

http://nymag.com/daily/intelligencer/2017/09/does-the-gop-base-love-trump-more-than-it-hates-amnesty.html

For example:


The authors conducted a survey with YouGov of 1,300 voters broken into five subgroups, each of which was asked 10 questions using a research design that employed “both ‘conservative’ and ‘liberal’ Trump cues.”

1. “Do you support or oppose increasing the minimum wage to over $10 an hour?”

2. “Donald Trump has said that he supports this policy. How about you? Do you support or oppose increasing the minimum wage to over $10 an hour?”

3. “Donald Trump has said that he opposes this policy. How about you? Do you support or oppose increasing the minimum wage to over $10 an hour?”
The result: The more strongly a voter identified with the Republican Party, the more likely she was to follow Trump leftward.


More evidence of a sort that leads me to think it's a mistake to think of identification as being primarily driven by ideology or "values" or "world view." Those explanations, IMO, are far too simplistic. Seems to me that people identify for a host of reasons, that can vary over time depending on context. IMO, it's fairly arbitrary, and at some point it may be pointless to expect a clearly describable cause-and-effect. Methinks that sometimes, and perhaps often, identity is just identity.


Critically, the same held true when voters were asked to identify themselves by ideology rather than by party: Strong conservatives were more likely to embrace a Trump-backed “left-wing” position than those voters who identified as more ideologically moderate.

In other words: It appears that when most voters say they are strong conservatives, they mean that they are strong Republicans — and when they say they are strong Republicans, they mean that they are loyal to the Republican president.

This finding is supported by a large — and growing — body of political science research. Most Americans do not have strong, coherent ideologies — but do have strong group identities, which tie them to one of the two major political parties.

Seems to me that people, regularly, will hold contradictory views based on competing axes of identity orientation. Nothing particularly unusual or profound about Kentucky-farmerism, IMO (assuming that is a good example) or other forms of "knowing disbelief."

Edsall opens his piece with one stark testament to this fact: In 2011, 61 percent of white Evangelical Christians disagreed with the statement, “an elected official who commits an immoral act in their personal life can still behave ethically and fulfill their duties in their public and professional life,” according to a poll by the Public Religion Research Institute. This made such Evangelicals the single least-forgiving cohort in the institute’s survey.

Five years and one “grab ’em by the pussy” tape later, 72 percent of white Evangelicals told PRRI that an elected official could behave ethically in public life even if he had committed immoral acts in his private one. Now, no other religious group is more tolerant of sexual libertinism in its political leaders than the one that belongs to “the moral majority.”

Related article:

Related study

September 15, 2017 | Unregistered CommenterJoshua

"I have worked in several businesses, and I don't agree with you."

Really? You've worked for companies that deliberately and knowingly endangered the health of children? What did you do about it?

"A backfire here suggests that Republicans think that other relevant information that they had received that improved their opinion of the tax should be recast in a new light by the Democrats' anti-business message. Why?"

Well, I'm not a Republican, and I already knew about the science, so this may not be relevant. But my reaction to reading the arguments being used to support the case for the tax would have been to switch from the default-in-the-case-of-ignorance hypothesis that there was probably at least some real (if not necessarily conclusive) science behind the proposal to the hypothesis that it's all unscientific rubbish made up by anti-corporate partisans. They're using a classic ad hominem fallacy. If they had a real argument, surely they'd have used it?

If somebody tells you that the company who makes it say skin cream X reduces the rash, you might be broadly sceptical given the source, but accept that there's a significant possibility that it really works. If you're then told that they know this because a shaman told them the spirits had blessed it, and such blessings are an infallible cure for rashes, you'll tend to reduce your belief. Bad arguments reduce the credibility of the source, and if the source is known to be quoting the originator (rather than just giving their own interpretation) of the claim, that reduces the credibility of the claim itself.

I don't know if that's what the Republicans are doing, but I think it's a distinct possibility.

"The result: The more strongly a voter identified with the Republican Party, the more likely she was to follow Trump leftward."

Could that be because of the perceived credibility of the 'expert' source? The paper says: "Those least knowledgeable and most approving of Trump are more likely to react to a Trump cue." As one would predict?

"Now, no other religious group is more tolerant of sexual libertinism in its political leaders than the one that belongs to “the moral majority.”"

Similarly, it would be interesting to know if progressives had shown a similar shift of opinions on misogyny/sexism over the time interval from the aftermath of the Clinton presidency to Trump's parties... ;-)

(By the way, you're doing it again... Launching one-sided partisan attacks on political opponents using the tools of cognitive research. It's kinda ironic, given the subject matter.)

September 15, 2017 | Unregistered CommenterNiV

I wouldn't think that any of the general patterns of behaviors described on those articles are any more prevalent on the right than on the left, even if the specific behaviors described are focused on the right.

September 15, 2017 | Unregistered CommenterJoshua

"I wouldn't think that any of the general patterns of behaviors described on those articles are any more prevalent on the right than on the left, even if the specific behaviors described are focused on the right."

I agree. They're not. But my point is that every single one of your examples *is* focused on the right. It means low information readers on the right seeing your words are going to identify it as nakedly partisan argumentation, and automatically discredit the conclusion because of the unreliability of the source. It's like putting Democrat ex-Vice President Al Gore in charge of persuading Republicans that Global Warming is a threat to take seriously!

I know you know this. I know you know it undermines the effectiveness of your own argument, and the argument for taking cultural cognition seriously. I assume it's because you don't have any intention of having cultural cognition being understood or believed on the other side of the political aisle, and this is some sort of virtue signalling or identity defensive behaviour - but I don't know.

I just enjoy pointing out the irony of you exhibiting the sort of partisan bias we're currently criticising, and thereby triggering the backfire effect we were just discussing! :-) Is it a conscious or deliberate irony on your part?

--
Seriously, one of the problems with getting cultural cognition taken seriously in the political arena is that all the academic psychology researchers are predominantly left-wing, and the majority of the examples they pick are focused on the right, and are aimed at reinforcing public support for left-wing tropes. It's seen as a 'Republican Brain'-style left-wing attack on the rationality of the right - so low information voters on the left naturally assume the effect is asymmetric, and the right assume it's partisan rubbish. The entire field therefore triggers the backfire effect!

One of the things that most impresses me about Dan is that he consciously argues for symmetry of the effect, rendering it politically neutral. It makes me more inclined to take him seriously. It's just a shame that he doesn't get more support doing it.

September 16, 2017 | Unregistered CommenterNiV

NiV,

But my reaction to reading the arguments being used to support the case for the tax would have been to switch from the default-in-the-case-of-ignorance hypothesis that there was probably at least some real (if not necessarily conclusive) science behind the proposal to the hypothesis that it's all unscientific rubbish made up by anti-corporate partisans.

That sounds to me like identity protective backfire, because although you could validly give the anti-business message zero weight due to its wording and thus likely source, there is nothing in the message itself that is evidence of anti-business orientation about anything other than its particular messenger (and the existence of anti-business messengers is already factored into your priors) - thus no reason for a negative weight, nor of anything to prompt rational re-evaluation of prior evidence or default stance. If you can be moved to re-evaluate a default position about the potential for scientific evidence based on one messenger's framing, and if this new evaluation effect persists, such that future second-hand scientific evidence is viewed by you with sufficient suspicion to be underweighted (maybe even negatively weighted) vs. if you were never exposed to that one anti-business message, then it seems like Dan's polluted informational environment theory is spot-on here.

There are several things I think are problematic about the sugary drink tax paper. One is the lack of strength - many of the results are not statistically significant. The other is that the topic chosen isn't likely to demonstrate a symmetric response if there was one - hence it may be misleading because it can only show Republican identity effects. Another is that it doesn't distinguish between subcategories of Republicans that are likely to have different reactions to the anti-business message.

September 16, 2017 | Unregistered CommenterJonathan

@Dan,

I thought about it some more, and worked up a simple example to confirm my intuition. To rephrase it, hopefully more clearly, the point is that transforming to/from z-score shouldn't change the shape of the distribution at all, but it will stretch it and change where it is on an axis, relative to other distributions. Here's a simple example. I have the Python notebook I used to generate the plots if you would like it.

In other words, it shouldn't change how we interpret an individual distribution by itself, but it should change how we interpret a given distribution relative to other distributions.

Adding the regression on top complicates things, of course. However if the underlying disgust distribution changes location on the axis, I would expect the regression to shift and stretch/shrink as well. So when you compare between, say, GM foods and Marijuana use, where each has H = 0 relative to one another should change depending on whether or not you're using z-score or raw data. My hypothesis was that that relationship, between data sets, may make more sense than the current one does, under what I outlined in my post before-last.

Let me know if I haven't explained things clearly yet.

Best,
Zach

September 16, 2017 | Unregistered CommenterZach Hafen

"That sounds to me like identity protective backfire, because although you could validly give the anti-business message zero weight due to its wording and thus likely source, there is nothing in the message itself that is evidence of anti-business orientation about anything other than its particular messenger"

True, but there's nothing in the message to say which bits are by the originators of the claim and which bits have been added by the messenger. A common interpretative heuristic (especially for System 1 processing) is that a message written as a single coherent statement will be all from one source, unless there is specific evidence to the contrary. Another is to treat the argument as the whole argument (or at least the best argument) for the claims made, and the arguer as representative of the whole belief group, unless there is a motivation to look further.

So if you're in favour of a sugar tax for other reasons, you'll already be aware there are other arguments, and interpret this statement accordingly. If you've got no particular knowledge of the debate, or have seen only arguments on the anti-tax side (so you know the arguments being made here are false, but are not aware of the existence of better ones), you'll tend to interpret this as the whole argument for a sugar tax, and clearly invalid.

The problem is, I think this anti-business argument possibly *is* the best, most effective argument, if you're talking to anti-business Democrats! You are reminding them of a reason (one they hold a strong prior on) for discrediting any arguments the manufacturers might make. You're pointing to other things they say and do that your prior beliefs tell you are wrong, and thereby discredit anything else they might say. If your audience doesn't have that prior but has its opposite, you wind up discrediting yourself!

The authors of the statement, presumably, were under the subconscious impression that everyone held that anti-business prior belief, which is why they thought it would be a convincing "refutation", and therefore the best argument to use.

" If you can be moved to re-evaluate a default position about the potential for scientific evidence based on one messenger's framing, and if this new evaluation effect persists, such that future second-hand scientific evidence is viewed by you with sufficient suspicion to be underweighted (maybe even negatively weighted) vs. if you were never exposed to that one anti-business message"

Having identified the anti-business message with the entire sugar-tax campaign, yes, I'd be less likely to accept future unsupported Arguments from Authority from the sugar-tax campaign, and that effect is persistent. However, that wouldn't affect my acceptance of actual empirical scientific evidence. It would last until I received evidence that there were better arguments.

Degradation of source credibility should only affect Arguments from Authority. And since that's a fallacy, you could argue that this state is actually the more rational one! :-)

September 17, 2017 | Unregistered CommenterNiV

@Zach: I'll have to think about your point generally.

But for now consider this correlation matrix. It shows not only that (1) gm2 and z_gm2 (raw and z-score transformed values for disgust at gm food, respectively) are correlated at 1.0; but also that (2) the correlation between either of these variables and either the raw or z-score versions of "conservrepub" (the measure of political outlooks; aka "Left_right") is 0.04.

Also these figures, which show that gm food & all the other risks have the same relationship w/ conservrepub, regardless of whether the disgust measure is "raw" or z-score transformed. (Don't be confused by apparent difference in steepness of slopes in right-hand figure; that's a consequence of z-score being plotted on smaller region of the y-axis variable.)

In my understanding, all of these results are tautological, since correlations are computed using a z-score transformation!

I know that what you are saying would apply to log- & other transformations that are designed to "fix" the skew of data that is being modeled w/ linear regresssion. But I still don't (at least yet) see why z-score transformation-- which just uses a different metric for conveying information on a particular distribution-- would have this effect.

(By all means keep embedding URLs for any figures or other graphics you want to show me, but realize that if there are too many links in comment, the trigger-happy spam filter might take your post prisoner. He or she will get out, but our msg won't show right away.)

September 17, 2017 | Registered CommenterDan Kahan

Hi @Dan,

Thanks for your response.

The correlation values make sense, including in the context of what I'm suggesting, although I'm a bit surprised at the figures you showed. They suggest that the difference between using raw score or z-score for comparison is minor, at least for this data set, regardless of whether or not fundamentally there's a technical difference.

I still don't entirely understand why z-score would not affect comparisons between results, so I've included some more discussion below. However, at this point it probably won't make a difference for your data set, so read at your own discretion.

Anyways, thanks for the conversation. I learned a lot more about your work, and how subtle it is!

Best,
Zach

Further extra-optional discussion:

As an example of how using raw score or z-score could make a difference, consider two measurements: one for disgust of GM foods, one for disgust of marijuana. Suppose the GM food measurement is a disgust rating of 3, and the marijuana measurement is a disgust rating of 4. Further suppose that the mean for the disgust ratings of GM foods is 3, and the mean for the disgust ratings of marijuana is 4. Then, when transformed, the two measurements will be at the same location on the disgust (z-score) axis: 0. However, prior to transformation there was a non-negligible offset between the measurements.

Fundamentally this is because we are comparing data using two different axes, and, while both axes can be shared between samples, to get from one axis to the next we are applying a different transformation for each sample.

Here I've rephrased my statement in more mathematical terms, to make sure I'm thorough:
Consider N sets of measurements, each measuring n_samples_i ratings of X for subject Y_i (e.g. 700 ratings of disgust for GM foods, 1000 ratings of disgust for marijuana, etc ). Let S_i be the actual sample of ratings of X for subject Y_i (e.g. this is the distribution of disgust rating). Let SZ_i be S_i transformed to z-score, i.e. n_samples_i z-scores of X for subject Y_i. The transformation from S_i to SZ_i depends on the mean of S_i (mu_i) and the standard deviation (sigma_i), so each sample S_i has an different transformation (T_i). Now consider the situation where either ratings of X or z-scores of X are used as one of the axes of a plot showing, in some form, multiple S_i's or all SZ_i's. To get to/from an axis displaying ratings of X or z-scores of X, you can't just scale by some constant transform. Each S_i must be transformed individually. As long as each T_i is unique then SZ_i will end up in a different position relative to SZ_j, by definition.

In other words, if you're changing axes by applying a transform T_i to each data set, as long as T_i is unique to each data set, the location of the sample on the axes afterwards will change relative to the other samples.

September 19, 2017 | Unregistered CommenterZach Hafen

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>