Reading The Plaque - Looking At Primary Sources In Science Journalism

In the last year, I have probably read more science articles outside my field than ever and it has opened up many interesting ideas and at times rocked my world(view). The internet makes it easier to follow my interests but, at times is less serendipitous, the routes I tread through cyberspace sometimes become ruts. So often I look at curated sites, where they gather articles on a more general topic, like, in fact, this website.

Today I want to share the biggest lesson from my wanderings, and for the neat summarizing of the concept, I want to thank Roman Mars, whose podcast 99% invisible, about design in our world, has its motto “Always read the plaque.” Yes, I am talking about those little brass signs affixed everywhere that tell us about where we are standing in both time and space. As another put it, “When you see a plaque it means that something happened there important enough to someone to go through the process of erecting a plaque or monument. Surely that is worth your time.”

In reading science journalism we must always remember we are reading a secondary source, As the Smithsonian defines it, “summaries, second-hand accounts, and analyses of events created by someone who did not witness the event but may have read or heard about it.” So when I say, you should "read the plaque"

what I am getting at is that you should look at the primary source, the original report or study, without that intentional or unintentional bias of the secondary sources author (that would be me). It is a mark of reputable science journalism that there is a link to the primary source.

When you begin to look at science from the viewpoint of always reading the plaque there is another important understanding; meta-analysis and systematic reviews are also secondary, not primary, sources but science journalists conflate the two. After all, like a primary source a meta-analysis can be published in a peer-reviewed journal, and there is a whole methodology and results section, just like other science studies. But they remain summaries, second-hand accounts. And because meta-analysis is a frequently used as a means of substituting overview for understanding, especially for situations where we wish to influence thinking or policy, not looking at the primary sources can be hazardous to your world view.

Here is an example from the AHRQ’s The Effectiveness and Risks of Long-Term Opioid Treatment of Chronic Pain regarding chronic opioid use and the incidence of myocardial infarction. The quality of the evidence is ‘low,' but they summarize their review paper’s finding as:

Current opioid use associated with increased risk of myocardial infarction versus nonuse (adjusted OR 1.28, 95% CI 1.19 to 1.37 and incidence rate ratio 2.66, 95% CI 2.30 to 3.08)

However, when you look at the actual study [1] you find an interesting distinction between the current opioid users and nonusers laid out in Table 3. In the opioid user group the baseline incidence of unstable angina was five times as great as the nonuser control; Angina, Coronary artery disease, congestive heart failure four times as high; arrhythmias, diabetes, hypertension three times as great, cholesterol twice as great and smoking seven times as great as the controls at their baseline.

Do these seem to be matched controls in discussion coronary disease? You might do as well as conclude that opioid use was protective from myocardial infarctions given the much higher incidence of risk factors in the none use group. And one more thought, though it isn't a condemnation. The study was funded by GlaxoSmithKline who was embroiled in controversy surrounding coronary events in the COX-2-inhibitors. The study’s primary finding was the “Increased cardiovascular risk was associated with chronic use of analgesia in users of both opioids and COX‐2 inhibitors.”

Does this make you question the study? People often claim someone is bought off when they read papers that violate their worldview, so it probably does invalidate the results for some.

A final example. The Memory Palace is a podcast by Nick DeMeo, currently an artist in residence at the Metropolitan Museum of Art. He talks about memories and pasts and he speaks so well it's worth the listen. But there is a particular episode, about plaques, that captures what I have been trying to say: Episode 73 Notes on an Imagined Plaque to be Added to the Statue of General Nathan Bedford Forrest. It's eleven minutes in length; you can download it here.

 NOTE:

[1] For those of you following the links to original studies I know I have led you to another summary, because even though most journal studies are funded by taxpayers, the public often does not have access without paying a handsome fee to a corporation. But taking the PMID or the DOI numbers and submitting them to Sci-hub will get you to the original material