A recent article in the Times swallowed whole a junk study about an increase in death from eating red meat. ACSH advisor and biostatistician Dr. Stan Young dissected out the fat, bone, and grizzle, and found... nothing left. Here is his analysis, which also serves very well as a primer for anyone looking to debunk retrospective epidemiological studies.
"Background: Previous studies investigating protein intake in relation to mortality have provided conflicting results."
"Objective: We investigated the associations of dietary protein and protein sources with risk of disease death in the prospective, population-based Kuopio Ischaemic Heart Disease Risk Factor Study."
SY: This is an ongoing study, started in the 1980s. All manner of things are measured and all manner of things are statistically analyzed using whatever statistical method seems appropriate at the time. This is an exploratory study.
"Methods: The study population consisted of 2641 Finnish men, aged 42–60 y at baseline in 1984–1989. We estimated protein intakes with 4-d dietary records at baseline and collected data on disease deaths from the national Causes of Death Register. Cox proportional hazards regression models were used to estimate HRs and 95% CIs."
SY: There was an initial food frequency questionnaire with 126 foods. How is it possible that the foods you ate in 1984 to have anything to do with your current health? Your digestive system chops things up. Everything becomes soup to be reformulated. It would be magic. In fact, the magic is asking hundreds to thousands of questions with a 5% false positive rate. We are watching a great science business model.
"Results: During the average follow-up of 22.3 y, we observed 1225 deaths due to disease. Higher intakes of total protein and animal protein had borderline statistically significant associations with increased mortality risk: multivariable-adjusted HR (95% CI) in the highest compared with the lowest quartile for total protein intake = 1.17 (0.99, 1.39; P-trend across quartiles = 0.07) and for animal protein intake = 1.13 (0.95, 1.35; P-trend = 0.04). Higher animal-to-plant protein ratio (extreme-quartile HR = 1.23; 95% CI: 1.02, 1.49; P-trend = 0.01) and higher meat intake (extreme-quartile HR = 1.23; 95% CI: 1.04, 1.47; P-trend = 0.01) were associated with increased mortality."
SY: Where to start? First, let's give a clear table of results:
RR CI (low) CI (high) Trend
1. 1.17 (0.99, 1.39; P-trend = 0.07)
2. 1.13 (0.95, 1.35; P-trend = 0.04)
3. 1.23 (1.02, 1.49; P-trend = 0.01)
4. 1.23 (1.04, 1.47; P-trend = 0.01)
Second: Some background:
- A Risk Ratio, RR, is the ratio of two risks. A RR of 1.00 means no effect. One rule of thumb is that a RR should be larger than 2 or smaller than 0.5 to be considered reliable/important/worthy of interest. None of these RRs meet this standard.
- The true RR is expected to be within the Confidence Intervals (CI) 95% of the time. This means that to be anything close to reliable the CIs cannot include the no-effect value of 1.00. RRs 1. and 2. fail. RRs 3. and 4. perilously close to including 1.00. In fact, if the confidence intervals are adjusted to reflect the multiple testing, they would include 1.00 consistent with no effect.
- The researchers created groups of people by “cutting” the predictor variable, meat, into four groups, those eating approximately 76, 125, 171, and 251 grams of meat per day. These groups had 322, 309, 304 and 290 deaths respectively. On the face of it, eating meat is good for longevity. Not so fast. The authors found a way to “adjust” the results. They show 47 possible adjustment variables in their Table 1. Remember they had a total of 126 common foods and drinks in their questionnaire; any of these could be used to adjust the mortality. They were able to adjust the beneficial effect of meat into a detrimental effect.
The bottom line is that they could get any answer they wanted. They were able to get the answer they wanted: meat is bad for you.
"When evaluated based on disease history at baseline, the association of total protein with mortality appeared more evident among those with a history of type 2 diabetes, cardiovascular disease, or cancer (n = 1094) compared with those without disease history (n = 1547) (P-interaction = 0.05 or 0.07, depending on the model). Intakes of fish, eggs, dairy, or plant protein sources were not associated with mortality."
"Conclusions: Higher ratio of animal to plant protein in diet and higher meat intake were associated with increased mortality risk. Higher total protein intake appeared to be associated with mortality mainly among those with a predisposing disease."
SY: Using ratios is often associated with sleight of hand and it is found here. The authors were able to get a slightly better protein effect by using a ratio of animal to plant protein. Of course, the good meat effect (you live longer) was overturned by various “adjustments.”
"This trial was registered at clinicaltrials.gov as NCT03221127. Am J Clin Nutr 2019;0:1–10."
SY: Registration is being required by many journals to help stop p-value hunting/hacking among other things. P-value hunting is asking a lot of questions and then writing a paper about a small p-value that almost surely came about due to chance. P-value hunting is also called p-hacking. This study is a good example of a continuing p-hunting/p-hacking science business model/multiple testing scam.
Finally, the nutrition/biochemistry science is quite simple. Your stomach and digestive system break down proteins into amino acids (plant or animal) and your body puts them back together into human proteins.
(This sounds pretty bad. Is there anything good about the study. Well, yes. And it happens to be hilarious)
SY: The only good thing about this study is that people in Finland paid for it.
