This article is the collaborative work of Warren Kindzierski, Stan Young, and John Dunn
ACSH has written previously on this topic.
Our recently published study exposes some of these falsehoods and hidden biases, creating a lack of support for two critical elements in the media narrative; a failure to show that gas stove emissions are harmful and that they cause childhood asthma.
Gas stoves, NO2, and asthma
NO2 (a combustion product of gas stove cooking), gas stoves, and asthma are a heavily researched topic—a PubMed search from 1990 to now returns 1,430 records, 44 articles annually.
Allergens are molecules that stimulate allergic reactions. Allergens are the fundamental factor in asthma morbidity since asthma is an allergic disease. Carbon-based molecules – e.g., pollen and plant parts; biological fragments shed from furry animals, rodents, cockroaches, dust mites; and fungal detritus all have chemical/physical characteristics for allergens. These molecules can cause allergenic asthma reactions.
NO2 molecules are not carbon-based, and they cannot, by themselves, trigger allergic reactions - they are not allergens. A recent, official American Thoracic Society (ATS) report notes the ambiguous role that NO2 has on childhood asthma, stating: “It is unclear whether direct effects of NO2 … explain the causal link with asthma”.
The ATS report is silent on the biological plausibility of a NO2−childhood asthma link. Scientists in other settings have suggested but not demonstrated that NO2 acts as a trigger of chronic inflammation sensitizing the lung to allergens, or that it combines in the atmosphere with other chemicals to create allergens.
But epidemiology studies in the literature are not founded on proven biological plausibility of NO2 causing childhood asthma but on a statistical association. Moreover, epidemiology studies cannot pretend to be toxicological studies. Epidemiology studies do not measure scientifically based toxicological endpoints that, in the case of asthma, would be an allergic/asthmatic link to an exposure being studied.
An epidemiologic look
Meta-analysis is a procedure for combining test statistics from individual epidemiology studies that address a common research claim. For example… is childhood asthma associated with indoor NO2 from gas stove cooking? The important questions for honest scientists are whether the association is robust and consistent with scientific (toxicological) evidence to explain the association. We used search space analysis and p-value plotting to evaluate the reliability of a gas stove/NO2−childhood asthma causal claim made in a 2013 meta-analysis.
Search space analysis involves estimating the number of statistical hypotheses tested in an epidemiology study to assess the potential for multiple testing bias. Multiple testing occurs when researchers use their data set to test many statistical questions (hypotheses)—for example, trying multiple combinations of outcomes, predictors, subgroups, statistical cause−effect models, or confounders.
Many tests provide opportunities to find associations that are the product of random variations in population studies; they are poorly reliable – increasing the likelihood of making a type I (false positive) error to near certainty.
P-value plotting involves using a graph to visually inspect the characteristics of a set of test statistics addressing the same research claim. A p-value plot examines whether p-values support no association (randomness) or a statistically significant association.
Test statistics – relative risks (or odds ratios) and confidence intervals – are converted to p-values plotted according to rank (smallest to largest values) in a similarly scaled plot. If p-values follow an approximate 45-degree line in the plot, the set of p-values supports randomness (no effect).
If p-values approximately follow a line with a flat/shallow slope, where most (the majority) of p-values are small (< 0.05), then the set of p-values provides evidence for a real association.
Search space analysis
Our study counted the number of statistical hypotheses tested in 14 of 27 epidemiology studies used for meta-analysis of gas stove/NO2−childhood asthma (and wheeze) associations. Summary statistics for the 14 studies are shown to the right.
The median number of hypotheses tested in the 14 epidemiology studies was 15,360. A conventional threshold for statistical significance in most science disciplines is a p-value < 0.05. Note that a false positive result can occur by chance 5% of the time. The expected number of false positive (chance) findings in an epidemiology study in the meta-analysis based on the median count is = 0.05 × 15,360 (768).
How does one tell if a test statistic drawn from the epidemiology studies into the meta-analysis is a real or a false positive (chance) finding? Answer – you can’t tell! The research has not provided strong evidence for their claim. Moreover, a real finding establishes association, not causation.
p-value plotting
A p-value plot for 13 test statistics used for meta-analysis of the gas stove−childhood asthma association is shown below. The p-values are aligned approximately 45 degrees. The plot shows evidence of randomness (i.e., no association between gas stove/NO2 and childhood asthma).
The p-value plot is consistent with a data set supporting randomness (i.e., unproven harms for gas stove cooking).
Research on NO2 has not established that this combustion gas, composed of three non-carbonaceous atoms, is a carbonaceous allergen. As such, a biologically plausible explanation of a causal link with childhood asthma is tentative at best.
Questionable statistical methods (excessive hypotheses testing) appear routine in NO2−childhood asthma epidemiology studies. These studies are compromised with unknown false-positive results. This may be, in fact, a characteristic of the NO2−childhood asthma epidemiology literature.
A p-value plot for a gas stove/NO2−childhood asthma association that we constructed from a set of test statistics in the meta-analysis support randomness – this is not consistent with proof that gas stoves are associated with or cause asthma or respiratory harm to children.
The published epidemiology studies we evaluated, which are supposed to be reliable, were not. They carry hidden biases into the meta-analysis and do not provide convincing evidence for public health policymaking on gas stove harm to children. The public can consider the gas stove−childhood asthma claim unproven. It cannot justify a mandate, prohibition, or any other punitive regulatory policy.
