The American Council on Science and Health has long been concerned about the decision-making process at organizations like EPA and especially National Institute of Environmental Health Sciences, where baffling secret meetings have led to unreal levels of weight for solitary statistical analyses that defy the pattern of available empirical data. Epidemiology has become a secret trump card over science, while risk experts, toxicologists and other experts who care about public health have been routinely ignored.
Yet the government has been able to hide this process from taxpayers, and groups like EPA have used such "secret science" to legislate without being accountable to the public the way Congress and the President are accountable.
The public won't have confidence in EPA or NIEHS as long as secret meetings and a lack of transparency remain the norm. Consumers have already been jaded by "sue and settle" agreements, where decisions for politically favored groups like Center for Biological Diversity are the norm, such as when 100 Mississippi frogs placed on an endangered list in a sue-and-settle deal must then be transplanted to Louisiana, where not one has been sighted in my entire lifetime, and landowners there must foot the $30 million bill. (When we did a Freedom of Information Act request to get the justification for that decision over where the frog already lives, FWS told us all they had was raw numerical data - not even any headers to make it understandable, and no one at FWS thought it odd their consultant, Industrial Economics, Inc., whose only client is FWS, would give them data no one at FWS could read.)
EPA asked for comment on the secret science issue and here is the statement I provided after conversations with our Board of Scientific Advisors.
***
Environmental Protection Agency
Office of the Science Advisor
1200 Pennsylvania Avenue NW
Washington, DC, 20460
RE: Docket ID No. EPA–HQ–OA–2018–0259; 40 CFR Part 30, [EPA–HQ–OA–2018–0259; FRL–9977–40– ORD], RIN 2080–AA14, Strengthening Transparency in Regulatory Science
Dear Administrator Pruitt,
Your announcement that EPA will no longer use "secret science", a colloquial term for science in which the underlying data cannot be procured, is encouraging. The entire American Council on Science and Health and its Board of Scientific Advisors have long noted that how science is considered acceptable and useful for EPA rulemaking is a vital issue.
But reform has inherent challenges that must be addressed, and those challenges fall into two areas of concern that can become pitfalls. Chemicals designed for commercial uses by the private sector often include toxicity studies that may not be publishable, while most journals don't include all data.
In the past, peer-reviewed journal publication has been considered authoritative but that has inherent weakness. If enough tests are run, a positive finding occurs due to chance in one out of every 20 studies (1) and with 61 flips of a coin you can get five heads or tails in a row 7 times out of 10, a p-value of .05.(2) Knowing the potential for error and even misuse, replication remains important but we recognize that is not always possible. A safety valve for that is a higher level of scrutiny when it is not possible. Studies that can't be replicated should at least make sense within the pattern of available data, which in the case of compounds under EPA scrutiny will often include hundreds of other studies done according to federal guidelines.
There are occasions where replication is not possible and new claims are outliers from many other studies, and that is where the "secret science" problem has been apparent in the past. Small particulate matter (PM2.5) was declared to have no safe level by a previous Administrator even though we showed that not a single acute death was caused by PM2.5 during the entire history of EPA.(3) Problems of the past aside, in cases where studies are not replicated or are outliers from the consensus, they should absolutely be used if EPA risk scientists, without breaking confidentiality, can obtain the additional information needed in order to conduct their own analysis. There are numerous instances where that has not happened, and the cases where it did not often look more like political determinations than scientific ones. Even knowing that has happened, former EPA Administrator Gina McCarthy and former acting Assistant Administrator Janet McCabe (4) have argued that EPA should continue to include such secret science even if the creators will not submit underlying data.
Such thinking in the past has led to capricious and arbitrary decision-making by EPA and you are right to put an end to it.
Yet there is also a ready solution to those more complex cases and that solution is implementing a risk assessment perspective (5) from risk scientists charged with protecting public health who have too often been missing from decision-making.
ACSH has argued since 1978 that the judgment over which epidemiology and/or toxicology data to use for risk or safety assessment should always include risk scientists. A close reading of your proposal finds this to be entirely consistent with your decision. We agree that:
1) The public’s interest is best served when science is replicable and consistent with other information.
2) On occasions when studies cannot be replicated or when such studies are not consistent with other information, use of those studies depends on having access to the underlying data for independent analysis. When the underlying data are not provided, it is difficult to use this study to make a credible risk judgment, much less national rulemaking, as you know.
Your proposal is a solid risk assessment approach.
The public is often worried about chemical exposure, as we all are when such exposure exceeds a safety level. But the public’s interest is best served by trusting in EPA experts dedicated to public health protection and that cannot happen if groups are allowed to withhold scientific data from independent analysis and still find ways to argue how their "secret science" should be held in higher regard than other studies which match standards for inclusion.
References:
(1) https://www.nas.org/images/documents/NAS_irreproducibilityReport.pdf.
(2) https://geneticliteracyproject.org/2017/12/13/viewpoint-many-scientific-...
(3) https://www.acsh.org/news/2017/01/04/air-pollution-and-deaths-associatio...
(4) https://www.nytimes.com/2018/03/26/opinion/pruitt-attack-science-epa.html.
(5) https://www.washingtonexaminer.com/opinion/op-eds/the-epas-new-secret-sc...
