The Sturgis Effect

Let's start with the people attending the annual motorcycle event because, from a health point of view, it is easy: no significant social distancing or use of masks. So, shame on them. They have rights, but they also have responsibilities. Maybe, we should have a Bill of Responsibilities. As for protests, the same rules of social distancing and masks apply. There is one caveat here, many of the protesters did wear masks, and the largest gatherings I could identify were in the low thousands; Sturgis was estimated at 450,000.

Now, let's get into the Sturgis "study." For people who follow my writing (thank you, family), I love studies that bring together disparate data. Using the geolocation feature of ubiquitous smartphones, they could reasonably approximate who came to Sturgis, how long they stayed, and when and where they returned home. They could also use the same phone data to geolocate the residents of Sturgis. 

They made use of nationally reported, reputable COVID-19 cases, from Johns Hopkins' COVID-19 Dashboard, and other sources, broken out by census tracts. They did not consider deaths or hospitalizations, simply positive tests. With all of this real data in hand, they began the computations.

First, using the daily COVID data, they constructed a model of how these cases increased during the period from June 6^th to September 2^nd, incorporating the two-week run-up to Sturgis [1] and several weeks afterward. The difficulty here is that there is no control to compare with the observations. The researchers created a "synthetic" control group, a standard technique when you can't for logistical or ethical reasons have a control group. To be fair, they took great pains here to identify towns with similar populations outside the area of Sturgis or the geography of the visitors; matching population densities, the degree of "stay at home" behavior, and the number of COVID cases or mortality for periods up to 28 days before the Sturgis rally. The difference in actual cases between the Sturgis geographies and those predicted by the synthesized controls represent the excess cases caused by the gathering. This is not an observational study; it is a computer simulation. At this time, there are about 260 actual COVID cases traced to Sturgis, with one death. Keep those numbers in mind as we consider their findings. 

Based on the smartphone geolocation data, Sturgis witness a large increase in population during the gathering. 

• 90% of those phones were from out of state, 20% from surrounding "border" states, the remaining 70% from across the country. – Sturgis had a sizeable geographic impact.
• Foot traffic, as measured by cellphones, increased in restaurants, bars, retail establishments, like grocery stores, pharmacies, and liquor stores, as well as in sports arenas, amusement parks, casinos, hotels, and campgrounds.
• Even the locals got out a bit more, their median "at-home" time dropping from 10.1 hours to 8.8 hours daily. The researchers took this to mean that these individuals were participating in the Rally, but it is equally plausible that they were working overtime to meet the needs of the large influx of visitors. In any case, they were exposed to more people.

Based on the difference between the synthetic control and the actual COVID-19 data, the researchers report

• A 6 to 7 cases/1000 increase in Meade County where Sturgis is located, equivalent to 177 to 195 excess cases. As you multiply these figures to the state level, cases rise to 3,200 or more. According to the Hopkins Dashboard, the cumulative cases in South Dakota for the period were 3729, so if their simulation is accurate, nearly all the new cases in South Dakota are attributed to Sturgis. Is that likely?
• Based on the number of individuals traveling to Sturgis from outside South Dakota, the researcher defined four "inflows" to Sturgis. The largest resulted in a 10% increase in COVID cases in their homes,  the second largest a 12% increase, the third-largest a 7% increase, and in the smallest, no change. 
• Based on these calculations, the researchers felt this was a superspreader event. 
• In those counties that contributed a large number of visitors to Sturgis, there was a range of increases; some of those counties have only a 2% increase. The researchers attribute this to more stringent social distancing and masking in those communities. That well may be true, but again, ask yourself how many of these participants who wore no masks or socially distanced in Sturgis are going to be compliant at home? 

Finally, the researchers estimate cost. Using their calculated 10% increases outside South Dakota as well as the increases within the state. The Sturgis Motorcycle Rally accounted for

"266,796 or 19 percent of 1.4 million new cases of COVID-19 in the United States between August 2nd, 2020 and September 2nd, 2020."

From here, cost becomes a multiplication problem, citing a statistical cost estimate by other researchers [2] of $46,000/case, they arrive at the much mentioned $12.2 billion. But let's take a deeper dive into that cost per case. The table summarizes the data points.

The prevalence of COVID-19 outcomes are CDC estimations for research. The assigned dollar are numbers the Department of Transportation utilizes to "values a statistical life" – a value based on an auto accident. Those values are the judgment of 4 expert physicians on how increasing severity of injury would impact the quality of one's life, short and long term. While there is a growing population of long-term problems associated with COVID-19, the long haulers, I am not aware of any measurements of how or how many are in this category. (Dr. Francis Collins take on the issue can be found here.)

As these researchers readily point out, they do not precisely match up to possible valuations for COVID-19 and suggest this might be an upper bound. CMS currently reports that the average cost of hospitalization for one of their beneficiaries is presently $25,000, a fifth of the research valuation. The difference reflects the long-term, day-to-day consequences, the loss of quality to one's life, of that illness over a lifetime – making that number age-dependent. Again, the researchers recognize that, and suggest that when corrected for age, these valuations may drop by 30%. After all, those over 65, about 30% or more of those hospitalized for COVID-19, have shorter life expectancies and, therefore, less "economic-value."

To arrive at their final figures, they need only multiply by the cases in each category. Rather than use the current COVID-19 case volume of 4.7 million, they believe, as do others, that the cases are undercounted. So, the number of cases they use are eleven-fold greater. At the end of all that math, and all those estimates, a seemingly concrete number appears $46,000/case. You can decide for yourself how solid those headline-grabbing financials of a $12.2 billion motorcycle rally are.

Every paper provides a learning opportunity; this one is no exception. Here are a few take-aways.

• Computer simulations are helpful but are not "studies" of health outcomes as much as predictive exercises.
• Synthetic controls are a way to identify differences in "treatment" when it is not logistically or ethically possible. Despite the researchers' efforts, these "controls" are estimates and increase the uncertainty of the reported outcomes.
• When values like case counts are divided by larger and larger numbers, they get smaller. Witness how the increase in 2 cases/1000 in Sturgis proper, was reduced to 1.5 cases/1000 in the bordering counties, and 1.1 cases/1000 for South Dakota
• Alternatively, when case counts are multiplied by larger and larger numbers, they get bigger. The estimated 177 excess cases in Sturgis become 266,000 when you consider all of the population that might have been exposed to those going to the Sturgis rally.
• Our behavior has direct consequences, in hospitalizations, and indirect ones, in the cost to our overall economy and the quality of our lives.

I loved the research; I thought it was great how they tried to match up disparate datasets and the care they took in creating their controls. Do I agree with all of their assumptions? No. Do I think the use of language like "dose" is warranted? No. Does it change my point of view? No. I still believe that social distancing and wearing masks are the cheapest public health measure we can currently take. Here is the best line from this paper, based upon their $12.2 billion estimates,  

"This is enough to have paid each of the estimated 462,182 rally attendees $26,553.64 not to attend." 

For my conservative friends, wearing a mask and social distancing will reduce hospitalizations, reduce costs, and ultimately your taxes. For my liberal friends, wearing a mask and social distancing will reduce hospitalizations and allow us to redirect those resources to other critical social and economic needs.  

[1] The Sturgis Motorcycle Rally is in its 80^th year; it ran from August 7^th to the 16^th, a ten-day period, and attracted approximately 450,000 visitors to the town of Sturgis, population 7,000.

[2] The Forgotten Numbers: A Closer Look at COVID-19 Non-Fatal Valuations. Social Science Research Network 3680348.