Censored state data scientist launches Florida COVID-19 dashboard independently

Jones was the original architect of the dashboard that was hailed by Dr. Birx from the White House Coronavirus Task Force as one of the best in the country.[1] She made a different kind of news in May when the Florida Department of Health removed her from her post[2] after reprimanding her for making unauthorized public statements.[3]

Her supervisor has reported that the incident that led to her removal was specifically about the “EventDate” data, the self-reported date that a patient remembers first developing symptoms. This kind of information can be useful for the kind of timing analysis that researchers like Steve McConnell are using to provide open-readiness reports at the state level.

Jones released a public statement saying that the issue was that she refused "to manually change data in a misleading way to drum up public support of resuming pre-pandemic level of activities without a science-based assessment of each county’s respective risk or readiness to do so.”[4]

If refusing to mislead the public during a health crisis is insubordination then I will wear that badge with honor.

Is Florida ready to reopen?

The COVID-19 Task Force at the State of Florida announced in their Plan for Florida’s Recovery that Florida would follow the White House Guidelines for Opening up America Again as a baseline for reopening. That plan requires satisfying three benchmarks before reopening:[5]

• A downward trajectory of influenza-like illnesses (ILI) and a downward trajectory of COVID-19-like illnesses (fever, cough, shortness of breath)
• A downward trajectory of documented COVID-19 cases or a downward trajectory of positive tests as a percent of total tests (at or increasing volume of tests)
• The capability to treat all patients without triggering surge capacity and robust testing program in place for at-risk healthcare workers, including emerging antibody testing

Florida is in the final stages of reopening, including in South Florida. But do we meet those criteria? Shouldn’t we be able to monitor that?

Now you can see that information

Jones has used her skills to launch an independent dashboard using the same public data that she used in the Florida Department of Health dashboard. Her dashboard is called Florida’s Community Coronavirus Dashboard.

One of the key differences is that her dashboard includes the county-by-county “report cards” for monitoring whether each county satisfies the reopening criteria. It shows that many counties do not meet the State of Florida's criteria for reopening. For example, most of Florida is in Phase 2 of reopening, and Miami-Dade County is in Phase 1.[6] Orange county went to Phase 2 on Friday, June 5, and theme parks have reopened. But Jones’ dashboard includes an assessment from June 5 showing that Orange County does not meet the criteria:

Report Card for Orange County

Last report date: 6/5/2020

According to data published by the Florida Department of Health, Orange County does NOT meet the criteria for entering the next phase of reopening.

Have the following criteria decreased for a minimum of two reporting weeks?

• ER Visits for Influenza-like Illness: NO
• ER Visits for COVID-like Illness: NO
• New cases by date: NO

Information about Orange in the last one reporting week (5/30/2020-6/5/2020):

During the most recent reporting week in Orange County, ER visits for Influenza-like illness decreased by 4% and visits for COVID-like illness decreased by 35%. New cases increased by 46%.

Criteria data is measured and determined based on the trend over the last two reporting weeks, excluding the current week, summarized from Sunday-Saturday. The last report date was issued 6/5/2020. To review the state's reopening criteria, click here.

Whether or not a county meets the criteria is based on the previous two-week trend, so data from the last reporting week may show a decrease in cases or ER visits but the county still may not qualify for reopening if those figures increased during the previous reporting week. Both of the two previous weeks must show a consistent downward trend, and those that increase one week or decrease the next, or for counties where there was no change, would not meet the specified criteria outlined by state.

Miami-Dade County also does not meet the reopening criteria:

Report Card for Dade County

Last report date: 6/5/2020

According to data published by the Florida Department of Health, Dade County does NOT meet the criteria for entering the next phase of reopening.

Have the following criteria decreased for a minimum of two reporting weeks?

• ER Visits for Influenza-like Illness: NO
• ER Visits for COVID-like Illness: YES
• New cases by date: NO

Information about Dade in the last one reporting week (5/30/2020-6/5/2020):

During the most recent reporting week in Dade County, ER visits for Influenza-like illness increased by 6% and visits for COVID-like illness decreased by 12%. New cases increased by 29%.

Criteria data is measured and determined based on the trend over the last two reporting weeks, excluding the current week, summarized from Sunday-Saturday. The last report date was issued 6/5/2020. To review the state's reopening criteria, click here.

Whether or not a county meets the criteria is based on the previous two-week trend, so data from the last reporting week may show a decrease in cases or ER visits but the county still may not qualify for reopening if those figures increased during the previous reporting week. Both of the two previous weeks must show a consistent downward trend, and those that increase one week or decrease the next, or for counties where there was no change, would not meet the specified criteria outlined by state.

Other differences

The new, independent dashboard also counts coronavirus deaths differently from the official Florida dashboard. The number of deaths is higher on the new dashboard because it includes deaths of non-residents who were infected in Florida. Including deaths of visitors could help to address a serious issue in tourist areas like Miami Beach.

Consider an extreme example like the Las Vegas Strip: Very few people live on the Strip. A significant number of people who will be infected in Las Vegas as it reopens will not show up in the coronavirus data for Clark County, Nevada. Their positive tests, hospitalizations, and deaths will be recorded in other places like California and Arizona instead of in Nevada. The absence of those cases could lead to statistics that give a false sense of security by under-reporting the impact of reopening. Paying attention to deaths resulting from infections in Florida’s tourist areas will be important for assessing the true situation places like Miami Beach and other popular destinations.

The number of cases is also higher on the new dashboard because it includes positive antibody tests, not just positive tests for the live virus. At least one doctor has cautioned against including these extra tests because of the false-positive rates of many of the antibody tests.[8]

The false-positive rates are a serious concern because the errors resulting from a test that is "95% accurate" can be surprisingly significant.

Including antibody tests is a problem

A surprisingly low false-positive rate can have a significant impact on conclusions about overall infections because of the base rate fallacy.[9] In an example from Wikipedia:

Imagine running an infectious disease test on a population A of 1000 people, in which 40% are infected. The test has a false positive rate of 5%, and no false negatives. 400 people would receive a true positive, and 30 people would receive a false positive. A person receiving a positive test could be over 93% confident that it correctly indicates infection.

Now consider the same test applied to population B, in which only 2% is infected. 20 people would receive a true positive. 49 people would receive a false positive.

In population B, only 20 of the 69 total people with a positive test result are actually infected. So, the probability of actually being infected after testing positive is only 29% for a test that otherwise appears to be “95% accurate”.

The base rate for coronavirus infections is currently closer to the range in population B than it is in population A. Please remember this when interpreting coronavirus data that includes antibody tests in the number of cases.

Thank you, Rebekah Jones

The push from the governor’s office to reopen Florida counties that don’t meet the state’s reopening benchmarks is an obvious political problem for the governor's office. It wouldn’t be possible for the Florida Department of Health to provide a dashboard with the level of transparency that the White House Coronavirus Task Force praised if the dashboard were to show that Florida isn’t ready to reopen. Thanks to Rebekah Jones and her independent dashboard, we now have that level of transparency again.

References

1. 1. Downey R. Dr. Deborah Birx highlights Florida Department of Health COVID-19 website. Florida Politics; 20-April-2020 link (Accessed 2020-06-12 09:30)
2. 2. Sassoon AM. Coronavirus: As Florida re-opens, COVID-19 data chief gets sidelined and researchers cry foul. Florida Today; 18-May-2020 link (Accessed 2020-06-12 09:30)
3. 3. State Records Show Rebekah Jones Fired For Violating Health Department Policy On Public Remarks. CBSMiami; 24-May-2020 link (Accessed 2020-06-12 09:45)
4. 4. Jones R. Response to media: #Insubordinate #Scientists. Rebekah Jones; 22-May-2020 link (Accessed 2020-06-12 09:45)
5. 5. Plan for Florida's Recovery. The State of Florida; 28-April-2020 link (Accessed 2020-06-12 10:15)
6. 6. Florida Gov. Announces Phase Two of Reopening; Miami-Dade, Broward Excluded. NBC Miami; 3-June-2020 link (Accessed 2020-06-12 10:15)
7. 7. Jones R. Florida's Community Coronavirus Dashboard. link
8. 8. Fired scientist Rebekah Jones builds coronavirus dashboard to rival Florida’s. The Palm Beach Post; 11-June-2020 link (Accessed 2020-06-12 10:30)
9. 9. Base rate fallacy. Wikipedia; 4-June-2020 link (Accessed 2020-06-12 10:45)