Sunday, September 19, 2021

Happy to be wrong (Covid update)

Earlier this semester, I thought we likely had out-of-control spread on campus from people with breakthrough cases that were going undetected.

It looks like I was wrong.

The University of Illinois decided not to do surveillance testing this fall semester, instead focusing on regular testing only of unvaccinated people and selective testing of vaccinated people if they were in a place with an outbreak. (I still think that's a mistake - see below). The campus also returned almost all of its students—including a record-size first-year class—and encouraged attendance at several mass gathering events including football games. I expected this combination to be disastrous. And, at first, it looked like I was right.

In the first weeks of the semester, we saw a substantial spike in cases. Despite daily testing numbers that were just a fraction of those from last year, we saw new cases at a rate comparable to the February surge and daily positivity rates as high as during parts of the fall 2020 surge.

Over the past week or so, though, the daily numbers of cases have dropped dramatically—we're currently down to an average of under 7 detected cases per day. What we don't know, because we're not testing everyone, is whether those 7/day constitute all of the cases or if we would detect many more than that if we were testing everyone. That is, would we still see only 7 cases/day if we were conducting 10,000 to 20,000 tests/day necessary for surveillance rather than the 2,000 to 5,000 per day we're currently conducting. Or would the number of cases scale proportionally with the number of tests. 

Aside: We wouldn't even need to test everyone to know—we could randomly select a few thousand vaccinated people to test each week and that would give us an estimate. As far as I know, were not doing that, so we remain blind to what's actually happening. I don't know why we're not doing that sort of testing (other than cost).

 Here are some possible reasons things might be going better than I had expected they would:

  • Maybe high vaccination rates coupled with masking requirements were enough to eliminate spread on campus.
  • Maybe spread from breakthrough cases to people who also are vaccinated is even less likely than thought, so undetected breakthrough cases aren't spreading Covid further (given high vaccination rates on campus).
  • Maybe those people who are most likely to engage in riskier behavior—the ones who were most responsible for the outbreaks in the fall/spring—have extra immunity because they both had covid already and have been vaccinated. So, even if they're taking risks, they aren't getting Covid and spreading it. And, everyone else is doing what they were all of last year, taking appropriate precautions to avoid getting Covid or giving it to others.
  • Maybe breakthrough cases acquired from another breakthrough case are more likely to be asymptomatic, in which case we could have spread on campus but it wouldn't be detected because those students wouldn't seek out a test. (worst case that could be detected via the random sampling strategy I noted above.).

Regardless of the reasons, it's great that case numbers are down, and I hope it means that spread on campus is under control. 

That said, the pandemic is far from over. Illinois Region 6, the set of counties that include Champaign, is hovering around 8% positivity (not great). We still have many county residents hospitalized, and we've seen increased deaths over the past month. Southern Illinois has zero open ICU beds for a 20-county region. Other Midwestern states (e.g., Ohio) are seeing a huge surge, and neighboring states are not taking the same precautions that Illinois is (no indoor mask mandates). 

For those reasons, I'm not thrilled with unrestricted attendance at sporting events (especially the indoor ones). If people who aren't vaccinated attend those events, including those traveling from outside the area, we could see spread from them to the campus or vice versa. I'd like to see UIUC require negative tests or proof of vaccination to attend campus events (just like the United Center will). 

It's a relief and great news that campus appears to be doing relatively well right now. I'm glad that my concerns about the fall semester appear to have been overly alarmist. It'd be wonderful to be proven wrong about that. But, that doesn't mean we shouldn't take precautions, especially when the campus and broader community interact.

Tuesday, August 31, 2021

Covid responses at Duke and UIUC

According to the Duke University dashboard, they had 349 positive cases among students last week. Duke has about 15000 students (about 2/3 are grad students) and a 98% vaccination rate. You might recall from last year that Duke implemented a robust testing process last year and had far fewer cases than the University of Illinois did. They appear to have continued their approach from last year of testing all of their students. Although their dashboard doesn't break down cases based on vaccination status, most of these positive cases are likely to be breakthrough cases. In other words, they are testing everyone so that they will have some idea of how bad things are on campus. That has led them to implement mitigation (according to Washington Post article): Indoor and outdoor masking, allowing faculty to shift to virtual teaching for two weeks, suspending indoor dining, limits on student activities. 

Duke has a higher vaccination rate than the University of Illinois, and when they tested everyone last week, they found 349 positive cases. When the University of Illinois tested primarily the unvaccinated students, we found 165 cases. Yet, unlike Duke, we don't know how many people are actually infected because we're testing only a small subset of our campus population.

Duke is doing surveillance testing to monitoring the covid situation on its campus and they're taking steps to detect cases among the entire campus population and to stop spread. The University of Illinois is not, even though we could be. We have made no effort to detect cases among the vast majority of students who are vaccinated even though breakthrough cases can spread Covid. And, if we don't know when someone is infected, we can't isolate them to break the chain of transmission. If I had to guess based on our vaccination rate and positive tests among the unvaccinated students, I'd estimate that we have 500+ cases on campus right now and that we've detected only about a third of them. And, those cases we haven't detected will be spreading Covid rapidly.

The University of Illinois has regularly touted our "success" last year in preventing community spread (although the data on that are debatable), so they recognize how important that is. Yet, they went ahead and encouraged a full-capacity crowd at a football game with no meaningful mitigations in place. Many people who attend football games come from the surrounding communities that have low vaccination rates and high levels of spread. If preventing community outbreaks is important to the university, maybe that wasn't such a good idea...

Yesterday, Awais Vaid of the Champaign-Urbana Public Health District (who has worked closely with the university SHIELD team) told the News-Gazette that we'll know in a week whether having 41,000 fans at the game will result in more Covid cases for Champaign County, and he encouraged fans who attended to consider getting tested. Perhaps the possibility that the game would be a super spreader event is something that the university should have raised and addressed before the game. Instead, the university offered free tickets to faculty and staff in order to fill seats (thanks for the invite, but I noped as far away from that one as possible). 

Unlike Duke, the University of Illinois appears to have adopted a "let's wait and see what happens" strategy, while simultaneously blinding ourselves so that we can't see clearly. We have the capability to test everyone, but the university has chosen not to. Hoping everything will go well isn't a strategy—Covid doesn't care what we hope will happen.

Monday, August 30, 2021

Covid numbers - August 30, 2021

During the first week of classes at the University of Illinois, campus testing identified 165 cases of Covid on a total of 22,296 tests.

For comparison, in the first week of the spring semester (Jan 25 - 31), we had a total of 128 cases with 63,208 tests. We're finding more cases than at the comparable point last spring despite testing far fewer people. Whether that's due to Delta or to reduced restrictions and more potential super spreader events (e.g., football games with unrestricted attendance, massive new student orientations with lots of yelling, open bars, etc.) is hard to know.

And, as noted in my post earlier today, we're missing the vast majority of breakthrough cases because we are not doing surveillance testing of vaccinated students. If the vaccines have 60% efficacy against infection and the exposure risks are comparable for vaccinated and unvaccinated students (an assumption that might not be justified), we could have up to 3x as many breakthrough cases that have gone undetected. That is, had we tested everyone on campus, we could well have found a much higher number of cases (perhaps hundreds). Any breakthrough cases that that we didn't detect (because we're not looking) potentially could infect others. Even if people with breakthrough cases do not spread covid as easily as unvaccinated people with Covid, that's still a lot infected people who come into close contact with many other people.

We don't definitive evidence of vaccine efficacy against infection in this population, and we lack information from the university about the total numbers of people being tested, the number of each subgroup of people on campus, or the numbers of vaccinated and unvaccinated people in each group being tested. Without that information, we can only rely on guestimates about efficacy as well as infection rates. But, even if we ignore the possibility of large numbers of undetected breakthrough cases, the first week numbers aren't great - they look worse than at the start of the spring semester.  If there are a sizable number of undetected breakthrough cases, as seems inevitable, we won't know how much spread we're seeing on campus and we won't be able to disrupt chains of transmission. I hope breakthrough cases aren't as infectious, but if they are even somewhat infectious, we're risking undetected exponential growth.

Covid at Illinois - 30 August 2021

During the 2020-2021 academic year, I posted regular updates about Covid at the University of Illinois. I tracked the number of positive cases on campus, estimated the percentage of students infected, and computed various metrics of the infection that were more meaningful than the ones provided by the university (e.g., rolling averages of cases, cumulative cases by semester, cases by week, etc.). Those analyses were necessary because the campus dashboard featured meaningless statistics like 7-day average positivity and the all-time number of tests, neither or which provided useful information to gauge the extent of outbreaks. The campus dashboard also gave no breakdown of the infection percentages, the numbers of students being tested each week, the compliance rate for testing, etc.

In addition to those daily updates, I also discussed many of the logistical problems that contributed to the infection of an estimated 20% of the undergrads on campus last year (4935 out of about 24000) as well as some of the misleading claims coming from the administration. In addition to the daily summaries, I posted a detailed summary of the fall semester issues as well as a more in-depth analysis of how delayed testing results likely contributed to the fall 2020 surge. 

This year, changes in testing policies (not testing everyone) mean that I won't be able to estimate the infection rates on campus, and it will be much harder to evaluate whether or not we're in the midst of a large outbreak. It'll be more guesswork than analysis. I'll try to do that guesswork and will post updates here on my blog. Occasionally I'll post analyses as well. But, there's not much benefit to the sort of daily tracking I did last semester because there won't be enough information available on the campus dashboard to do it. I fear that we're running blind this semester. Here's where things stand on campus and why we won't have the information we need.

Vaccine "requirement"

This summer, the campus announced that they would be requiring vaccines. That sounded fantastic, but it turned out that the implementation had no teeth and it was actually more of a "nudge" than a requirement. If you weren't vaccinated — and you didn't need to give any reason why not — you had to continue testing. 

Last week, Gov. Pritzger announced that all school and university faculty and staff in Illinois must be vaccinated. Shortly after that announcement, the university announced that they will now mandate vaccination for everyone (except those with a medical or religious exemption). I had thought that "requirement" and "mandate" are synonyms, but apparently they mean different things to our administration. I also have no idea how hard it is to get a religious exemption. In my view,  you should be required to present a holy text from your faith showing that vaccines are not permitted...). In any case, it's great that we finally have an actual vaccine requirement/mandate coming into effect. 

Changes to testing

Last year, the campus tested everyone twice weekly (allegedly - there was a lot of non-compliance). Over the summer, when cases were low and vaccines were available, the university developed their plan for the fall: They dramatically scaled back on testing by only requiring unvaccinated people to test. The number of testing sites was reduced to 4. The used vaccination as a get-out-of-testing incentive. Unfortunately, they didn't radically change their plans for Delta. They did increase the required testing frequency for unvaccinated undergraduates to every other day because modeling showed that without doing that, people would spread Delta before they knew they were positive. Delta becomes infectious faster than the earlier variants. 

However, by electing not to test vaccinated people on campus, we won't detect breakthrough cases (unless they are symptomatic and elect to get tested, by which point it's too late to stop spread). We also won't be able to break the chain of transmission by isolating infected students and quarantining or testing their contacts. We're going to be largely blind to the outbreaks on campus, and they could easily get out of control. The campus has the capacity to test everyone, but they chose not to use it. It's a potentially disastrous decision. Here's why.

The consequences of testing only the unvaccinated

There are far more vaccinated students than unvaccinated ones on campus. About 88% of students are vaccinated, and that soon will be closer to 100% now that we have an actual mandate. If we estimate vaccine efficacy against infection to be approximately 60%, that means there will be nearly 3x as many breakthrough cases than there are cases among the unvaccinated!

To see why, imagine that we test all of the unvaccinated students on campus on a single day and that 1% of them test positive. If we assume that the unvaccinated and vaccinated students had the same exposure to covid via close contacts, that means 0.4% of the vaccinated students would have tested positive had we tested them too (60% efficacy against infection). With about 35,000 undergrads on campus, about 30,800 of them are currently vaccinated (88%) and the remaining 4200 are unvaccinated. If 1% of the 4200 tested positive, that would yield 42 cases. If 0.4% of 30,800 vaccinated students tested positive, that'd be 123 cases. So, we should expect about 3x as many cases among the vaccinated population as among the unvaccinated population. There are a lot more vaccinated than unvaccinated students (which is great, of course). 

In the extreme, if 100% of students were vaccinated, and we're only testing unvaccinated students, we could have a massive outbreak on campus via breakthrough cases and we'd know nothing about it (until people start showing symptoms and seek testing or treatment. But by the time they're showing symptoms, they've been infectious for a while).

Last year, the University was (somewhat) able to contain outbreaks by quickly identifying positive cases—often before they were infectious—and then isolating them. We saw the surge, implemented mitigations, and brought numbers down at least somewhat. This semester, up to 3/4 of the initial cases could go undetected. Unlike last year, we'll be blind to spread from those cases until it's too late to stop it. 

Vaccines are highly effective at preventing severe illness, hospitalization, and death. And, students might be at lower risk than older people as well. But, the vaccines aren't airtight protection, and even with the campus's indoor mask requirement (again, that's great!), there are many opportunities to spread Covid both on and off campus (see the massive unmasked crowds at the football game, at bars, etc.). If the University allows unmitigated spread on campus, it likely will reach the broader community this semester. 

As was the case last year, the University of Illinois is in much better shape than many other universities. First, Illinois is in better shape than many other states. We have a governor who is implementing good public health policies (mandating vaccines for all schools and requiring masking indoors) and a local health district that is proactive and communicates well. We have a campus mask mandate and are implementing a vaccine mandate. Due to the amazing test that researchers developed here and the testing capacity that was built up over the past year, we have the potential to know a lot more about how bad the outbreaks on campus are and to end them before they spread too far. We have the capacity to limit the spread and consequences of Delta, but we're not doing everything we need to be doing.

Even if the University prefers not to test everyone due to costly logistics, there are approaches we could take that would help at least determine how much spread we have on campus. For example, the campus could test randomly selected, somewhat large subsets of vaccinated students each week. If those tests revealed a high rate of positives, we might need to adopt more extensive testing and restrictions. The dashboard could also separate results for vaccinated and unvaccinated people. That would show whether we're seeing a spike in vaccinated people seeking tests (a sign that we have spread). Without some form of systematic surveillance testing, though, we won't know how bad things are until people start getting sick. 

Thursday, December 10, 2020

 Analyses of Covid trends
at the University of Illinois

Since the university reopened in Fall 2020, I have closely monitored the numbers of Covid cases on campus and the reasons for spread. My analyses are intended to be a reliable source of publicly available information. The University has not always been transparent about the situation on campus, so I have provided additional estimates of the information that was not available. I also provide daily updates about hospitalizations and spread in the community surrounding campus.

Here are links to the various summaries and write-ups I've been tweeting about and posting publicly on Facebook.

  • I post almost daily updates at http://www.dansimons.com/covid.html. Monitor that page for the most recent daily case numbers, 7-day averages, hospitalizations/deaths in the community, etc. The page includes daily updated graphs and explanations of them.
  • My summary of the entire fall semester is available at http://www.dansimons.com/Covid/fall2020summary.html
    • That summary includes a discussion of the modeling that was used to justify reopening as well as how it was both right and badly wrong.
    • It also discusses how the University placed blame for the initial surge in cases on a handful of non-compliant students without taking any responsibility for logistical failures and inaccurate initial assumptions about compliance.
  • My essay on how to interpret positivity ratios correctly, including a discussion of why positivity is the wrong metric to use when you are testing everyone in your population of interest repeatedly (as was the case at the University of Illinois this fall) is at http://www.dansimons.com/Covid/positivity.html


I have turned off comments for this post. If you would like to comment, please do so in response to my regular tweets (@profsimons) on this topic. This post is a placeholder so that the links are findable.

Monday, May 15, 2017

A new journal at APS: AMPPS

I am thrilled to announce the launch of the newest journal published by the Association for Psychological Science (APS):


I will been named the founding editor, and I have assembled what I think will be a terrific editorial team. We are now taking submissions, with the first issue slated for publication in early 2018.

The journal's primary mission is to communicate advances in research methods and practices to the broad membership of APS and beyond. We hope to bridge subfields of psychology, bringing advances from within an area to the rest of the field. 

AMPPS will publish articles on research practices, tutorials to help researchers develop new research skills, and empirical papers that illustrate innovative methodological approaches. It will be the new home for the Registered Replication Reports that previously were published at the APS journal Perspectives on Psychological Science (my co-editors for those papers, Alex Holcombe and Jennifer Tackett, have joined the new editorial team). AMPPS will also publish registered reports, adversarial collaborations, multi-lab consortium studies, simulations and re-analyses of existing data, meta-science papers, commentaries, and much more.

You can read more about the journal's mission, see the editorial team, and read the submission guidelines on the journal's website.

APS also issued a press release about the new journal last week.

Please help spread the word! If you tweet or post about it, please use the hashtag #APS_AMPPS

Thursday, December 10, 2015

Visual effort and inattentional deafness

Visual Effort and Inattentional Deafness

Earlier this week I was asked for my thoughts on a new Journal of Neuroscience paper: 
Molly, K., Griffiths, T. D., Chait, M., & Lavie, N. (2015). Inattentional deafness: Visual load leads to time-specific suppression of auditory evoked responses. Journal of Neuroscience, 35, 16046-16054.doi: 10.1523/JNEUROSCI.2931-15.2015
In part due to a widely circulated press release, the paper has garnered a ton of media coverage, with headlines like:
Focusing On A Task May Leave You Temporarily Deaf: Study

Did You Know Watching Something Makes You Temporarily Deaf?

Study Explains How Screen Time Causes 'Inattentional Deafness'

The main contribution of the paper was a link between activation in auditory cortex and the behavioral finding of reduced detection of a sound (a brief tone) when performing a more difficult visual task. 

This brain-behavior link, not the behavioral result, is the new contribution from this paper. Yet, almost all of the media coverage has focused on the behavioral result which isn't particularly novel. That's unsurprising given that most of the stories just followed the lede of the press release, which was titled:
"Why focusing on a visual task will make us deaf to our surroundings: Concentrating attention on a visual task can render you momentarily 'deaf' to sounds at normal levels, reports a new UCL study funded by the Wellcome Trust"

Here are a few points about this paper that have largely been lost or ignored in the media frenzy (and the press release):

1. The study did not show that people were "deaf to their surroundings." In the study (Experiment 2), people performed an easy or hard visual task while also trying to detect a tone that occurred on half of the trials. When performing the easy visual task, they reported the tone accurately on 92% of the trials. When performing the harder visual task, they reported it accurately on 88% of trials. The key behavioral effect was a 4% reduction in accuracy on the secondary, auditory task when the primary visual task was harder.  In other words, people correctly reported the tone on the vast majority of trials even with the hard visual task. That's not deafness. It's excellent performance of a secondary task with just a slight reduction when the primary task is harder. 

Aside: much of that small effect on accuracy could be due to a difference in response bias between the conditions (Beta of 3.2 compared to 1.3, a difference reported as p = 0.07 with an underpowered study of only 11 subjects).

2. The behavioral effect of visual load on auditory performance is not original to this paper. In fact, it has been reported by the same lab.

3. A number of other studies have demonstrated costs to detection in one sensory modality when focusing attention on another modality. This paper is not the first to show such a cross-modal effect. See, for example, hereherehereherehere (none of which were cited in the paper). Many other studies have shown that increasing primary task difficulty decreases secondary task performance. Again, the behavioral result touted in the media is not new, something the press release acknowledges in passing.

4. The study doesn't actually involve inattentional deafness; the term is misused. Inattentional deafness or blindness refers to a failure to notice an otherwise obvious but unexpected stimulus when focusing attention on something else. The "unexpected" part is key to ensuring that the critical stimulus actually is unattended (the justification for claiming the failure is due to inattention); people can't allocate attention to something that they don't know will be there. 

In this study, tone detection was a secondary task. People were asked to focus mostly on the visual task, but they also were asked to report whether or not a tone occurred. In other words, people were actively trying to detect the tone and they knew it would occur. That's not inattentional deafness. It's just a reduction in detection for an attended stimulus when a primary task is more demanding. And, as I noted above, it's not really a demonstration of deafness either given participants were really good at detecting the tone in both conditions (they were just slightly worse when performing a harder visual task). 

Note that the same lab previously published an paper that actually did show an effect of visual load on inattentional deafness.


ConclusionThere's nothing fundamentally wrong with this paper, at least that I can see (I'm not an expert on neuroimaging, though). The link between the behavioral results and brain imaging results is potentially interesting. I would have preferred a larger sample size and ideally measuring the link between brain and behavior in the same participants performing tasks with the same demands, but those issues aren't show stoppers. I can see why it is of interest to specialists (like me). That said, I'm not sure that it makes a contribution of broad interest to the public, and the novelty and importance of the behavioral result has been overplayed.