My Beliefs About COVID Were Wrong
It's important to be confident in your own judgment when it's warranted
Lots of people I respect are very good at predicting things.
For a start, look at the records of Scott Alexander or Zvi. The former publishes a long list of predictions around once a year and reports on the results after the year is up. The latter has spent the better part of the time since COVID’s discovery regularly predicting the upcoming batch of statistics. Although Scott’s most recent set was among his worst performances on record, his accuracy graph is still quite impressive:
He was overconfident this past year—more events he predicted didn’t happen than he thought—which mostly stemmed from his belief that COVID lockdowns would be much shorter than they turned out to be. But he is still amazing at having a developed intuition about near-future events.1 Today I want to focus on COVID-related beliefs. Even as I write, around a quarter of the US population is fully vaccinated and we proceed to give around 4 million doses daily, but there’s nowhere in Texas I care to go that doesn’t still have a mask mandate or social distancing rules. Scott definitely got this one wrong.
So did I. I was on the phone with a friend last year, a couple of days after lockdowns began, and I recall telling him I thought we would be out of it by June. He told me there was little chance of that. My parents told me the same thing, particularly my mom—in her view there was zero chance we would be out of it by June, and she told me so. I dismissed both these opinions as far-fetched because I just couldn’t see society locking down for that long or being all that good at following restrictions on a long timescale, and I guess I thought political incentives would make politicians cut restrictions quickly to avoid excessive economic damage.
Well, it turns out I was very wrong, and my friend and parents were right from the beginning. It’s not just that they paid more attention than me—at the beginning, it was all any of us were thinking about—but it’s that their intuition and analysis were both more spot-on than mine despite that attention.
This post is my attempt to describe my own journey of COVID-related opinion and my track record with predictions in general, and to generally explore why I’ve been such a bad predictor of things until recently (hopefully).
The issue of the day
I was traveling for business the day the first lockdown was announced in March 2020. I had just started on a contract with a particularly large card company in Kansas City, and my team had arrived there the previous day to meet their team. My cohort was aware of COVID—it was rumored there was exactly one confirmed case in Kansas City by then, although I never took the time to verify that—but nobody cared all that much.
I cared about it enough before my flight that I wanted to stay home, but my concerns were ignored by my manager and everyone up the chain supported that. I remember listening to this podcast on the plane ride there, in which Sam Harris implores people to take COVID seriously and not assume it’s just the normal flu. I recall telling my girlfriend, a teacher, that she might not make it to spring break before lockdown began. I was right—her school shut down one day early and didn’t start up again for months.
I remember my team’s negotiation in Missouri once the national lockdown was announced. We all received an urgent email from our COO saying something like:
Leave the office and return home immediately. If you are currently traveling, immediately conclude your business and take the earliest possible flight home, regardless of cost.
My manager, who didn’t take me seriously, also didn’t take my company seriously. He encouraged us not to come home until the end of the following day (over 24 hours later), which violated both company policy and my intuition. Remember, this was a time of great uncertainty: nothing like a national lockdown had happened before. Nobody knew how long it was going to last or how deadly or transmissible COVID was. Most importantly, nobody knew if they would shut down the airlines or take other drastic steps to stop people from traveling. With the benefit of hindsight, this obviously was never a concern, but nobody could claim to know that at the time.
I reacted to this by ignoring my manager and switching my flight to the soonest possible return home, which prompted a deeply judgmental shake of the head with some verbal chastisement. In doing so, I potentially directly avoided getting COVID because another member of my team later tested positive as one of the first confirmed cases in the Austin, Texas area, so that was my vindication. I was following along with the consensus I saw in the more intellectual sphere online to the extent I could get away with it. (My manager later dropped the issue after some directives from up the chain clarified he wasn’t going to have allies there anymore. I’m grateful to them.)
But that—the fact society would take lockdowns seriously for a while—was the last detail of COVID that I would get particularly right for six months. For one thing, I thought lockdowns and mask mandates would end rather more quickly. In July, I thought that New York City wouldn’t experience another spike because they were probably already close enough to herd immunity. Then I was surprised again in December when it became clear how effective the vaccines were and how quickly they would be distributed. (I had lots of fun trying to get the vaccine as early as possible.)
To add another layer: I was also wrong about the outcome of the presidential election. I thought it would be an uncontested landslide on the first night, but instead it was quite close and unclear for the better part of a week.
Basically, I was mostly wrong over the course of the last year or so. Why?
Take heed in your own opinions
Sometime around January, I read Inadequate Equilibria by Eliezer Yudkowsky, a book I’ve repeatedly recommended in the last few months. The book is a powerful case for trusting your own judgment over society when it’s justified and offers some heuristics for when it is and isn’t justified.
For instance, in areas like the traditional stock market, where there are institutions with giant teams whose sole job is to research every inch of the market to find profit opportunities, it’s not so likely you as an individual would ever be able to outperform it. But in areas without direct profit incentives, it’s much easier.
For example, the book opens with the example of the Bank of Japan’s monetary policy. Because nobody outside the Bank of Japan can change its policy decisions nor profit off the calculated belief that those decisions should’ve been different, and because the Bank of Japan’s board itself only has an indirect interest in the economic prosperity of the country (i.e., the country may benefit, but the individual board members making policy decisions won’t personally profit, and more importantly they won’t be personally penalized if they don’t hit their forecasted targets), nobody inside nor outside the institution is incentivized strongly enough to find the most accurate possible beliefs. There is no Goldman Sachs-like institution spending countless hours on diligence to argue whether or not the Bank of Japan should’ve made decision X differently five years ago. The real Goldman Sachs is just contending with those policies’ downstream effects in the actual market, and academics etc. have plenty of other factors to worry about, few or none of which relate to personal profit.
Another example is housing prices. Say you believe a certain house is overvalued by $100,000. What do you do? Well, you can’t do much except pass on buying it. It’s possible to go long on real estate by investing in it, but you can’t really short the housing market—there is no financial instrument that behaves like a house other than a house, and so there’s no instrument that behaves like the inverse of a house either. The closest investors can get is to work with shares in publicly-traded real estate investment trusts. Importantly, shorting these shares puts no actual downward pressure on real estate prices, so there’s no way for prices to correct themselves downward in a manner that benefits the wannabe arbitrageur even if the house truly is overvalued.
The point is this: there are cases where it’s reasonable to believe you know better than a large institution at its own game. The most important component to such thinking is consideration of profit incentives: who wins, financially, if a certain belief is or is not calibrated? If there are no clear winners, it’s likely nobody is spending that much time worrying about it.2
COVID was an absolute gold mine of examples of this phenomenon, as well. From the beginning, it’s clear that the political left cared much more about lockdowns than the right did—most people seem to think this is mostly because of Trump’s influence, and that may be, but I think it’s also a relatively standard ideological disagreement between the two sides: the left would rather regulate, and the right would rather rely on individuals to make decisions in their own communities.
For the first, say, six months of COVID-world, I was a fervent supporter of the left view. I abhorred DeSantis for opening Florida so quickly, and I appreciated Cuomo and Newsom for keeping their states locked down. But then Cuomo didn’t reopen indoor dining in time for the NYC winter and both states’ case counts spiked anyway. By the time the press acknowledged for the first time that DeSantis probably made the best decisions of the bunch, I’d become grateful to live in Texas rather than fearful. This was doubly true after I got my vaccine because I was quickly able to do things with relative normalcy. I don’t think I would’ve been able to if I was in California.
I think both sides of the political spectrum exhibited unimpressive groupthink concerning COVID. The left doubled down on lockdowns and mask mandates so hard and preferred to brand people “covidiots” etc. rather than acknowledge the reality that asking people to stay home and not see their families for a year is unreasonable and people wouldn’t comply. The right, for a time, decided that mask mandates (by and large the least disruptive major mitigation policy) were the most shameful example of governmental overreach of the last decade and must be stopped at all costs.
How does this relate to me? Well, for the first several months of the pandemic, I was absolutely in the former camp. I wanted everyone to stay home and had trouble imagining how hard that would be on society’s mental health, etc. I thought compliance would continue to be high.
I can’t pinpoint the exact moment, but within a week or so after I finished Inadequate Equilibria, I realized that I hadn’t really been relying on my own judgment to analyze COVID. I deferred first to the mainstream media, then flirted with the right-wing contingent, then deferred to the opinions of my more well-informed friends. I don’t think I gave myself permission to think freely about it, so to speak, until I read the book; I assumed I’d never know as much about it as the better-informed members of my peer group or the “experts” that I presumed to be among the ranks of the New York Times etc.
It turns out, though, that anyone able to look out their window could’ve (and probably should’ve) seen that lockdown efforts would fail due to non-compliance after no more than a month or two. Most of the people I know, even the most progressive, pro-lockdown friends I have, started violating lockdown restrictions regularly to see their families and friends—and everyone I know also had an impression like that from their own friends by the middle of the summer. There also wasn’t a robust correlation in Europe between lockdowns and case rates (see e.g. the mid- to late-2020 numbers for Sweden versus everywhere else in the EU).
Prolonged lockdowns were never going to work because the average person just doesn’t care about the risk of being personally shamed by Andrew Cuomo if they see their families for Thanksgiving. And this is all to say nothing of the economic damage. I think this was obvious months ago, and it’s an intellectual failure on my part because it took me so many months to see it.
I was talking with a friend last week about how I wasn’t incentivized to think it through very deeply. That’s probably true. The lockdowns actually didn’t disrupt me, an introverted homebody who works remotely, all that much compared to my friend who had just relocated to Chicago a few months before and had his entire plan to build a social life there ruined. It’s natural he would be more concerned, think more deeply about the entire situation, and be more skeptical of cultural messaging. But I started out being more well-informed and concerned than average, so at some point I dropped the ball.
Hindsight bias as a scapegoat
There are some facts I think are beyond dispute at this point: the COVID vaccines are safe and effective; the COVID vaccines are the only reasonable way out of pandemic conditions, and they are working to that effect; the mainstream press, the FDA, CDC, EU, WHO and many state and national governments throughout the world have all been shamefully ridiculous at various times throughout the pandemic;3 etc.
There was an interesting discussion on a recent podcast with Julia Galef and Vitalik Buterin. Julia asks Vitalik if a good rationalist ought to have seen the value in cryptocurrency sometime in the last decade, before the broader population. He answers in the affirmative and explains that it’s basically like investing in Google in 1998, except that there was the added factor of a thousand people shouting as loud as they could online that it was like investing in Google in 1998. I’ve had this conversation a few times with people since, and the response tends to be that I’m speaking with great hindsight bias. Maybe, but I’m not convinced, because I remember the people shouting online that Vitalik was talking about—I just decided to ignore them and, worse, to not look into the facts all that much.
I think COVID was rather like this. I remember the good discourse on Twitter (which I would describe as all the COVID-related information prior to, say, March 1, 2020). At this time, Twitter was probably the best available source of accurate information about COVID. Then at some point in March, it flipped and Twitter suddenly mostly contained misinformation. I can blame the misinformation all I want, but I was previously exposed to the good discourse, and there were plenty of respectable thinkers in my timeline (I think particularly of Ben Thompson of Stratechery) who explained to me that the discourse was good. So I can’t deflect blame; I just have to acknowledge that I became overconfident in opinions that I hadn’t thought all that much about.
What about now? This post is a reflection on what previously went wrong with my judgment, and my broad answer is that I don’t think I’ve paid attention to my own reasoning enough. I like to think that the last two months have been a time when I’ve begun to develop and express my own opinions more confidently and with less regard to how popular or common they are—meaning, I’m not conflating “more mainstream” with “more likely to be correct” anymore. I hope this yields good results.
I’m also interested in taking some advice from the conclusion of Yudkowsky’s book. He suggests that, to incentivize yourself to be more correct, you should bet on your beliefs, preferably with real money. So I’m going to do this: sometime in the next few weeks, I will post a set of predictions in a format similar to Scott Alexander’s. As a start, these predictions will relate to a time frame no longer than the current calendar year, and I’ll accept bets on them. I’ll explain how to participate in that post if anyone is interested.
In 2018, Scott Alexander also did a batch of predictions spanning five years. We have to wait until 2023 to score these.
Not discussed herein, but there is also a factor of creativity: sometimes there is a good idea that nobody has thought of yet, and it is possible to be the first to think of something.
I’ll spare my readers the rant, but I refer to (among other things): misleading messaging about the deadliness of COVID, inconsistencies in policy across relatively short spaces of time, deceiving the public by repeatedly saying lockdowns would be short-term and then advocating for extending them and other restrictions repeatedly, the fact the US government hasn’t exported its stockpiled doses of the AZ vaccine yet, the ridiculous suspensions of the J&J and AZ vaccines in various jurisdictions due to a supposed blood-clotting issue, and of course the many transgressions committed by national media institutions more interested in scaring and shaming people than communicating true information about COVID. Oh, but I digress.