with Spencer Greenberg
the podcast about ideas that matter

Episode 098: What we really mean when we say that something will "probably" happen (with Walt Hickey)

Enjoying the episode? Want to listen later? Subscribe on any of these apps or stores to be notified when we release new episodes:

March 31, 2022

Graphic novels have been around for a long time, so why have graphic novel sales skyrocketed suddenly in the last year or two? Is that growth representative of a possible trend away from reading text-only books? Do animated movies and TV shows allow for greater suspension of disbelief than their live-action counterparts? Does animation make it easier for movies and TV shows to push the boundaries of what's culturally acceptable? What is "soft" power, and why is it so important? How important has America's soft power been relative to its hard power over the past century? How is America's soft power affected by Americans' own views of their country? What's lacking or misguided in the ways most people think and talk about probabilities and statistics? Is it better to report predictions with very specific probabilities (e.g., "It's 67.5% likely to rain today") or with looser, more casual probabilities (e.g., "I think it'll probably rain today")?

Walt Hickey is a data journalist who founded and writes Numlock News, a daily morning newsletter about fascinating numbers buried in the news. His primary interests are in pop culture data journalism and how journalists can better integrate stats into stories. He's currently writing a book about how pop culture impacts its consumers and is the senior editor for data at Insider. He previously worked as the chief culture writer at FiveThirtyEight and has written for You can follow him on Twitter at @WaltHickey and subscribe to his newsletter at

JOSH: Hello, and welcome to Clearer Thinking with Spencer Greenberg, the podcast about ideas that matter. I'm Josh Castle, the producer of the podcast and I'm so glad you've joined us today. In this episode, Spencer speaks with Walt Hickey about print media and graphic novels, soft power, American political identity and forecasting. By the way, this episode was recorded about nine months ago. So please keep that in mind when they reference events which were current at that time. And now here's the conversation between Spencer and Walt.

SPENCER: Walter, welcome.

WALT: Hey, how's it going? Thanks for having me.

SPENCER: Yeah, great to have you on. So the first topic I want to discuss with you is about the sort of shifting media landscape and how media consumption has changed. And I think you have an interesting take on this. Do you want to tell us a bit about that?

WALT: Yeah. So again, thank you, again, so much for having me on the show. It's a really fun time. And I love the concept and the ability to kind of get deep on a topic. And so this is kind of based off a recent news item. But it's also a little bit about these broader changes that we're experiencing when it comes to media consumption. Like a lot of folks are very obsessed with the streaming transition, and our movie theater is going to stick around. But there's actually like there's a really substantial other transition that has happened just kind of in the past year, when it comes to how adult fiction books are being sold. And so the stat that just came out rather recently (It was at a Publishers Weekly.) was that graphic novels. So these kinds of long-formed illustrated, comic books, or what have you. Graphic novels, in the first six months of the year were 20% of the adult fiction market. That's up from 9% last year. (So in one year.) And so the other part of this is that book sales overall were like flat or slightly up. So it's not like everything else fell apart, it's just that graphic novels ate just a huge chunk of the market. Like last year, they were 9%, this year, practically doubled. That really kind of coincides with some other trends that we're seeing on the web. Mainly, I just kind of wanted to talk a little bit about like, these different ways that we consume fiction and things that we enjoy. I think that that is such a substantial jump and it's such a substantial shift. And we've already seen the web change our capacity to consume large video or, texts without images. And I would love to kind of talk to you a little bit about your experience with with that online and your experience with perhaps graphic novels and things like that.

SPENCER: Yeah so as starter, what do you think is going on here? Like, why would a doubling of graphic novel sales?

WALT: Yeah, there's a couple of things kind of like at play. One thing that I'm super personally interested in is like the data behind pop culture. And one element of that, that I think that a lot of times we miss out on when we talk about either box office, or ways that people consume things, or streaming numbers is like the international component really can't be slept on enough. And I think that to give folks a little background on this, obviously, serialized comics have been popular in the United States for a very long time. There are two very large companies that focus on them to a large extent in Marvel and DC. But like that tradition hasn't happened in a vacuum. Right? There's obviously a extremely deep and long and reliable amount of manga out there from from Japan has been just kind of a whole industry in and of itself that has really kind of percolated around the world. And then you've also seen a number of Korean style web comics, in particular, really hit the mainstream. And so I think that what you're kind of seeing here is twofold. One is that other countries that really kind of share some fairly direct cultural links to the United States these days have really started shifting very heavily towards these visual novels. And I think that American publishers are kind of picking up on that. Then the other half of this is obviously like, really, when you talk about how, whether it's Millennials or Gen Z, or what have you, there's already a very large current through this generation of consuming comics on the internet, web comics, obviously, have been a staple of the internet since the earliest days of it. And so I think that you're seeing a level of literacy with this medium that you're not seeing before. And the pandemic alone — when people really started buying books and experimenting with that kind of stuff — you just got a whole lot of believers when it came to that. What do you think of the hypothesis?

SPENCER: Yeah, so you got me thinking about the time I visited Japan, and I went to this place called the Manga Library, where they just have a huge amount of manga that you can read, and some of the English so I wanted to check it out. And I flipped through a whole bunch of manga there just to sort of get a sense of like, what is this phenomena? Because it really doesn't seem like comic books in the US, right? I always thought of them in the US as being really targeted towards young boys, because I'm totally graphic. And when I was a boy, I used to read comic books, manga seems much broader. And it seems like as many more different diverse groups is trying to target also many different styles. So I flipped to a whole bunch of it. And I was kind of shocked at the kind of really intense themes that are dealt with. For example, there was one manga I read about a weathervane that had fallen in love with the owner of the house and it was this really intense like strange story about how the weathervane like turned human in order to try to have a relationship. It was like, what, like, “Who is the demographic?” or “What is this? What is this?” And then I remember there was a whole wall of manga that was just stories of like boys falling in love with boys. And I was just like, this is just a completely different phenomenon. So, I wonder if what's happening is the thing that Japan has been doing for a long time, it's just finally spreading around the world, rather than it being a really kind of unique new thing that's happening.

WALT: Yeah, I think that — so, I definitely don't want to reduce it all to Japan — but you've definitely hit on something where like, I don't think that Barnes and Noble had a reliable manga section in perhaps an era before the ‘90s and early 2000s, when a lot of folks who now are like the tastemakers and cultural consumers, whether it's the Gen Z, if you want to put it that way, or even just like millennials and whatnot, who kind of grew up on this stuff. There has just been this kind of cultural osmosis for a long time, as well, as you kind of see like the rise of like mainstream anime and whatnot, and how that can kind of link into it. But the other big chunk of this that I kind of want to get into is like, not just looking at the published adult fiction market. So there's two huge players online — one is called Webtoon and one is called Tapas. And in the past six months, both of these companies have sold for an incredible amount of money. These are basically companies that do like mobile-first very scrolly style web comics that are very kind of global in nature, I would say. One of them, like Webtoons, also, got 600 million copies sold for like 500 million. And so one got bought by Nava, which is a South Korean media giant kind of thing. And so I just think that like, all of a sudden, you kind of saw somebody that had kind of been in the background for many, many years really start striking out. And I don't know whether the amount of material just on the surfaces increased substantially over the course of a pandemic, where everybody's kind of stuck in their houses for a little bit. But like, I thought, it's just really interesting that like, in the course of a year, you saw brick and mortar stores large a huge substantial chunk of the election market. I know for a fact that if you go to like a library in New York City, a lot of the time, some of the most frequently checked out books are from the manga section and whatnot. So it's really has hit with kids in a way that I don't think folks I've really noticed yet. The bigger question that I would kind of like to throw your way is like, this is potentially just kind of a new storytelling medium that clearly has interests like in the same way that Tik Tok started out as just like people doing dances and stunts, but then like, now, there's news on it. And there's people attempting to like, mainstream people attempting to work in the media and that kind of stuff. And I guess, like, do you kind of potentially see this as like, if you build it, they will come? Like, are people going to begin increasingly using these kind of visual first media to explore?

SPENCER: Yeah, I don't know the answer to that, but it does seem to tie into a sort of trend of people not liking to read.

WALT: Yeah, okay. Let's go with it, too, like, I think that's definitely part of it.

SPENCER: I mean, just seem like, increasingly, people want to listen to things. So podcasts obviously have taken off in a huge way. It also seems like people want shorter things like they want us text. At least that's the impression that I have, and also that people want things to be in smaller pieces. So I wonder, do you think it's playing into those kinds of trends? Or do you think those trends are real, or do you think everyone just always think that, “Oh, the kids these days — they don't like to read,” and that's just a perpetual belief?

WALT: Wow, great question. Do people think that kids always want to read less? Yes, that has always been the case. This is not an actual thing. Everybody thinks that the screens are eating their minds and all that stuff. That has been the case since ancient Greek times. There are stories that have been passed down of just like in ancient Rome of just like, this younger generation, they're all dissolute drunks. I think that that's a misperception on one hand, but on the other hand, (so I don't know if people actually want to read less.) I think that people think that they want to read less. I think that people want to commit less to stuff. The reason I don't think that's actually true, is that if you sign up to read one of these series, these things are like dozens, if not hundreds of episodes long, right? If you are interested in a franchise, Western or otherwise, there aren't really one shot comics that do particularly well. If you look at the top selling ones — I think, a lot of it is like My Hero Academia, and things like that — so like these long, ongoing series, that obviously are multimedia in whatever fashion you have it. I think it's really interesting that much to the contrary, these wouldn't be the biggest sales, if they were just kind of “I only want to read a little bit.” I think that there is something to be said for like the effectiveness of comic books as a medium that is able to cut a lot of words in a way where you don't need to have a paragraph describing action where you can just have an image. There's a reason that they say “A picture's worth 1000 words,” I suppose. So I think that this is a really good question, “Is the information density available to graphic novels superior to the density that people would tolerate within a written fiction book?” you know.

SPENCER: Well, I think that's a good point. It's not about necessarily wanting to spend less time with the material, maybe just wanting there'd be less words. I don't know how reliable a survey is, but there was a survey by the National Literacy Trust. They said about 26% of under 18 spend some time reading each day, and they claim this as the lowest level they've ever seen since 2005. So maybe there is something to this. People want to read less time, maybe there's just more competition or just like more things to do now besides reading, right? It's just like there's growth in all these other forms of consuming media.

WALT: Yeah. So I will put a counterpoint to that, and this is going a little bit further back. So one reason I'm really interested in this stuff is because I'm working on a book right now. The gist of the book is like, “How does pop culture affect people and its consumers?” And so much of that comes down to time usage. And so there was the study that came out that people have felt that television routed kids brains pretty much since the invention of television. We know this, because they wrote about it in magazines. But there was a study that happened that when television was first rolled out across places in Northern Canada. The reason was that the way that you roll out television in the 1950s, and ‘60s was like “You got to build a thing.” And then that thing can send a signal in a certain amount of distance. As a result, you had a lot of really good natural experiments that take place where they would put up a tower to broadcast or amplify television waves going over the prairie and whatnot. And there would be some towns that got tell efficient, and then some towns that did not. And these would be similar towns on paper prior to it. So you can kind of see this natural experiment for me. The reason I bring this up is that they expected to see that the towns that got television would see kids time switch from “They would do less homework, and they would watch television” And what they actually found was not that at the slightest. They found that kids spent less time reading and playing games and just doing other recreational things in exchange for that television time. So they leaving that like they weren't like taking time that they would have otherwise spent, like studying math homework. In fact, the grades were rather, if I recall their study correctly, were pretty much the same in each one. You didn't really see an up or down. But like the long and short of is like whenever you kind of look at like how kids spend their time, you always have to ask “How did you collect that data”? And it's not even necessarily just like, “Are they reading fewer books?” Probably. “But does the study count graphic novels books? Does the study count webcomics books?” Just because they're a digital native medium, even though they are potentially some of the same stuff? I think that like a lot of the times when you see this kind of stuff, the question that people don't ask when they think about like media consumption studies is “Okay, so consumption of Tik Tok is up to two hours per day. But like, where did the two hours come from?” Well, probably came from YouTube or came from Twitter, or came from a... and so when you talk about zero sum games with media consumption, oftentimes it's not eating into the actual, beneficial day to day life stuff. Or like, sometimes you'll lose a little bit of sleep. I think when you shave some of these one way or another, but like beyond that, I think so much of the interesting stuff about this is like, compared to what and so I would be really interested to see about that study of our kids reading less, or kids reading less books, because they can still read stuff on like the fanfiction like sites of the internet have never been as popular. And they've never been as popular among the teens and kids that we're always trying to talk about getting into reading. And so like, I always love like the comparisons, like, sure, book usage is going down, but like compared to what?

SPENCER: That's a good point. And if there's one thing I really learned about reading social science studies is that you have to look really carefully at the details a lot, even something like “Oh, the people spend time on Facebook.” Well, what are they doing on Facebook? Right? Are they looking at cute animals? Are they comparing themselves to their friends? Are they reading interesting thoughts from reflective people? Like there's just there's a lot of different stuff going on Facebook.

WALT: Yeah. Like, it's always just like, Oh, these kids are texting all the time. It's like, no, they're not, they're talking to their friends, which they would be doing if like they were in the same room as those friends.

SPENCER: Right, I guess it depends on if you think texting sort of is a sufficient replacement for hanging out in person, or if you think it's somehow inherently inferior.

WALT: I don't know. I mean, most of my age, like, again, I am 30. I am a child of the ‘90s, fundamentally, and I think that most of my close friendships with people in high school were absolutely cultivated by AOL Instant Messenger, a medium that is dead and has now fundamentally, but like I think that like communication's communication, whichever way you cut it.

**SPENCER:**Yeah, abs olutely. I mean, I grew up on AOL Instant Messenger as well, and actually felt that it facilitated deep conversation in a way. They're sort of it's not anonymous, obviously, who you're talking to you most of the time, but there's some way in which like, not having the person there facilitates really like saying what you think and what you feel. I feel like that's less true. It's like something like a text message. Maybe because text messages sort of asynchronous more inherently, you know, you don't necessarily expect a response right away with Instant Messenger, you're kind of actually really having a conversation but you're just gonna have to see the person's face.

WALT: Yeah, text messaging consistently feels like a utility, like I use it to make plans or use it to do that. Like there are other mechanisms like, I think it's also the for whatever reason, the phone tapping just feels like a uniquely bad way to kind of like have that kind of mental conversation with people and that kind of stuff. But that could just be me, it could just be again, me dating myself.

SPENCER: So finishing up in the graphic novels topic. There's one more thing I want to mention about that, which is that I feel that drawn visuals can actually facilitate going deeper or into darker topics, then you can kind of get away with in some other mediums. And so example of this scene is in shows like South Park or Rick and Morty, where you can have like extremely dark themes. And because it's a cartoon, sort of like everyone's okay with it. And think about like, “Could this be done in a live shot with like actual actors, and I think like, a lot of it couldn't just get away with it.” And I wonder if graphic novels are sort of like that, too — you can push it like really intense themes that actually would hard to get away with other mediums.

WALT: This is such a good question. And so I'm going to recommend like a YouTube channel that I'm kind of obsessed with. It's called Sideways. And he's basically a music analyst, and analyzes the use of music and film, I have no ear for music in a way. And so like watching this kind of channel is like, watching somebody with a superpower, but like, so this idea is for him. And so the idea is if you think about why the Disney animated musicals worked, (and for a lot of people, the Disney live action musicals didn't) is that the reason that musical films for children can work is that they're kind of taking the language of Broadway, and the language of the stage and that kind of stuff.That works differently than a movie does because you are in this kind of heightened level of unreality that like, “Oh, you need to look past the obvious play with sets” and your brain kind of has an elevated tolerance for suspension of disbelief, because obviously, with the real people in front of you, you as a human being can understand that this fictional thing being put upon you. Whereas so much of film's ability to affect people is that it is directly kind of imitating the ways that our brains perceive the world around us. And you can get that kind of visibility, right? It's the reason that like a movie like Saving Private Ryan can like really kind of use all the tricks in the book to make you feel like you are at Normandy, whereas like, if you were to do that on a stage, it would not be anywhere near as effective for a couple of reasons. And so the reason that I bring that up is that the animated films are built in structured exactly like a Broadway musical where you have different songs for different utilities. The idea is that a song only happens when the emotional intensity of the scene rises above spoken word. And then in the course of the song, a dance happens when the emotional intensity of the song rises above your ability to sing it. That's a big Howard Ashman idea of what the guy who was the musical architect of a lot of the original Disney Renaissance films. And so the reason that is effective in an animated film is that like, just like the stage, it has this heightened sense of reality, where your suspension of disbelief is just elevated, right? And as a result, you're able to tolerate huge musical numbers busting out simultaneously, that when it's put onto a live action film, like the Lion King film, or Beauty And The Beast, I think did a rather good job with a lot of it. But still it doesn't really match the original for a lot of that kind of sense, as the animated medium really does kind of allow you to get away with a little bit more just through the very basics of it, right? Because you have this mental acceptance of “The rules are gonna be a little bit different here,” right? And it's real hard to pull that off in a direct film adaptation. And so I think, with the comic stuff, you can see a lot of really unsatisfying comic book adaptations, like the first Watchmen movie, I think it really visually attempted to imitate the comic, right of the Alan Moore comic, but in doing so, it still didn't come off as realistic in a way. Whereas if you see later adaptations of the HBO Watchmen, which kind of takes a little bit more nuanced view, and really grounds it out of that in the comic book framing, but more in a real life setting, it's able to get a little bit more of that for similitude. Or if you're able to look at like the Spider-Man: Into the Spider-Verse, they're able to kind of take the tech like the look and feel of a comic book and make it much more like kinetic in a way that perhaps The Amazing Spider-Man movies couldn't necessarily be.

SPENCER: You mentioned realism and sort of heightened reality. One thing I think about is when watching horror movies to go off and like cover their face and maybe look at the screen through like a little slit in their fingers. They're sort of lowering the resolution essentially, like making it feel less real, because it's like, feels too real. It's scary. And I feel like cartoons kind of do that for us, or animated things, or even drawn things like a graphic novel, where because it doesn't look totally realistic. It sort of like reduces the real lifeness and then that allows us to experience something that's more intense, where if it did feel like real life, maybe we couldn't handle it or be disturbing. Whereas like, if something really disturbing is happening in a graphic novel, it sort of feels more separated, like you can deal with it better than if it's like a super realistic scene in the movie where you have to let's say watch a person be tortured or something like that.

WALT: I love that because I think that it really gets that like, there's this kind of notion I think within some of the larger companies these days that like we just need IP the IP is what's good. We can convert anything from one to another and across the board, right? Like books can become movies, movies can become comic books in that there is this kind of like parallel thing and for some things it works. The MCU did a very good job of figuring out how do we situate these like larger than life characters within a realistic world to an extent. But that was an evolved thing because it started with a character that was Ironman that was very, very based in technology, right? And then they got more and more fantastical as time went on. I think if you started that movie with Dr. Strange, you don't have an MCU for the obvious reasons, right? And so like, not everything can convert very well. But I think that comic books, in particular, in graphic novels in particular, there's a reason that they're making a Sandman show, but there's a reason they called it unfilmable. Like, there's a lot of I think, like unfilmed, mobility or unsatisfying film ability that comes across that medium. I also think that like, another good example of that is like, video games, most video game movies bomb entirely, like, the highest grossing video game movie, I believe, is the Sonic movie. And that just kind of very much tries to adapt the character not like the story of the Sonic video games, right? And like, you see the same thing, because the reason that video games are emotionally affecting are because they place you within this space, and you have to act within that space, thus immersing yourself further. If not, then you get a movie like Doom, where it's trying to like, kind of imitate that sensation. But by losing that manual physical control, you lose the investment with it. And so they don't have enough to kind of stand up. And so like, I think that you've hit on a really, really good point, which is that, like, the adaptation stuff isn't super as clean as otherwise might be. And I think that the fact that we're seeing graphic novels as a medium grow so much is perhaps just because like, there are things that you can't do in written adult fiction or can't distribute or can't get across that you can in these kinds of visual features, whether they are the web based ones or the physical publish worlds, you know?


SPENCER: So totally changing topics here. I know that another interest of yours is Soft Power, who would tell us what is soft power? Why do you think it's so important?

WALT: Yeah, this is a great question. Because it is like, we all kind of talk about it a lot. But we don't actually like specifically kind of address it. And like the idea of soft powers that like so you have hard power. And we're talking about countries here. And countries have hard power, like United States has several aircraft carriers, these aircraft carriers have lots of people who know how to use planes on it. Those planes can do significant damage to any country that the orchestrator of those aircraft carriers, if it so choose. As a result, the United States, if it so chose, could use hard power to obtain its goals. There could be wars, there could be assaults, there could be all that kind of stuff. That's hard power, military power, things like that. And for a while, that was how you developed influence around the globe, that was how you would drive a bunch of ships into Tokyo Bay — you would be the British Navy, you would have this kind of physical footprint of force that allows you to exert cultural influence in places that you otherwise would not be able to write through compulsion, intimidation, or any kind of thing with that. So soft power is different. And soft power is basically like how countries are able to obtain goals or recruit allies or come to a geopolitical place that they would like to be not through the use of force, but rather through making other countries want to be their friend. And there are ways that we can do this, you can talk about the business side of it, if we export a lot of nice goods to a place and we become friends through trade, that's a very viable way that these things happen. But the way that I'm most interested in, particularly in an era, like the one that we're in right now is that cultural soft power is really hard to get and really, really valuable when you have it in the way that I think that as you kind of look forward to the next couple of years and decades of geopolitical competition. That's going to be a playing field that is kind of not really talked too much about and I think that soft power is the kind of thing that we all kind of intuitively understand and the way that like if you think about why you like Monty Python, and how that makes you feel about Britain for why you like Pokemon and how that makes you feel about Japan, or why you like BTS and how that makes you feel about Korea. You can really kind of understand the implications that this has and why you don't feel the same way for France or you don't feel the same way for Indonesia, even though they're countries of either comparable sizes or economies that don't have the cultural soft power exports that some of those do have.

SPENCER: Yeah, there's this interesting quote about soft power from Nye, and I think kind of illustrates the point, it's when one country gets other countries to want what it wants. It might be called soft power, in contrast, the hard or command power of ordering others to do what it wants. And you thinking about the United States, one of the incredible powers of the United States is to influence culture worldwide. And it seems like it's done this tremendously well, or at least effectively, if you think about the movies people watch. So much of the movies to watch come from Hollywood, and all of the kind of influence that has on the way people think about, like what's important and what's valuable, and what do they want their culture to be like. I do wonder whether the United States is sort of declining in soft power, whether we've lost a lot of respectability over the last 20 years. So I'm curious to hear your thoughts on the United States' soft power and how that's changed over time.

WALT: Yeah, I think the big question, like, I think that no matter when folks (Obviously, China is a country that many folks either in the US foreign policy, establishment or otherwise are very kind of concerned about competition with in a way and a lot of that conversation comes down to military or military applications. And I will leave that to the people who know about that.) But I think that the more realistic way that these things get settled, and I think that like a lot of the Cold War was settled in this fashion was “Do you have a country that is producing things that other people enjoy and want to be a part of?” Right? And for a while, that can be as simple as the United States kind of developing and exporting a considerable amount of culture, to kind of back it up a little bit like, one of the reasons that the United States emerged as kind of the center of the movie industry on Earth, for a couple of reasons was that after World War One, Europe was really fundamentally broken in a lot of different ways. Obviously, there was a huge economic fallout of that. And obviously, there was an entire generation was lost. And so as the technology of movies was really becoming very popular, like the 1890s, which is where all this kind of innovation takes place. 1900s 1910 — that's when you can see the technology becoming popular, and really just kind of the footprint of these beginning to expand. And then obviously, you have the Great War. And whereas Europe was kind of burned to the ground in a lot of different places, the States was not, and so you would have investment from the financiers back East. And then you had lots of different little domestic fights about “Would Edison control the movie business out of New Jersey?” or “Would Florida come up?” and then eventually, Los Angeles for many different reasons became the kind of global seat of that, whereas like, British actors would leave Britain come here to work predominantly, right? And from there, the states never really missed a step. Like there was never really a time when you saw the movie industry begin to form. And I mean, in the industry, again, like this is not in any way to like diminish the vast cultural output from many of these other Western countries and other sophisticated economies, I'm just kind of referring to the actual business and capital of the stuff right? Again, obviously, also, you have corporations like Sony, which is a Japanese corporation that owns a studio, but I'm mainly just talking about the center being within Los Angeles. So,from there, you just kind of get to this point of what's it going to take to lose it. And I think that you were kind of alluding — you mentioned past 20 years, which I think is an interesting timeframe, because obviously, you were kind of referring to like Iraq and all that.

SPENCER: Yeah, exactly. You know, basically the last few presidents have affected sense of American power, well, sense of American respectability, and so on.

WALT: Yeah, respectability is an interesting word. What do you mean by it?

SPENCER: Well, it's hard to know, because I feel like I don't have the most unbiased news source about like how people all over the world perceive the US. But it did seem that a lot of people were upset and angry with US over a whole variety of decisions that were made over the last 20 years — at least, that's my perception.

WALT: I think that that is correct. And I don't want to undermine that. I think that, respectively, it just struck me as interesting. I don't know how much of that is a part of it. I think that, obviously, people had problem globally speaking, right. If you look at surveys that pew runs of other countries, attitudes towards the United States dropped considerably during the Bush administration and dropped considerably once again during the Trump administration. And the thing that I like, really think on that front is what if that doesn't really matter as much because of cultural soft power that has been accrued over the course of 100 years. The movie business is making more money than ever, at least was prior to the outbreak of the novel coronavirus, but if you look at where the actual footprint is, I strongly feel like, South Korea, Japan, the United States — that integration has really become much tighter I think over that period that you were just referring to than it was at the outset of it right. And if you look at the reputation on the continent than that does change kind of year to year, depending on who's kind of in the White House at the time. But Germany still buys a lot of tickets to Sandra Bullock movies pretty disproportionately. And then the American producers are still putting out films from European talent. Ken is still an extremely popular outlet for major Hollywood studios to help with their content. And so I think that you can kind of see, like, the thing that I find the most interesting with soft power is like, when you have it, it does take quite a bit to squander it. Obviously, the Trump administration saw a very substantial diminishment to soft power in the United States. And I think that that was kind of the biggest threat that he posed, in many ways towards the long-term, state of the state's reputation, just by kind of whether it was undermining organizations like NATO, which are like, I guess, a military hard power thing, but also kind of a soft power alliance. But like, there is something really to be said that, like the American cultural industries will did manage to kind of continue bail us out of some of this in a way. But I think that like, no, I do think it made a dent. Like, I think that like, the reputation went down, but then like it did go back up again during the Obama years. So I wonder how much of that volatility is just kind of result of the immediate geopolitical prescients versus like the long term earned capital that is real hard to get rid of?

SPENCER: Yeah, I think we can distinguish with at least a few different ideas of soft power, which I think you were alluding to one of those is cultural influence — things like movies, right? That can be very subtle, people watching us American movies, or sort of getting a dose of American values in the background, right. Another one is a sense of trust in America as an entity, right? Like if people around the world believe America is on a fighting for good? Do they trust America to make good decisions? Do they want America to be a policeman for the world? Or do they think that's dangerous, and so on? And then there's also just something else, which I think is like, the general sort of respectability among the elite? And like, how did elite, I don't know, like, let's say, academics and policymakers around the world, like, how did they think about America? I think all these things can be somewhat different. And even though even the fourth one, which is just like, do people want to be here, man, like, is there a huge demand to move the United States to kind of have the way of life people in United States, which I think during the Cold War, like was one of the assets that US had is that like, people just emulated like, they just wanted the American lifestyle, and that was like, very attractive to a lot of people around the world.

WALT: Let's think about soft power from a couple different like, colleagues, let's say, like, I think that this one we set for, like the OECD countries are a lot to kind of cast like rich do. People who live in other rich countries want to move to America, it depends, I think you kind of brushed up at it. But I think that another huge agent of American soft power is the universities and colleges. And that is a world that I am not really a part of. But I understand that, again, that is a huge draw for a lot of places. And like the ability to study here and that in advance that kind of stuff. But like the other part of that is like, let's talk about a country like Turkey, where developing very quickly is like a hard power ally, but also like, the soft power makes a difference there. You can see that with India has always been a rising power, any with the exception of China, like a lot of the BRIC countries, the role that the American media plays in some of those, like, it's nuanced to different and that kind of stuff. And so let's talk about it from like, not only just like, how does Britain feel about us, which is, a long standing open question. And then versus how do like upcoming countries that just have a chance to, like have shows on their airwaves and stuff like that care about?

SPENCER: Well, yeah. I mean, it seems like there's not just the issue of how does UK feel about us, but also, how do Americans feel about America? In my lifetime, I've never seen Americans be more negative on America, I don't think that I don't have survey data to back that up. That could just be the impression of like, the circles that I'm in and just seeing people basically, view America as bad and evil and corrupt, and so on. I'm wondering, do you see that trend? Because it seems like if Americans think America is bad, that also might have an influence of like how everyone else perceives America.

WALT: Yeah, this is a really good question because like, on one hand, you have mistrust of elites and mistrust of institutions on the rise. And there's a lot of different reasons for that, because I think that if you look at the active disruption of institutions like that has been kind of a consistent trend for the past decade or two, whether that's through social media kind of ascending to kind of replace institutions like the media, or at least kind of subvert some of the expectations of what is from that, as well as also being able to kind of galvanize resistance and anger towards those thing. Gallup had their survey out recently, basically asking people like, “How are you doing? Are you feeling pretty good?” and then more people that didn't the entire history of the survey, which goes back to the early 80s said that they're doing great that they're thriving, they're making money. So like, it's the get sussing out the national mood is always like a little bit of a toughy. Just because I think that number one, we've had an extremely volatile couple of years, specifically owing to the pandemic but like, I think when you really kind of get down to it kind of like we're talking about earlier, it's not like, “Are they spending more time on this?” I'm spending less time on this. It's like, “Are they spending more time on this compared to this?” And I think that if you ask people, how do you feel about America, it goes ups and downs a little bit. But I think that if you ask people which prevailing global viewpoint do you tend to hold with, would you say the Russian viewpoint, the American viewpoint or the Chinese viewpoint, I think that that has remained rather stable, even during the Trump years, even in the places that specifically didn't jive well, with Trump diplomatically. I don't really particularly think that changed, right? I mean, you'll even recall, potentially, during there was that big G7 meeting in the first year of the presidency, were just some notoriety Angela Merkel and Trump got into kind of an argument. And there's a rather famous photo, and like you saw Shinzo Abe, who was then the President, rather Prime Minister, I believe, of Japan. He ran the show in Japan. But like he was a coming up alongside Trump, because again, they kind of realized that, in this particular situation, the ties that bind, our countries are a little bit more durable than one person or another.

SPENCER: That's a good point. So just in terms of okay, if you had to pick one country to put your votes behind, would it be the US or Russia or China? And you're saying that that's maybe been more stable, even if the opinion of America fluctuates? I guess what I'm noticing is sort of a brewing left-right divide and how people feel about America, it seems to me increasingly, the left feels that it's sort of bad to be patriotic, that like being patriotic is a right wing thing or something like this. Have you noticed the trend in this direction?

WALT: I don't think that the trend that you're hitting on is new. I think that the left has always felt anxiety since Vietnam about draping themselves in the flag. And I think that the right has rarely had qualms about doing so. And I don't think that that is related to how people feel about America. I think that that's how people feel about being boisterous. I don't really think that you're seeing at least widespread. I mean, you got to remember like, there's Twitter and then there's the rest of the world and like, I don't really think that it is as anxiety-inducing as anywhere else. Like the best firework shows on the Fourth of July, I believe, like are going to be in the in cities, which are overwhelmingly blue. Your call last year like LA in New York had fireworks quite a bit during during the course of the Fourth of July weekend. So I think that like outward celebrations in recognition of identities American is — I think that what you're kind of talking about a little bit more — is more a matter of taste than an actual matter of how people feel in the heart. How do you feel about that?

SPENCER: Yeah, it's interesting. Yeah, I'm not quite sure what to make of it. I mean, it's tricky, because every country is a mix of good and bad, right? There's almost no country that's pure evil, and almost no country is perfect. Okay, maybe there's a small number of exceptions, something like that. So within every country, you could always say, well, it has this history, and then these bad things, it's about these unjust wars. And it serves seems like there can be trends and like which aspects you emphasize, right? You emphasize the times when your country did really good things or times when really bad things. And I guess I do see it, there's sort of a fashion on the left right now of emphasizing the worst parts of America on the right defending America. But then, at the end of the day, maybe people on the left would still say, okay, but of course, like I pick America over Russia, any day, and so maybe, for all the internal critique that at the end of the day, this show on America side, I do wonder, though, if people were really wide spoken about how horrible America is, in America, whether this has has an influence on soft power worldwide, whether people listen to this, and, they're saying, well, even Americans don't like America, and maybe, kind of tarnishes the brands some extent? Or maybe it maybe it's just on Twitter and has very little global influence. Idon't know.

WALT: There's always like domestic issues, and then international issues that are domestic. Nobody loves, what about them more than the PRC these days, basically being like, Oh, well ignore what we're doing out in the province out out west, because you guys had slavery. So there's always like, what about this? What about that, and that's kind of a historical technique used by totalitarian states to kind of undermine the message of liberalism, like capital "L" liberalism, I should say. But like I also just don't necessarily think of like, I don't think that the left is emphasizing the bad parts about America. I think that like if you have a house that you like a lot, but like one of the little of the dry walls rotted in the bathroom and somebody's like, we need to acknowledge the the rotted drywall on the bathroom so that we can eventually fix the dry, rotted drywall in the bathroom. Nobody's trying to bring the house down, right? I don't know if it necessarily like it is emphasizing the bad stuff so much as it is tempting to kind of like you mentioned, there's no perfect country, kind of the urge to fix and address and that kind of stuff. I don't necessarily if that comes part and parcel with fundamentally rejecting the current home that we're in.


SPENCER: Okay, so again, switching topics totally, I was gonna try to make a transition.

WALT: But the best transition is switching topics. Totally.

SPENCER: Yeah. So I think you and I are both really interested in probability. I know you have a newsletter that talks about numbers and the media, what's the name of your newsletter? So if people want to find it.

WALT: Yeah, of course, it's called Num-Lock News, you can find it at like will get you there. It's hosted on sub sack at Num Lock dot sub sect FM, but it's an unlock like the key on the computer that forces you to use numbers.

SPENCER: So tell us about your interest in probability and why you think sort of it's a problem the way people usually talk about it.

WALT: Yeah, for sure. So I was a stats and probability math major in college, and I got into journalism, kind of through that. My interest has always been in using numbers to kind of tell stories and understand where kind of things are going. In the newsletter, one reason that I kind of do is that the gist is that it you know, it goes into seven stories a day, but specifically surfaces like the fact number buried in it, whether it's the amount of being spent on something, the percentage is something, that is something of a forecast or prediction for something. And I always find that it's just great to kind of elevate that, because I think like as a journalist, what we're always coming up against is people who want to better understand where the content of the story that we're telling us coming from. For a while you had people dismissing reports as fake news. And one reason I think for that, and like, again, I'd quibble with the with the structure of that. But one reason for that was that they kind of simply didn't understand how the sausage got made. And they disputed the story. And because they didn't understand where the factuality was emanating from in the story, they felt empowered to kind of dispute it publicly. Right? And I think that that's a cool thing that data offers us, which is that if you've sent her your story around a data point around the fact, if you say this intersection saw an increase in collisions over this year, that helps your story really saying more than if it's just like this is a dangerous intersection, right? And so, as a result, I always try to kind of find the nut of a story — what is the data being discussed. And so what I wanted to bring to y'all today, which I think is this really powerful idea that has really helped me, at least kind of understand where language is taking us when it comes to statistics, and why people have issues with it is these this kind of idea of words of probability. So like, the words of estimated probability is like the academic-ish term for it, at least when I've kind of come across it written about from that kind of setting. And the idea is like, if I tell you that there's a chance of rain tomorrow, or if I tell you, I maybe will be coming out to meet you at the bar tonight, where if I tell you, I probably will have a story on this next week. In my head, I know the probability is I'm trying to say when I say a chance of or maybe, or probably, I mean, a 30% chance, I mean, a 50% chance. And I mean, like a 90% chance, right? Whereas when I tell that to you, you might be interpreting those words differently, you might instead sign a chance to be 50% chance of rain, you might have maybe be 80%, and be expecting me to come join Smith at the bar, and you might say probably is just around 60%, like a little bit better than half. And I'm so interested in this because it really gets to the heart of communicating about data in a way that just really gets at the soul of what I what I do and like forces me to really ask myself time after time, “Am I actually accomplishing what I hope to accomplish?”

SPENCER: I think it's really interesting that our language is too ambiguous around probability concepts. And so it leads to basically miscommunication or misunderstanding. And you linked me to this really cool chart people can find it by structuring perception of probability words.

WALT: Yeah. Yeah.

SPENCER: And basically what it shows is that each of the different phrases like improbable are unlikely or highly likely, or very good chance has a sort of a distribution of interpretations that spans quite a wide range. Some of them are kind of more precise. Like, if you say better than even, that's actually pretty precise. Or if you say bad, even that's really precise, actually, yeah, but other ones can be really uncertain, like we believe, is the kind of very ambiguous or we doubt is very ambiguous. And if we're trying to communicate information, it's probably much better to just say, I'm 70% confident than to say, probably this will happen. Is that sort of your patients, do you think we should be communicating with numbers?

WALT: See, here's the thing. I don't know. I think that (and this is why I cannot bring this up) because I think that sometimes, when we talk with numbers, when we don't need to talk with numbers, it is bad. And it turns people off, and they actually don't understand what we're talking about. I think that knowing that this is a problem — that knowing that it's like a game of telephone, where you want to make sure that you're getting the message across as clearly as possible, because you know for a fact that something's gonna happen in the middle of garbled a little bit — I think just being like really cognizant, like, so I think I sent you this paper from the CIA because the CIA was one of the first people to kind of highlight this. And did you get a chance to read that?

SPENCER: I've looked at it before. Yeah, yeah.

WALT: So it's, like the gist of it for folks who are unfamiliar with it is — in 1951, the Central Intelligence Agency was concerned about Yugoslavia and whether or not the Soviets would invade up Yugoslavia. And so this report came out. And the key judgment of it is like, listen, we don't know what's going to happe, but we believe that the extent of a attack on Yugoslavia should be considered a serious possibility. And they had the meeting, they were like, “Alright, great meeting everybody. Point that across, everybody heard the phrase, a serious possibility.” And then like a week later, at the watercooler, they were like, “Wait, you thought that meant definitely gonna happen? And you thought that meant, not gonna happen?” And so then they all got back together again, they were just like, "Wait, what did everybody mean by that?" And they had to actually talk it through. And then eventually, they ran a survey of like NATO leaders being like, what do you think these words mean? And then they found that this distribution is so wide. And I think that just kind of like, as a communicator, I think that the way to kind of articulate that is like, really just give as much context as you possibly can. And don't hesitate to assign a number to it in general, but I think as like a reader, the thing that I was almost kind of preaches like skepticism, and if you see people make a forecast or a probability assessment, really think about what they're trying to get across, you know?

SPENCER: Hmm, yeah. Well, I suspect that partly why this happens is that people hide behind these vague words, right? If you say, I think there's 90% chance this will happen. It's much easier if you can falsify what you said. And if you say, I think this will happen. And then people were like, “Well, it didn't happen.” You're like, “Yeah, well, I just thought it was gonna do.” You don't mean it's like, well, if you say, 90%, that's a much, much harder prediction, to wiggle yourself out, right? Of course, there's always a 10% chance you're wrong, it's greater evidence that you made a mistake if you're wrong. And so I wonder if part of it is just people don't want to put a number down because then they can be proven wrong more easily.

WALT: So like, in this study, if it first of all look at efforts by Mosteller and Youtz, they basically just aggregated a lot of different surveys of this kind of stuff. And were able to kind of discern this, and like the computer thing called the interquartile range, the longest sort of is that they figured out which words have the biggest spread, like which words are truly meaningless when it comes to articulating, like what it means. And like, possible is an example of it like, well, it's possible. And the answer is that like, the median person thought, a 40%, but like, the range was, like 75%, to 50%. And so like, the word possible, genuinely means nothing, if you're trying to actually articulate what you get across. Whereas if I say, well, there's an even chance of something happening, you instantly know what that means, you know that the image transplants 50-50, you know, what a coin flip as a as a person, you're able to kind of follow through on it. And so I think that like being able to kind of suss out which words are suspect, like not infrequent, or like, improbable, like that kind of thing is a good way to kind of get a little bit more literate when it comes to how people are potentially tempted to kind of deceive you when it comes to that, knowing what they don't know and acknowledging what they don't know when to forecast.

SPENCER: You know I think another factor that might be going on here is that if we have hard data on something, we could say like, oh, yeah, 20% of the time that X occurs, Y also occurs, right? You know, I think a lot of people are very comfortable, in that case, just giving the number because you're just doing a calculation, and you're reporting the results of that calculation. But when you're kind of synthesizing a bunch of information and evidence, and coming up with your own belief, people have more discomfort assigning a probability to that, because it is more subjective. It's sort of somehow they arrived at it. They couldn't necessarily defend why they're saying 80%, not 75%, not 85%. Right? And an example of this is Toby Ord, who I recently had on the podcast. He in his book, The Precipice talks about the estimated chance of an extinction event or some kind of other major crisis in the next 100 years, and he actually gives it a probability Like, he could probably say things without why is that number but it's probably hard for him to defend more than six versus one in five versus whatever. So you from a it actually be like in the sort of mathematics world, should people report these kind of “subjective probabilities” that are just trying to take their own belief, map it to a probability that is mapped their their sense of belief to a number between zero and one, or in these kinds of situations where it is actually subjective? Should you avoid doing that? Is it somehow giving it sort of a false sense of precision, or being an abuse of sort of the mathematical notation?

WALT: Yes, this is such a good question. Because when you mentioned earlier, why don't we just say the number that we actually feel like, one of my favorite applications of this question is the weather report is that the weather report is probably the most reliable time that your average human being who isn't working in math or stats or whatever, will work with stats, right? If you are doing a barbecue on Saturday, and you would like to know whether or not you to do that indoors, you need to discern what's 30% chance of raining, right? But to your point, this is great, because there was a study that was basically done being like, in addition to like, how do you feel about these words, in the context of a weather report, how much do you actually think a slight chance of rain means there was this other chunk of it that was basically just like, alright, you see, on the on the weather report, there's a 25% chance of precipitation today. What does that mean? And for the most part, people didn't get that right. Some people said, “Well, one of the fourth of the area on the screen right now gets rain.” Other people were “Well, the odds of rain are this to this.” Other people were just like the proof the guy gets it right, one out of four times. And like, the actual reality is like out of 100 days like today, the simulation says that 25 of them ended. And as a result, I think like kind of exactly to what you were just saying is like, even when we put a precise statistic, and even when we have evidence of that precise statistic, there's no guarantee that people actually know what it mean.

SPENCER: Yeah, I could be wrong about this. But I have a suspicion that the weather report numbers are not true probabilities. But as a mathematician would think about them because, for example, I've noticed that if you look out like your some apps will give you like a long-term forecast, like a seven day forecast. And if you look at the chance of rain at seven days, I'll see it at like 80%. And I wondered to myself really, can you really be that confident with the uncertainty in these weather systems that seven days out, you really are able to make that higher production. My suspicion is that if you actually took into account all the true uncertainty in the systems, you would not be able to get such precise estimates at that distance out. And you'd have to kind of shrink them down to the prior of like, what's the chance it rains on that particular day of the year in this particular city. You know, you kind of have to smash it down. So I've wondered about that, whether these are even real probabilities at all.

WALT: Yeah, so my old boss, Nate Silver, wrote the book, The Central Illinois, very former effects on this, and he had been in there, there's basically the local TV weather guys will oftentimes inflate the probability of rain, because they don't want to be wrong, like the cost of a rainy day for your reputation where you are the reason that somebody did not think there was gonna be rain today and get there was was substantial enough that like across the board, when you average all out, your local TV weatherman is going to fudge the numbers a little bit, the fox five gets more angry letters, if the it literally rains on your parade is the prevailing idea. Again, it's been a minute since I read the book. And so that is like a deliberate fudge or at least like, again, when we're allowed to use these kind of weaselly words or these probabilistic words, we can kind of like, talk our way into that kind of stuff, right? However, I don't think that it applies to the National Weather Service. Like the weird thing about weather reports in America is that like, all of the actual data comes from the NWS, and everybody else is just interpreting it putting their flavor on it, that kind of thing. Right? And so I think that they actually, like there was this really terrific book called The Weather Machine. Have you ever heard of it?

SPENCER: No, I haven't.

WALT: Okay, it's rad, it basically talks about how we got to becoming to predict the weather. And it goes into the scientists behind it and like, what goes into the models. And just the simple reality that I learned as a result of that book and following this base a little bit is that the reason why we now have very reliable seven day weather reports, and we'll soon have 10 day weather reports, and the reason that we can predict hurricanes now 48 hours before we could two years ago, is literally, we put a thing in space, like there was not a thing in space. And then we put a new satellite in space. And as a result of that data gathering capacity, we actually just have far more immersive and better data to accommodate that kind of stuff. And so I think that the actual guts of the weather system are working pretty good. Like I wouldn't doubt too much if they don't say what they mean. Obviously, the interpretation of weather reports is kind of a matter of like, there's probably a difference between what the Weather Channel tells you and what the local guy tells you and then potentially what you find if you actually go to the NWS data, but like, my understanding is that like they're good and getting better.

SPENCER: Yeah, absolutely. My understanding is that they're reporting the percentage of simulations that Yeah, different outcomes. And I just wonder whether that truly is the same as the probability of the event occurring?

WALT: I mean, that's a good question. Because obviously, that's how most of these things work, right? If we talk about like, if you've ever read like a sports forecasting model, typically, they just run a certain number of simulations until you the percentage time so that comes up, Oscar models, like my run like that, like, that's how the all the like 538, and Upshot election models end up working. And I think that like that is just kind of one of the more reliable ways that you can convert a series of assumptions, data and belief into a probability in a way. And like, I think that you're right that like, is that are they exactly right on that? It's a worthwhile question. I think that they really have improved it materially enough. And I think that to do enough confirmation to like, again, they don't just kind of sit on their laurels, right, they go back and look at how accurate their predictions were in comparison to what actually happened. And that is what they're optimizing for, if you follow me. So I think that like, particularly for the weather system patterns, they have so much ample data of what was forecasted and what happened. And as a result, they're able to go back in and the model improves, if you follow me.

SPENCER: Yeah, absolutely.

[conversation paused]

JOSH: Hey, pardon the interruption, but we wanted to let you know that we looked into this topic after the episode was recorded, and found that the meaning of the probability of precipitation is not as straightforward as you might think. And we'll link to a couple of articles about this topic in the shownotes, for those who are curious.

[conversation resumed]

SPENCER: So another thing you mentioned, when you're talking about people assigning probabilities to words, is that in people's own minds, you suggested that they sort of know what they mean. Whereas like, when they're expressing it, they're converting it to an English word or phrase that you're losing information, I guess I would challenge that a bit and say that I think one of the advantages of expressing things as probabilities instead of words is actually forces us to be more precise than we might normally be. In other words, if you're trying to make a prediction, what's going to happen and you all you have to report in is I think this is gonna happen, or this might happen, then you actually don't even have the incentive to trigger get really precise with yourself, or do you mean 60%? Or do you mean 70%, or 80%, right, like, and I think I found, so I started to do various kind of probabilistic forecasting exercises to help prove myself with this, one of them is, I use this tool called prediction book, where when I have important things fire in my life, I make predictions about them, I make probabilistic predictions, and it tracks them for me. And that way, I can see how my predictions are and how calibrated I am like, if I say 70 to 80%, likely, it doesn't really have an 80% of time, if I say 70%, like zero happens suddenly for some time, and so on. And just going through that exercise of like having to pick that number, I find really helps fine tune my thinking and my ability to make forecasts.

WALT: Yeah, I think that when you work at FiveThirtyEight, you have to make progress all the time. It's part of your job. It's an occupational hazard. And so, I definitely understand the idea of when you say, here's what I think and you assign a number to it, that definitely kind of forces you into an honest assessment of where you're at. I think that's like why prediction markets are kind of really fun to watch. And I think like, provided that they're large enough in size can actually be rather useful when it comes to offering insight into market conditions and what have you. I think that that's really valuable. But again, I think that you are also right, where like when you force yourself to assign a probability to something, there is a point at which you are just kind of converting into a gambling person, and potentially just trying to get closest to the pin, I suppose. Whereas like, again, I think acknowledging uncertainty is a very, very hard thing. And it's a hard thing for journalists to deal with. It's a very hard thing for data journalists to do, because it's not in our blood. But like, we want to put a good number on a thing. And so I think that like being real assertive about like, Listen, this is what I'm putting out there. Do I think that there's uncertainty involved? Yes, here are the five sources of the uncertainty in it. And then just kind of doing what you can to kind of whittle those down until you've got as few as as you're comfortable with.

SPENCER: So what it was, like, if I'm FiveThirtyEight, in terms of making predictions, like how was that actually implemented?

WALT: Oh, I mean, we all kind of had different degrees of expertise. A good example is that my friend Neil and I both like hockey. And just recently, we were trying to decide who we thought would win the cup and whatnot. Using his extremely wonderful, Elo model that he had designed to replicate the NHL during this weird pandemic season.

SPENCER: You have a model they use for deciding how good people are at chess, it gives you points based on who you beat, based on how good that other person is.

WALT: Yeah, totally sorry, I should have explained models for Elo. It's a system that was designed originally for chess back in the early 1900s, I believe, but has this been adapted into many of the different sports and the ideas like if you beat somebody who is rated lower than you, you don't really gain all that much because that's not really new information. If you beat somebody who's rated rather higher than you, your score goes up and their score goes down in the zero-sum settings. The reason that it's advantageous for a thing like chess is that you'll have a limited number of matches per year and not a lot of repeat matches So for this kind of system, your score can change as it goes, if you take a year off from chess, then you can come back and kind of your score will not have degraded in the interim typically. And so for a system, like, the hockey league, it's just a very advantageous place to kind of start, cuz it's a great system that you can immediately adapt to do like, again, like, just like I did one for my fantasy football league, it's that easy off the rack to kind of implement if you have the stomach for the stats for it. But either way, typically, we'll just kind of use that and then use that as a framework of like, well, do I agree with this, what isn't being captured by it, Elo doesn't capture who's playing, it just captures wins and losses, and oftentimes the score, and so, if you know a pitcher is injured, and you can see the major league baseball Elo score, then that they're probably a little bit overstated, because, you have this information that the model lacks, and anytime that you're able to kind of interrogate what information do you have that the model lacks, and then build your estimate from that, that's how you can kind of use stats for I betting purposes on that kind of?

SPENCER: Got it. So how do you think that affects the culture of FiveThirtyEight, like that kind of betting culture?

WALT: I wouldn't describe it as betting culture necessarily. I would say, that is informal, generally, and again, it's been quite a while since I was there on that front, but I don't necessarily think that would be fair to call it betting culture.

SPENCER: Ok got it, got it. Maybe a culture making predictions?

WALT: Yeah. I mean, I think that's pretty baked as the into the business, right? Obviously, the site became rather prominent, immediately, just because of the predictions around the then 2008 presidential primary. Damn, what an error. And then since then it evolved, like being able to construct an argument using stats and all that you kind of have to bear. I think that that's a very good mentality that folks in general should pursue, I think that, it can be useful for a lot of different things, I think it kind of forces you to be very honest with what isn't being captured. it kind of requires you to be very straightforward about how you understand the system before it's in motion. Right? And I think that it's just kind of a valuable kind of way to perceive the world.

SPENCER: So what would you want experts to do when they're making a prediction on TV or in media?

WALT: This is really kind of valuable question because I think that there was this other kind of component when I was talking to you earlier about what I thought was interesting about this, which is that when you look at the probabilistic words, and you look at how and what they mean to different people, you know, you notice that there's very specific things that mean, like 190 to 100%. Like I say, something will almost certainly happen. Pretty much everyone understands that. I mean, between 90 and 100% of the time, something happens, right? If I say that there's almost no chance something happens, then everybody pretty much intuitively understands that there's like 0-5, less than 10% chance of that happening. If I say it is even better about even everybody pretty much understands it's a coin flip. But then there's like these liminal spaces in between, where we don't really have a word in English, or at least a word that is correctly understood by everybody at the same time. That means I think that there's a 35% chance that this happens, there's no word that I can tell you that everybody will kind of buy into. And so I think that if you are making a prediction on television and you think that something has a 95% chance, or a 5% chance or a 50:50% chance, you should definitely make that prediction on television, why not? You will absolutely get across what you are trying to say. But like, the real authority, is when there's a 30% chance there's a one in three chance of something happening. And for whatever reason, we don't like any of the words that one would use just in conversation, like we think that something's unlikely or probably won't happen, or we don't think that there's a great chance that it'll happen. Like those all mean, anything from like zero to 50%. And it's just so hard to properly articulate what you mean. And so I feel like that's where people get in trouble. Because that's where a lot of stuff is like, if you remember 2016, there was a ton of blowback to the presidential forecast, because like, the good ones said that there was a 35% chance that Donald Trump would be elected the president. And lo and behold, that's what happened. And so it was just difficult to get through. Because no matter what you kind of say, if you say that there's a 35% chance that Donald Trump as President, the Democrats assume that it means 50:50 year that means 100% chance of not right. And so as a result, you're kind of like backing your way into a corner when it comes to how people will listen to you. Right? Like they will hear what you're saying, but they will listen to you and interpret something differently. And so I think that that just kind of sets people up for difficulties when they do this, you know?

SPENCER: So it sounds like you know, one world we could we could hope to live in as one where experts and forecasters just always use percentage chances and I'd be pretty happy with that world. I don't expect it to happen anytime soon. Another possibility is that people could be trained to use words that at least have a more precise meanings, right? So avoiding these words like possible where nobody really knows what that means focusing on just the ones that sort of have tighter meetings were more established narrower ranges. But then as you point out, or interestingly, there are spaces in the probability space that don't have words around them, like 30%, I guess you're suggesting. So then how would you communicate that? And so maybe the answer is we just have to use percent chances if we really want to be really want to communicate well, though, I don't know how on earth we would get from where we are now to a culture where that was, was a normal thing to do.

WALT: Yeah, I think that the word in the English language for one to three chance is a one in three chance. And like, it just it's hard for that thing, because like, you can't like be like, Well, imagine rolling a dice and you get like a coin flip every flip coins in their whole life, right? But like real anytime that you got to use a, like a casino metaphor you're already losing. And so yeah, I think that it's absolutely a struggle. But it's one that I always enjoy whenever this kind of thing comes up, because it kind of — well wait, let's make sure that we all understand what this means. Because I went into several predictive events in my life, whether it was trying to predict the Academy Awards, or whether it was trying to like even just being a fan of a football team and wanting to that team to persevere. How do you find a way to understand the stat involved and at a certain point, so you're saying there's a chance can mean a whole lot of different stuff.

SPENCER: So Walter, thanks so much for coming on. If people want to learn more about other work you're doing what's the best place for them to do that?

WALT: Yeah, for sure. So I'm on Twitter @WallTiki. I am the senior editor for data at Insider. And so you can find some cool stuff there. And then everybody should check out the newsletter. The newsletter is Numlock, it's daily weekday morning free, no Ed newsletter, and then Sunday's there's some things for paid subscribers, but everybody should check it out. It's called Numlock. You can get it at And yet, don't hesitate to get in touch. This is a ton of fun. Thank you for having me. I love the idea of doing like deeper looks into these kinds of topics. I've really enjoyed the depth that you're able to get into on this podcast. It feels like so much of the way that we talk about stuff has to be an abbreviated segments. And I really enjoyed getting to talk about probability today and also comic books to my favorite things.

SPENCER: Yeah, I feel like podcasts give us really unique opportunity to really dig into topics and get kind of the longform experience and because there's, you know, podcasts on every conceivable topic now. The sky's the limit. If you want to dig into some topic, you can probably find a podcast that will go into. Thanks so much.


JOSH: A listener asks, What is your educational background? And what did you do before you started Spark Wave?

SPENCER: Some education about background, I got really interested in high school in computer programming and to some extent math. I then went to Columbia University in New York where I started Applied Math with a minor in Computer Science. And then later on, I eventually did my PhD in Applied Math at the Courant Institute at NYU, where I focused on sort of the mathematics of AI and machine learning. I will say though, that I'm a pretty extreme anti-credential list. So I really don't care about credentials at all. And you may have noticed that in that we don't actually keep the credentials of our guests in the episode. We do put them in the show notes, but we don't say them on air because we want you to really think about what the person's saying. Do you believe what they're saying based on what they're saying not based on what credential they have, in terms of what I did before I started Spark Wave. I was in my PhD program at NYU. And prior to that, I actually started a quantitative investment firm by design an algorithm for buying and trading stocks.




Click here to return to the list of all episodes.


Sign up to receive one helpful idea and one brand-new podcast episode each week!

Contact Us

We'd love to hear from you! To give us your feedback on the podcast, or to tell us about how the ideas from the podcast have impacted you, send us an email at:

Or connect with us on social media: