JOSH: Hello, and welcome to Clearer Thinking with Spencer Greenberg, the podcast about ideas that matter. I'm Josh Castle, the producer of the podcast, and I'm so glad you joined us today. In this episode, Spencer speaks with Will MacAskill about longtermism, Bayesian reasoning, and strategies for calculating expected value.

SPENCER: Will, welcome!

WILL: Thanks for having me on.

SPENCER: I'm really excited to talk to you about an incredibly important topic, which is, how do we do the most good and in particular, an answer to that topic that you've written a book on recently. So, I recommend people check out “What We Owe the Future," Will's new book. And the topic is longtermism. So do you want to just explain what longtermism is? And then for those that have never heard about it before, we'll talk about what is this theory. But also, for people who have already heard about it a bunch, I want to go really into the nuts and bolts of it in a way that probably people haven't experienced. So hopefully, there'll be something for everyone in this episode.

WILL: Terrific. Longtermism is about taking seriously the sheer scale of the future and about taking seriously just how morally important it is to make sure the long-term future goes well. And it's about thinking about the events that might happen in our lifetimes that might have an impact, not just for the present generation, but also on all future generations. And then finally, we take action to ensure that we can carefully navigate those challenges that we face in our lifetimes that will help set humanity on to a better path and help steer civilization so that things go better rather than worse over the coming hundreds, thousands or even millions of years.

SPENCER: One way I've heard people describe longtermism, and I'm curious if you agree with this characterization, is as an ethical stance that gives priority to improving the long-term future over, let's say, trying to improve the present, and arguing that future people matter as much as people today. And that because there's going to be vastly more future people if society survives, if we don't go extinct, then the future may matter a lot more than the present.

WILL: I distinguish between longtermism and what you could call "strong longtermism", where in the book and kind of in general and just defending the weaker form, which is saying that trying to positively impact the long term should be a key priority of our time, like at least one of the things that humanity now is really trying to do, but not necessarily the key priority, not necessarily saying it's more important than all the other things we could be doing. The strong longtermist view would be saying, "Yeah, it's the priority, it's the number one thing we should be focusing on." And I am sympathetic to that. I think it's more likely than not that that's true. But given how things are at the moment, where we spend almost no time or attention or money on trying to help the coming centuries or thousands of years go well, that even the weaker claim, just "this should be one priority among many" is more than enough, I think, to radically change what we're doing as a society.

SPENCER: I suspect it's a lot easier to convince people that it's one thing among a number of things that we should be thinking about, and that we should be prioritizing more on the margin, given where resources are invested saying, "Hey, if there's things we can do to impact the long term future, to make the long term future go better, we should do some of those things now." But the strong longtermism claim, I think it loses a lot more people, saying "it's the moral priority." And so I'm curious, you said that you think more likely than not strong longtermism is true, maybe you could lay out the case for why you think that.

WILL: So the case for thinking that is primarily based on just how vast the future might be where (like you say) I think future generations should have very similar or the same moral status as people in the present. I certainly don't think that time alone should be a reason to discount someone's interests. We do have some additional reasons for caring about people in the present: we have special relationships to people in the present, people in the present have benefited us in a way that can give a kind of obligation to repay that benefit. However, people in the future have moral worth too–very significant moral worth. And the future could be really big, where if we live as a typical mammal species, that's like a million years or 700,000 years to come. If we last until the Earth's no longer habitable, that's hundreds of millions of years ahead of us. And if we can think about time spans even longer than that, if we managed to live not just on Earth, then it's like many billions of years that could be in our future, even trillions of years. And if you then think about that, and also just population size as well, even if we're continuing it just the same population size as we have now, then we're thinking of trillions upon trillions of people who are yet to come, future generations outnumbering us by 1000 to one, even if we're using some of the most conservative estimates of how long civilization might last or humanity's life expectancy. And so, if there are things that can impact that future, it's just impacting far larger numbers of beings that have moral worth. So the size of the impact you're having is just much greater. And then finally, I think there really are things that we can do that will positively impact that long term future, such as by reducing the risk of extinction, or reducing the risk of kind of unrecovered collapse, or by influencing the values that guide the future. And so increasing the value of the future, even in those worlds where we continue to last a long time.

SPENCER: Right. So to summarize that, basically, if you do the math, there could be many more people in the future than have ever existed (certainly, than exist right now), and those people matter as much or even close to as much as people today, and there's some way to actually have a positive impact on those people. And then on top of that, those people could exist for much longer than the current civilization, maybe millions or even billions of years in the future. You just do the simple math calculation, and you're like, "Oh, actually, most of the value is in the future." Is that a fair summary?

WILL: Yeah, that's right. Basically, for whatever you care about (that can be wellbeing but also knowledge, art, cleaning, just society, and exploration), almost all of that is still yet to come. We're very early on in civilization. We're much more at the beginning of history than at the end, at least if we don't cause our own extinction. So that means that impacts we have on future generations, or if there are things we can do to positively impact the long term, that's just of enormous importance just because so many more people are being affected.

SPENCER: You say that you think more likely than not that strong longtermism is true. But it sounds like there's some uncertainty there about whether it's just AN important thing to work on or the most important thing to work on. Where would you say your uncertainty stems from?

WILL: The biggest thing is just a kind of appreciation of how little we know about everything [chuckles]. So both how little we know morally and how that could change as we did further moral research over the coming decades. And also just what we know empirically, as well. So if we learned over time that there are things I can see us coming to believe that I don't currently believe, but wouldn't be crazy to meet if we figured that out in the coming decades or even longer, so one could be just like there actually aren't any things that we can do to, even roughly predictable way, impact the very long term. So perhaps there's really no way to predictably reduce extinction risk, perhaps there's no way to guide future values over such a long time period. That could be one thing I can imagine me believing given sufficient further research. A second thing is, perhaps, there's just some annual risk of extinction of civilization that is just very hard to eliminate entirely and that will just keep going for a very long time. So even if there's a 0.1% chance of extinction every year as we look into the future, that means that we do not have a life expectancy measured in the millions of years. Instead, it would be measured in thousands of years,

SPENCER: Where you'd have to be incredibly low each year to get to millions of years of continued existence, right?

WILL: Exactly. So it has to be the case that we can get to a state where we lower the risk of civilization-ending catastrophe to some very low level per year. And now I think that's certainly on the tip, like actually pretty plausible, but you could imagine as a very mature, well-developed civilization that really got advanced technological power and using that power to do good so that they deliberately act so the risk goes very low. But it could be that there are just some structural reasons that I am not aware of why that's actually impossible. And that would really eat into how big we think, in expectation, the future should be. And then maybe the final thing is just that this argument has a consequentialist kind of flavor. I don't think it is consequentialist. I think any moral view should care about the consequences; it's just whether only the consequences matter. It still feels at least somewhat consequentialist flavored, and perhaps that's systematically the wrong way to think about ethics. Instead, perhaps, ethics is about acting in a virtuous way or just kind of avoiding violating certain constraints and acting justly with respect to people with whom you have special relationships. And the idea of this “impartial good” that we should be trying to improve, trying to increase the value of, perhaps that just doesn't make sense at all. If so, then I think much of the motivation for strong longtermism would at least potentially fall away. There could be other arguments as well, and I think those arguments might justify the weaker form of saying it's a priority. But I think it might make it a little harder to say it's the strong claim, like the most important thing that we could be doing right now.

SPENCER: I really appreciate how nuanced you are on this. It's such a relief to see someone who's not just pushing their view saying, “Oh, my view is definitely correct.” But more like really understanding all the details. I wouldn't expect anything less from you. But it's really rare to get that.

WILL: Yeah, it's funny. I get this quite a lot. People say this, and I feel a bit almost confused, because I don't understand what it would be like to inhabit the psychology of someone who was just super certain and doctrinaire about things that are just so intrinsically complex.

SPENCER: [Chuckles] It just shows what a bizarre person you are, Will.

WILL: Maybe, and this is just a personality trait. I remember as a teenager, people would have these conversations about politics, and they'd have some really strong views like about the war in Iraq or something. And they just be like, how on earth can you have such a strong view? This is just this incredibly complex topic, and you're not an expert in this topic. And it just baffled me and it still baffles me today. So yeah, I still feel a little bit surprised when people say that because, surely, this is just the appropriate way to think about the world.

SPENCER: Yeah, I guess there's psychological quirks of the human mind that make it so when someone's advocating for a position, they tend to advocate for it as though it has no drawbacks, and as though it's certain even if it's arbitrarily complex. It's just kind of surprisingly rare to see someone who's advocating something, but being like, “Oh, yeah, but maybe it's wrong, and here are reasons it might be wrong.” So yeah, I think that's a really wonderful trait.

WILL: I mean, I feel terrified that I'd be kind of onstage advocating for ideas that might be false, so I really want to figure it out if they are.

SPENCER: Yeah, that makes a lot of sense. Longtermism is a big idea that has potentially very important consequences, right? We don't want to be wrong about that. [Laugh]

WILL: For sure, exactly.

SPENCER: I posted on Facebook and Twitter, just asking people what their strongest arguments against longtermism are (I was just curious). One person just kind of basically wrote this long thing about how terrified they are about the idea that today we would dictate the future of humanity and how scary that would have been if, somehow, people 500 years ago could have determined the world today, and how messed up that would have been. (It's just that there's a lot at stake in these things.)

WILL: Yeah, it kind of is terrifying, and it's an argument in favor of longtermism. Because whether we like it or not, it seems that there are things happening in our lifetime that will impact the course of the long term future, such as risks of extinction from engineered bioweapons, third world war, lock in of certain values via AI, or just takeover by AI systems themselves. And if we do nothing, then we're just constraining the future in certain ways. Whereas what we can aim to do is to give as much optionality as possible to future generations. So, we close off as few possible futures and try instead to bestow a state where people who are hopefully far smarter and better than us can make decisions about how their future goes.

SPENCER: So there might be a default bias thing going on here, from your perspective. You may view it as lock-in might happen just automatically, and therefore we shouldn't prefer that state just because of the default. We should say, “Wait, but do we want to lock things in? Maybe it's much better not to lock things in. But to not lock things in, we actually might have to take actions to affect the far future.”

WILL: Exactly.

SPENCER: So going meta for a moment, how do you think about handling this uncertainty you have about longtermism? Does that push you more towards a larger portfolio approach to doing good? What's your response to that?

WILL: Yeah, it pushes me in two directions. One is in favor of just intellectual inquiry, trying to get us a better understanding of things. And that's for both “Is longtermism true” but also “What follows from it?” which I'm considerably more uncertain about. Maybe the best thing we can do is AI alignment, maybe it's the reduction of bio-risk. I would not be surprised if I were surprised by some new priority that was even larger, or if some existing thing that we think is a priority is not as great — it used to be that future-oriented people and people worried about existential risks thought that nanotech was the big existential risk, and now very few people think that. The same could happen over the coming decade with other risks that we currently face — so that's one thing that follows from being comparatively uncertain. A second is that it favors, kind of broadly, building up well-motivated resources. So I think that growing the effective altruism community, in general, such that the world has more resources that are careful, reflective, cooperative, altruistic, and morally motivated seems like that's going to be useful even if we significantly change our view in one or two decades time. And then finally, I think it favors what you might call “robust Effective Altruism.” So, doing things that look good on a wide variety of worldviews for different ways the world could go, rather than just some very narrow view, even if that's your best guess. And it's kind of striking that many people, who make big decisions in practice, actually kind of favor this (what they call robust decision-making) — it's an idea that came from RAND originally — over doing the naive kind of cost-effectiveness or calculation or something. And the idea is not like that Bayesianism is incorrect at the fundamental level, but more just when you're trying to put it into practice, taking into account the fact that there are many, many scenarios that you haven't considered, perhaps this is actually something that's better in practice. And maybe that means like, reducing the risk of war is really good because war, especially great power war, is just really bad across a wide variety of scenarios. And it's bad even in worlds that we haven't even thought about. Whereas certain other things you might do that are much more targeted, perhaps, might only be good on a narrower selection of ways the world could go.

SPENCER: A discussion I've gotten into with a handful of people is around, “Should we just try to do the most high expected value thing, regardless of how uncertain we are?” And so I've heard people argue, “Look, even if our methods are terrible, even if we plug numbers in a spreadsheet that we're extremely unconfident in and are just guesses, we should just do the thing that is highest expected value according to our best guess, and then put all resources into that because that is maximizing expected value.” I feel a lot of discomfort with this argument, but I think it's a little bit tricky to critique what exactly is wrong with it. I'm just curious how you react to arguments like that, that say “Even if we're really unsure, if you think the highest expected value thing is, just like strong longtermism, just put everything into that.”

WILL: I think two things about it. One is that most people in the world just don't use this style of reasoning nearly enough. Most people never put numbers on things and try to do a back of the envelope calculation about how much good they could do via different things. And even in my own case, I might be really excited about some project, and then I try to put numbers to like how much it would cost, or how much time it would take up, how much good do I think it would do. And in the course of doing that, I become a lot less convinced or think it's a lot less exciting. However, do I think that's the only thing you should do and you should pay no attention to heuristics too? No, I don't think so. So, if I'm making a decision or advising someone to make a decision about their choice, for example, one thing that they should do is just try and explicitly put numbers on how much impact they think they can have in different paths — it's easiest if you're thinking about earning to give; how much money you can make, how much good that could do — and give it a go, try to compare that with the impact you think you could make by doing journalism or research or going to politics or policy. But you should also just write down, like pay attention to a whole bunch of heuristics like where you're going to be learning the most, what seems to play to your special strengths and so on. And I think it's just totally justified to do that because — let's even assume the fundamental truth is that you should maximize expected value, but that means for every single way the world should go, you have a point estimate of the probability of that, given your evidence. So if you're doing this in a perfect way, then it'd be like, “Okay, it could go in this way, and I have 0.3729468 credence in that.” And then this other way the world could go and you've got precise values for all of those things. That's what the perfect ideal Bayesian reasoner would do, and obviously, we're not doing that when we write down these numbers. Were doing something that's a very far cry from that, and so — you can see good heuristics that seem like they've held up well in practice that we've kind of learned from experience that seem to produce good outcomes as a counterweight to the fact that we're these incredibly imperfect reasoners that can get kind of misled by these naive cost-effectiveness calculations.

SPENCER: So does that argue in favor of just spreading out efforts more, even if your best attempt at expected value calculation says this type of intervention (let's say strong longtermist interventions) are higher impact, that you still want to spread across other ones, because you know the limitations of your thinking and you know the limitations of this kind of methodology?

WILL: I mean, probably, I don't think you'd have to guarantee that. But if you think, in general, different areas have diminishing returns, and then on your kind of naive calculation, Cause A is 10 times better than Cause B, probably if you did a full calculation and you were this perfect Bayesian reasoner, it wouldn't be quite as extreme. Once you've got all of the further considerations, then you could best back towards the mean a little bit. — I mean, it's very hard to put this into practice because, okay, I do this cost-effectiveness calculation, Cause A has higher expected cost-effectiveness than Cause B. Oh, I should take into account the fact that I'm a poor Bayesian reasoner. So actually, I'm probably overestimating. Let's say it's only five times. But then that's me still being a poor Bayesian reasoner. Should I regress it again, and then indefinitely? I think probably no, you can take it into account once. — But it would mean you get far fewer radical differences and cost-effectiveness between things. It also might mean that you just do different things, where if there's one thing, given the things you can foresee, looks really good with a 10% probability or something, but you're only seeing 10% of the possible space of the way the future could go. Whereas there's something else that just looks like it's making the world better, basically, no matter which way the world can go, then you can kind of infer perhaps that this second more robust thing will also be good, even in those ways that you've not predicted the world could go. And that might mean less like, “Oh, I'm distributing more across different cause areas that I'm already familiar with.” But actually, maybe I should be investing into different causes, things that actually don't even look best on any particular worldview I have, but look really pretty good across many of the worldviews I have.

SPENCER: This reminds me of the question of how you should estimate the chance that something will happen. We have a nice mathematical theory of this; you can use Bayes rule, and like you said, you can have priors over everything. And you could try to explicitly calculate Bayes' rule for daily things in your life, like, should I go to this restaurant or that restaurant. Now, I'm pretty convinced that if you actually tried to do that, it would not lead you to be a very good decision-maker. Because while in theory, in the absence of computational constraints, if you were a perfect reasoner with unlimited computation, you could do this calculation to get really good outcomes. But due to the vast limitations of our brains and the way our brains work (like the sorts of things we are good at), trying to do these calculations (decide where to go to get dinner) are actually very bad in practice. And I worry this comes up also with things like, “Okay, I put a bunch of numbers on these things to calculate which of these interventions is best, and I decided this one has a higher expected value — while I completely agree with you that it's a useful exercise, and I think it often brings to light important information and questions and is a good way to become more skeptical of something that seemed promising, because you could catch things in that in that process. They're like, “Oh, wait, this is actually not as good as I thought.” — that also trusting the output of it can be quite iffy, for similar reason that trying to do complete Bayesian calculations on where to go to dinner is iffy, which is, in my experience, when I've looked at people doing these calculations, often I will notice like, “Oh, on step three, I think their assumptions are not valid and invalidates the entire process of the rest of their data reasoning,” but they seem to not think that. And so the output, to me, is just like sort of random noise. And I think I usually feel that way when there's like more than 15 variables in the model or something like that.

WILL: Well, yeah. There are formal reasons for thinking this. Do you know the optimizer's curse?

SPENCER: No, could you explain?

WILL: Okay, here's the thought. Say, I'm deciding between 10 courses of action, and I do a cost-effectiveness analysis for all 10 of them, but there's some noise. And then I pick and do the one that has the best naive cost-effectiveness grounds. And when I say there's some noise, it means that I can just make errors in how I'm reasoning about these things, perhaps, because I don't really know what the true parameter values should be or just because I make mistakes along the way. If I do that, then it's very likely that the thing I'm doing is not as good as I think it is, because sometimes the noise will bias estimates upwards, sometimes it'll bias estimates downwards. But I'm looking at just the one that I've estimated to be best. And if it has ended up estimated to be best, probably that's in part because noise has helped it rather than hindered it. Now, that doesn't necessarily mean that it's not the best thing. But it does give a reason for thinking that, in general, the thing that you've estimated to be best via this kind of process of explicit cost-effectiveness calculation is not as good as you think it is. That's one kind of systematic way and even formal reason for having a bit of a skeptical attitude towards benefit cost calculations, especially when they involve many made-up feelings or very non-robust parameter values.

SPENCER: I think that's a great point, and I totally agree with that. But I would say that I often lose these models at an even more fundamental level, where I just think some assumptions in the whole process were wrong, and then it renders the whole output meaningless. I'm not saying this is always the case. I think it depends on how much philosophy is going into making this calculation (like these difficult philosophical questions that people could probably have debated for 2000 years). And additionally, how many really difficult empirical questions are involved in these estimates. And often, I find that I just lose these models at some point in this process.

WILL: For you, does this go as far as putting a question mark over longtermist arguments in general? So the argument that we mentioned a few minutes ago that there's this enormous amount of value at stake, you can reduce extinction risk by a certain probability with a certain amount of money, or decrease the chance of bad value lock-in with a certain probability or a certain amount of money, does that argument move you at all? Or does it move you a little bit, but not that much? I'm curious.

SPENCER: I do find those arguments somewhat moving. I don't actually view that as the kind of argument I am critiquing really here, because the kind of argument I am critiquing tends to be more like there's tons of different variables involved. What I like about the longtermism argument is that it's nice and simple; you can see it, you can keep it in your mind and kind of poke at it. I do think that there are some interesting critiques of it. I neither view myself as a longtermist nor a shortermist. I guess my attitude tends to be that I think there are a whole bunch of things that are really important. And I think we should work on all those important things. And I prefer a strategy where we divide up efforts across the important things, rather than go all-in at the thing that we think is currently most important. And part of why I view it this way is because I think that the thing that we currently think is most important, at any given moment, is pretty likely to change. And I think it is just not a robustly good strategy to go all-in at the thing that you currently think is most important. I see this occurrence or the micro of certain effective altruists who are trying to do literally the most important thing. But then like every six months, they changed their mind about what that is, and it leads to a series of accomplishing nothing.

WILL: Is the “we” there kind of humanity or is that the effective altruism community? When you say like, we should do many different things.

SPENCER: Both. I guess, the way I think about it is like if someone finds something really important to work on and is trying to work on it in a way that seems plausibly quite effective (or at least in expected value sense), I feel very good about them pursuing that. For example, take existential risks. It seems really important to me; there are straightforward arguments of why it would be really bad if humans went extinct or if humans got locked into a totalitarian regime that we could never get out of, right? You don't even need longtermism. Longtermism is one way to get there, but that's clearly really, really bad based on almost any view. So if someone's like, “I want to work on this.” Great, that's fantastic. But from my point of view, if someone's like, “Hey, you know what, people are living in extreme poverty.” That's really, really bad, and there's a lot of moral views that say that that's atrocious. If someone's trying to help that really effectively, I feel good about that, too. I think on these big philosophical issues, I feel very uncertain. And I also sort of have a heuristic against the all-in kind of thinking when there's so much uncertainty.

WILL: I agree, especially if there are all sorts of knock on or flowthrough effects. Sometimes you get these arguments saying like, “Oh, the cost-effectiveness of one charity could be like 10 billion times more than another because one's focused on making sure the long term future go well, and the other one isn't.” But the thing is, as long as there's basically any influence from things that are good in the short-term to things that are good in the long-term, it could be positive or negative (I'm just talking about magnitudes rather than sign), then it just becomes very hard for such enormous ratios of impact to really hold. And in the case, when I've looked back at decisions I've made in the past, such as co-founding and building up Giving What We Can, rather than, let's say, going all-in in 2009 working at the Future of Humanity Institute, that seems like a robustly good thing for me to have done. And it seems good even on the longtermist ground. And I think the reasons for thinking that it might not be just a total fluke that if you can really smash it at one area, even if that area is not particularly related to the thing that you ultimately think is most important, if there's some positive flowthrough from the thing you're really smashing at to the thing that you think is really important, then that can be better than doing this mediocre job at the thing that's kind of more directly aimed at the thing you think is most important.

SPENCER: Yeah, I can definitely see that.

[promo]

SPENCER: I'm curious to hear your reaction to the way I think more broadly on this topic. You mentioned before just how much uncertainty there is about things, and this is one of your hesitations. There's so much uncertainty about things in general, and I take that very, very seriously. I think that a lot of philosophy is too hard for humans (at least humans in today, and maybe one day humans will be smarter). And I'm not even the best philosopher. And I don't know that the best philosopher can answer these questions yet. So I think that very seriously, like, “Hey, a bunch of these questions are, actually, over the head of humanity.” And second of all, I take empirical uncertainties very seriously. I just think the world has a level of complexity that's staggering. And so our ability to estimate things is super high. And I think what this does is it puts me much more in a space of, let's try to do a bunch of plausibly really good things based on different heuristics and different considerations and different answers to major philosophical questions. And sort of cover the space, if you will, of major uncertainties and try to do things that are good, according to all different reasonable ways you could go on these different questions. And by we, I mean, ideally, humanity, but especially the effective altruism community, because they actually really want to do this stuff. So that's where I come from on this. And so yes, I think it's great to have some people who are going all-in on strong longtermism. But I also think it's great to have some people that are going all in just alleviating suffering today. And then other people are just taking completely different perspectives, as long as they're really trying to think what is really good to do, given sort of certain heuristics or certain considerations.

WILL: Yeah. It's tough, because it's ultimately a quantitative matter where, for the effective altruism community as a whole, there should be, I think, a certain amount of just, “Oh, cool, we can just make amazing progress in this particular area.” There's an argument for this that looks wrong in a lot of heuristics, even if it doesn't look as good on the naive cost-effectiveness grounds. But we'll get lots of learning value from this. And maybe it is the moral views you don't find as compelling, like looks really good. But then there's this question of what fraction of EA should be like that. And it sounds like you may think we should just be doing loads of different things, like hundreds of different cause areas and styles of projects. And hopefully, at least some of them are hits and big wins from the perspective of what's actually true. I think I'm going to be more inclined by a mixed strategy of some fraction. I think, actually, you maybe told me about this many years ago, but these ‘one heuristic for multi-armed bandit problems' is that you do the slot machine that you think has highest return a certain fraction of the time and the other fraction of the time you just do it randomly and see what happens. I think I'm kind of attracted to something like that as a heuristic for effective altruism as a whole. There's a certain amount of people and resources that are going for, like, exploratory things. Maybe we don't even know exactly what we're going to learn, but it would be interesting. And we think there can be big payoffs. And then, a very significant fraction, I would say the majority, is more targeted wherever we think is most important, although still spread over the number of causes and ideas.

SPENCER: I think that's called the epsilon greedy strategy, where like epsilon percentage this time you do this explore thing. And then the rest, you just do the highest expected value thing. So it's an interesting question of how top down should effective altruism be. And I think the way that you're thinking about it is, from a top down perspective, the community or the leaders or you have a view on what the highest expected value thing is, and we want to put a lot of resources into that. But then we still want this exploration going on so we don't miss better opportunities. Because if you go all-in on one opportunity, you might just completely be unaware of other ones or be equipped to take them if they come up because your whole community is oriented to one approach. Is that fair? Is it how you're thinking about it?

WILL: Yes. I wouldn't say top down. I think it'd be really bad if EA was structured like a totalitarian state or something, or even structured like a company. But there's something in between, which is like, what's the culture, what do people generally think is the right thing for EA to do? But then everyone's ultimately making up their own mind, where if I was writing on the forum, the arguments I would make is something like “We should have 75% focused on what we think are the most important things and 25% kind of open, interesting exploration.” And maybe if those arguments are successful, or then you've got lots of people making their own decisions about what to do. Like, do they think they should be one of the more exploratory people? Or do they think they should just double down what they think is most important? That's kind of in between top down and bottom up. It's top down in the sense that I'm presenting a kind of vision for the ideal allocation of resources. But it's bottom up in the sense that every individual is figuring out for themselves.

SPENCER: Yeah, and I guess I would feel more comfortable with a strategy where each of the people in EA community is thinking about, given the intersection between what they think is going to do the most good according to their view on what's the most good, plus things that they actually would enjoy doing, plus things that they actually be good at doing, they're finding strategies to execute. And so then, the allocation ends up emerging in a more bottom up way, if that makes sense. I guess I don't know how to get to top down. I think it's very reasonable for you to have a perspective and you can make an argument like, “Hey, maybe we should allocate 75%.” But then, let's say someone else really smart in the community is like, “Well, I think it should be 25%.” It's like, what do we do? We can debate these things. But I think, because I just view there being so much philosophical uncertainty, leading different conclusions, plus so much empirical uncertainty, that I don't know how to resolve these other than say, “Hey, if you're convinced of this philosophical position, and you think that the way to do more good is like this approach, go for it.”

WILL: Yeah, I'm curious on where you see the kind of big philosophical and empirical uncertainties are coming from because it seems to me the arguments for taking impact over not just the present generation but further into the future — again, not claiming that's the most important thing for humanity as a society — but the argument of at least saying, “Oh, wow, this is a really big deal,” seem very strong to me and very kind of philosophically robust, actually. Sometimes the way these arguments are presented are really bad, where it assumes a certain population ethics. I don't think that needs to be true, or assumes a certain way of handling decision theory, like, “Oh, it's a tiny possibility of an enormous amount of value.” I don't think that's true. So the philosophical back underpinning for at least a weak longtermist position seems really pretty robust. And then on the empirical side, I guess, at least various. But I think there are some things we can do, in particular stuff on protection against future pandemics. So, like early detection, advanced PPE, really rapid vaccine deployment for UVC light that can still sterilize a room. Those also seem just kind of robustly good things to me as well. I don't feel I'm going to be super surprised by them. So I guess I'm kind of curious on what are the kind of philosophical sides of things that you are feeling super unsure about when it comes to kind of EA prioritization that you think makes a really big difference?

SPENCER: Yeah, so maybe it is a good time to go into some of the critiques of longtermism, because I think this is quite relevant to your point. And there's different sorts of critiques, some are more philosophical, and some are more pragmatic. And so, I think they have different flavors to them. One thing that you mentioned is like, there's this idea of, “Oh, are we just getting Pascal mug to where there's some tiny, tiny probability of this becoming a super good outcome. And if we multiply the tiny probability times a really, really good outcome, because the outcome is so good, it leads to good expected value. And so we should just do that.” And people have a lot of discomfort with trusting that kind of intuition. I think, for good reason. There's all kinds of things that seemed like, “Oh, you could argue there's some slight chance of that thing happening. And if the outcome is good enough, that should determine what you're going to do.” It seems like not a very robust strategy. I think your view is that longtermism isn't like that. Like, it's not that the probabilities are so small. So do you want to kind of make that case a little more?

WILL: Yeah, for sure. So that's the key argument. I agree entirely if the argument was like, “Well, maybe it's a one in a billion billion chance that you can reduce existential risk. But the stakes are 10 to the 60 people in the future. And therefore, that's the most important thing.” If that was the argument, I would certainly not be going around promoting these ideas. I think it ends up extraordinarily non-robust, most obviously, to the argument that if you're taking tiny, tiny probabilities of enormous amounts of value seriously, why not take even tinier probabilities of infinite amounts of value seriously. And I don't see very many people going around creating new theological seminaries to try and figure out how we can best produce heaven on earth and so on.

SPENCER: Which religion has a slightly higher probability of being true or something?

WILL: Exactly. I think people should be trying to figure this stuff out. Like GPI has written a number of papers on this problem–paradox of tiny probabilities have enormous amounts of value. And unfortunately, it looks like there's just no answer that doesn't involve very unintuitive implications. And so we're not in a great epistemic state there in terms of what is the correct underlying decision theory when you're confronted with tiny probabilities of huge amounts of value. But that's really not the case for longtermism. Of the things that we're talking about, like World War Three in our lifetime is something I'm really worried about. I think the odds of that happening in my lifetime is like 20%, maybe one in three. Artificial general intelligence that leads to a very rapid rate of technological increase. Again, I'll put like one in three or more that resulting in some sort of existential catastrophe. I put something like 10%, though, I'm actually particularly worried about human misuse as compared to AI takeover. I think that's something we should seriously worry about, too. In the case of pandemics, and worst case, biorisk that could kill us all and end civilization. I put that like 0.5%. All of these very medium-sized probabilities are definitely not like one in a billion, billion. And if I tell you, “Oh, it can get in a plane, and alone, it'll have a one in 1000 chance of killing you.” You're not going to get on, you're not going to take that flight. And we invest enormous amounts on plane safety to reduce risks that are much, much lower than that. So it's actually a real shame that that became a meme, because it makes people think that we're talking about risks that are tiny, but we're actually talking about risks that are really quite significant.

SPENCER: With everything you said, I totally agree. Where I think the Pascal mug thing comes into play more is when you're thinking about the chance that one individual person is going to make that difference. Like, if someone's reasoning about what's the chance that I can stop World War Three, and I'm talking about let's say a random effective altruist. What probably should they put on that, right?

WILL: Yeah. I think in other areas, we actually are willing to act on really quite low probabilities often. So if I drive for a mile, do I wear a seatbelt? And I do. My probability of dying in that time is like one in 300 million. And that just seems totally reasonable. And it seems reasonable for the reasons the expected value theory would say it's reasonable, which is like the cost of wearing a seatbelt is very small, compared to the harm of dying, even when that harm of dying is divided by 300 million, because the probability is so low. And similarly, when it comes to other things that are moral campaigns or like altruistic action. So suppose you join a march, like a protest movement. And you might ask like, well, what's the probability that you, as the extra person in this protest, really make a difference? That's not how people tend to reason. But I think it actually justifies it, because you might be one of tens of thousands of people engaging in this protest. The probability of any one person making the difference for that protest being impactful as a result of you joining is gonna be very low. But the stakes are really big. And so maybe it is a one-in-a-million chance that your protest makes a difference on the UK's climate policy or something. But that would be a small chance of a very large difference. And I think the same would be true in the case of preventing a third world war or reducing some of these other risks, like, do I think that a committed group of people could reduce these risks by 10%? So if the risk of third world war was 20%, we can get it down to 18%. And I'm like, “Yeah, probably we can.” And then the kind of, “Sure, the probability of you making a difference is small, but it's still not like Pascalian small. And is, in fact, like other similar magnitude of loads of other decisions that we make, like joining protests, like wearing a seatbelt.

SPENCER: It's interesting because I don't view wearing a seatbelt or joining a protest as very similar. The reason for seatbelts is because we can do a real calculation on the probability of seatbelts saving your life. So it's real numbers we're putting into our probability estimates. With protests, I don't view it as equivalent because I think most protests, unfortunately, are useless. So I think most of the time without actually making [chuckles]...I mean, if they're doing it for non-consequential reasons, like their goal is not to actually change something in the world. I think most protests are not very effective at achieving their own ends, if their ends are consequentialist.

WILL: That might be true for most protests. There was one study done on Tea Party Movement protests that seemed like they were actually quite effective. I've never done the calculation in terms of how much value from the perspective of a Tea Party, someone who had like fairly Republican, right wing views, what was the value there, but they kind of used weather as...so if the weather was less nice, you got fewer people attending the Tea Party protests.

SPENCER: So it's like a quasi randomization.

WILL: It is a quasi randomization. Yeah, you did get in the areas that had larger Tea Party protests, you seem to get like a rightward shift in the views of subsequently elected politicians. — It's funny, I'm like dredging that study up for many, many years ago. And now as I'm saying it, I'm like, “Oh, yeah, so obviously that's like a study one in three that it replicates or something. — But I at least think it's plausible that a protest movement would have an impact, or just other things you might be doing that would have an impact. Like, you're part of a large organization like Greenpeace that's like engaging in some sort of lobbying. The lobbying doesn't have that larger chance of success. You're only one part of like a thousand people working on this or something. And we know that lobbying can have an impact. Once you start working that all through, probably you've got quite a small chance of having an impact. And it just seems kind of fine. I think the right intuition is when we're thinking about it in terms of large collectives, and we know that large collectives can make meaningful differences. And large collectives are composed of lots of individual people. And so they get like, to a first approximation, an equal share, although really, I'm just concerned about what each of their marginal contribution is.

SPENCER: Where I see this reasoning coming into play is at the individual level. So imagine someone says, “Hey, you know what, I think I have a one in a million chance of stopping World War III.” That one in a million, though, I'm extremely uncertain about because I can't actually calculate that. It's just like a sort of a hunch, and I put some numbers in a spreadsheet. It's kind of like all made of bullshit. I would rather go do this thing where I think I have a one-in-a-hundred chance of having this really big impact, not nearly as big an impact as preventing World War III, but at least the number one-in-a-hundred seems it's not made up. I'd be like, “Yeah, that seems pretty reasonable.” I guess I get uncomfortable with very made up probabilities. Whereas if someone on the other hand said, “Hey, I think I have a one-in-a-million chance of preventing World War III. I did the expected value calculation; it seems good. I'm gonna devote my life to it.” I'd also be like, “Cool, that makes sense.” I think I'm just less confident in what the right thing to do is in that case. I may be more liable to go with someone's preference on those like really difficult philosophical questions.

WILL: This kind of goes back to our earlier conversation where the upshot of longtermism is not that what you should do is just like write down the benefit-cost calculation using whatever numbers seem to you to be correct, and then follow it blindly. However, what is true is it saying the yardstick for comparing two different activities, ultimately, is how that impacts not just the present generation, but the entire course of future civilization. And it might well be that you're like, “Okay, I could do this one thing. It has a one-in-a-million chance of paying off, but I'm just completely making up this one-in-a-million number. Whereas this other thing, it doesn't look as good in the cost-effectiveness calculations. It is still impacting the long term, though. But just with lower expected value, but I'm feeling really robust about it, then I just don't think it's insane at all to go for the latter thing.” And so I see these kind of two different arguments, I think, of two different possible critiques of longtermism. One is just the “Look, this involves the relying on tiny probabilities of enormous amounts of value that seem goofy in other contexts.” And there, I think, the arguments don't work because the probabilities aren't like Pascalian. They're not so tiny, where a one in a million probability of making a really big difference is just like...I don't think that's such a tiny probability. And then there's a second critique, which is like, “Well, the numbers are kind of made up.” And I think that would apply to some areas and not to others, where some things we can do (like better biotech, defensive biotech to protect against the next pandemic) are a lot more robust than some other things (like, I don't know, work in AI governance or something). On the one hand, I really understand the intuition in favor of, “Okay, do the thing where it's more robust, you're more confident.” I do still have some reservations about that, though. Because then it's like, does that mean we just never do the speculative stuff? Like, as in do we never work on things that are harder to measure? Does that mean we just never do policy and politics around early stage technologies? Like that would just seem really bad. So maybe a kind of epistemic states there aren't so different? Because I'm also like, “Yeah, this is really tough.” And it's not obvious, if you're in that situation, which is the better thing to do.

SPENCER: Yeah. And I think you're kind of hitting the nail on the head of where I'm coming from this, which is that there are lots of bad arguments. And I think you and I both agree [that] bad arguments are bad. [laughs] Controversial statement, right? There are lots of people justifying something because it's just their preference and things like that. But I guess where I come from is, I think there's a lot of genuine uncertainties around philosophical questions and empirical questions, and I'm more reluctant to just sort of make my best guess on those and push for my best guess on those. Now, to be fair, you're a philosopher and you've spent a lot more time thinking about this and it's reasonable you have more confidence than I do on some of these questions. But I get pretty uncomfortable when we hit these difficult philosophical questions. I'm like, “Yeah, this is a hard philosophical question.” It may just be too hard for humans. Like, I'd rather have some people go by my advice, some go the other way, and be driven mostly by their own conclusion about that philosophical problem, rather than sort of make a best guess.

WILL: I agree with you that these problems are so hard. We probably are very ignorant in terms of philosophical considerations. But then that moves me in the direction, not so much like, “Oh, people should just do how things seem to them and what they are excited about.” But more like, “Okay, that places higher value on information, so we should do more research. We should build up general purpose resources. We should do things that keep humanity's options open as a whole in the future, such as reducing extinction risk. We should do things that seem very robustly good, and seem like they're probably good, even in worlds where something happens that we just totally didn't predict (where I think reducing the risk of war is one of them).” So maybe that's where the kind of difference is between us.

[promo]

SPENCER: I do just want to emphasize, I think the things that longtermism is talking about are incredibly important. I really take them very seriously. And I don't poo-poo them at all. I think the main difference is just that I'm more hesitant to put all our eggs in one basket. [laughs] And I have a lot more uncertainty. And I feel really good about lots of other strategies people are using that are not longtermist. So yeah. Another objection to longtermism is around the lack of concreteness in what it's doing, and the lack of legible wins. Now, I'm not sure whether you agree with this critique, but I guess the argument would go, “Look, if you're trying to feed children or something like this, you can make sure that they get the food, and you can see that they have the food, and you're like, yes, the children are fed.” It's extremely concrete. You can tell that you won. Take Against Malaria Foundation. It's slightly less concrete than feeding children. But it's still pretty concrete. You could measure how many people have malaria afterwards, right? With a lot of long term stuff, there feels like this lack of concreteness, just because it's like, “Well, how do we influence the far future?” How do we even know if we've done that until the far future comes? Even if we were in the far future, it wouldn't necessarily be that clear if we cause it to be different. But certainly, looking forward, it's not that easy. And a bunch of the things that are most concrete from a long term perspective feel to me just really good from other perspective, too. Like, trying to be prepared for the next pandemic feels like that's just obviously a really good thing, whether you're a longtermist or not. Pandemics are horrible from sort of every point of view. So yeah, we should totally be prepared. But then when you get to the stuff that's sort of uniquely longtermist that's not obviously good from other perspectives, then it just feels like very wishy washy a lot of times, And that doesn't mean we shouldn't do it. But I do think that that's a consideration,

WILL: Great. So the key thing is just, do you accept that most value is in the future? Once you accept that, then these things that are aimed at short-term benefits, actually the argument that ‘they have great feedback loops and so you can figure out if you're doing good and improve on that' doesn't actually work. Because, okay, yes, I have a short feedback loop for the narrow thing that I'm trying to achieve, which is to protect people against malaria, but I don't have feedback loops for what actually matters, ultimately, which is how does doing that impact the long-term future.

SPENCER: Right. If you're convinced that everything's a rounding error relative to the long-term future. Is that what you're saying?

WILL: No, it's not a rounding error by any means. But just that most values in the long term, and most of the impact of what you do in the short term will be felt in the long term. So there are all sorts of knock on effects. You engage in some development intervention, then perhaps you save lives, those lives go and do lots of things, they build infrastructure, they pay taxes, they eat meat, they contribute to climate change, you'll have changed the geopolitical dynamics of the world by slightly increasing the economy of one country over others, perhaps you've substituted against state capacity. And all of those things have knock on effects, too. And the magnitude of those effects, if you were to know them, will be much, much larger, I think, because they continue for all the hundreds of years or thousands of years to come. They would be much, much larger, I think, from the magnitude of the near-term effects. And so that means that in doing your thing, and you're like, “Okay, cool, I've reduced the malaria burden.” And obviously you know that's good in the short term. But, is that good, all things considered from this very long term...once you consider all the value that is still to come? And you're like, “No, I haven't learned about that thing at all. And that's the thing that ultimately matters.” And so similarly, you could look at things that are being done aiming to benefit the long term, and maybe you can make good progress on, for example, does this form of light radiation sterilize a room in order to protect against the next pandemic? And you can really work on that and make it better and you're getting feedback loops at this thing you're designing. But then how does that impact the very long-term future? Okay, you don't know. But then those two things are kind of like on a par as it were. In both cases, we're getting some kind of short-term feedback loops on whether we're achieving the thing we're aiming to achieve. We're not getting feedback loops about how that contributes to long-term value.

SPENCER: I want to separate out two different claims here, because I think they're important to distinguish. One is that most of the value is in the long term future. The second claim is that if you take a short-term action, like trying to help people not get malaria, that most of the effects of that will be in the long-term future. I think that's the second point that you're trying to make. Is that right?

WILL: Yes. I agree, that's a different claim. But I think it's true, at least as long as we're talking about, as a matter of fact, what actually happens, rather than what we would predict to happen given our current epistemic state.

SPENCER: There's also like a chaos theory thing here that I think is important to distinguish, which is like, it may well be — given the way the world works — that if you decide to leave like five minutes later to go to the dentist than you would have, that you will massively impact the whole future of the world, right? Because the cars that you're going to interact with are gonna be slightly different. And so some people are going to arrive at home like five seconds later than they would have. And that means they're going to make phone calls slightly later. And, literally, it's going to ripple out and affect the entire world eventually, right? But, in expectation, there's no effect of that. Even though there might be huge effects, like on average, we have no particular view on whether it's gonna be good or bad, so on average, we just ignore the chaotic effects. But I think you're not just saying there's gonna be chaos theory effects that are completely unpredictable. But you're saying that the impacts of doing something like reducing malaria, we can reason about their second and third and fourth order effects on the long-term future in such a way that we can actually come to believe that those effects are much larger than the immediate effects.

WILL: Yeah. So if we had a much better...suppose we're super genius, super plain people, we probably could have a much better understanding of how actions in the short term impact the very long term. And that would really matter. And I think, yeah, those effects would be much larger than the impacts just in the short term.

SPENCER: I think I agree with that, if we both had a sufficiently clear idea of what the future should be like. And additionally, we were smart enough to figure out how to get there. That might be true. I guess, I'm not even sure that's true. But like, I think it's probably true in that case.

WILL: But then, what I'm just saying is like, we're in the same position with respect to how does what we're aiming to achieve contributes to the good. How much does actually make the world better? If we're doing things that are kind of on longtermist lens versus things that are aiming at short-term benefit, where the argument against the things that aiming at long-term benefit is like, “Well, do you even really know that it's actually making things better because you're never really learning about how the thing you've done has actually made the world better in 1000 years? Who even knows?” But what I'm saying is that as long as you think that most value is still to come, then the same is true for the short-term actions. You're like, “Well, do you even know if there are positive knock on effects? Are the things you're doing positive or negative? You haven't actually learned in the course of doing this action whether the thing you're doing is making the world better, where the world includes the future as well as now. You haven't actually learned whether it makes it better or not.” And so there's an argument that one could make that once you start to include how things impact the future, you kind of end up clueless. But that would impact actions that are trying to benefit the short term as well as actions that are deliberately aimed at benefiting the long term. In fact, I think actions that are going to benefit long term do kind of systematically better in this regard.

SPENCER: I don't think I agree with your main point here. Because if I'm going to take an action — let's just say for the sake of argument — I'm really confident it's gonna benefit the short term. And I have no idea how it's gonna benefit the long term, just like I don't know how going to the dentist is gonna benefit the long term or hurt the long term. It's sort of just chaotic. It might actually impact the longterm a lot. I just don't know which way. On average, don't you just get the short-term benefit, unless you have some reason to think that actions that are beneficial in the short term are more likely than random actions to be harmful in the long term, like when you just calculate the short-term benefit and leave it at that?

WILL: I was responding to a slightly different critique, where your kind of original critique was, “Oh, hey, you're not learning over time, is your thing actually good.” And if you're doing, let's say, the AI safety work or something, the argument for thinking why you're not learning if your thing is actually good, is because you never get to see the kind of end result, how whether it's made the world better or worse in a billion years time. But what I'm saying is that as long as you accept that most value is in the future, the same is true for kind of near term do getting. You never get to find out whether your thing was actually good. Because your thing will impact the longer term too, and not merely in chaotic ways, but in ways that would be predictable, if only we knew more. So, you are also not learning whether your thing is actually contributing to making the world better or not. You're getting this little bit of information, like the benefit you're making in the near term. But that's extremely small, so the argument goes, compared to the overall impact that your action is having. And so it's like a very small amount of information about the impact of your action. And so in particular, you're learning that you can reduce the malaria burden, but you're not learning at all whether that aim is actually making the very, very long term better. And so again, you're not getting feedback loops on the thing that really matters.

SPENCER: Right. And I guess if someone thought that the near-term benefit really mattered, they would have feedback on that. You're just not having feedback on the long-term thing. And if you think that's most of what matters, then that puts you in a similar position to trying to work on long-term stuff.

WILL: Yeah. And if you think it's just the near term benefit that matters, then the critique is no longer about feedback loops. It's instead just, as you're saying, it's only the near-term benefit that matters.

SPENCER: You might think that the long term would matter, just that you can't figure out what the effects are going to be, right? And the short term also matters, right?

WILL: Okay, great. And I think I see that as a different critique. On that, I think that's kind of what the whole book is about, or at least a goof five chapters of What We Owe the Future is about, where can we do things that are predictable over such long time plans, where the very natural response is, “No, that sounds totally crazy.” But I think the arguments are actually quite good that we can. The very clear list is extinction, where if we go extinct (like we're not coming back from that), that is an effect that will last as however long as humanity would have lived otherwise, so is an impact that potentially persists indefinitely. That's not something that kind of washes out over time, or it's not unpredictable. If I prevented some kind of doomsday virus from being used by some bioterrorist, let's say, have I increased the probability that the human race is around in a billion years time? It's, “Yes, I've definitely increased that probability, in expectation, at least.” And then, I think the argument is a little bit harder for value lock-in. That requires more steps. And I discussed it in more length in the book. But again, I think the argument ultimately does check out. Like, if you had advanced artificially intelligent systems, you could create a regime such that the beings in power could just ensure that they persisted indefinitely, or the values they wanted to promote, or the ideology, persisted indefinitely too.

SPENCER: Yeah. I think it's really great people are working on existential risk. And one reason is long-term arguments for it. But I also think there are really, really good other arguments for trying to not let humanity go extinct and not letting one powerful group or one powerful AI ruled the whole world. So, I think existential risks should be both a longtermism cause and a short-termism cause, if that makes sense.

WILL: Yeah, and I completely agree. These are the risks we face that could impact the long term, and are just undesirable risks. And I think the world, even if we didn't care about our children or our grandchildren or their grandchildren at all, should still be taking way, way more effort into trying to reduce these risks down to 0.

SPENCER: Totally agree. Will, I could easily talk to you about this for two more hours, but I know you have to go. So, I'll end it there, but thank you so much. It was a really fascinating discussion. It's always really fun to chat.

WILL: Thanks so much, Spencer.

[outro]