SPENCER: Josh, thanks for joining me on this special episode.

JOSH: Yeah, thanks for having me.

SPENCER: So today we're going to do something very unusual. We've gathered instances where people have said that I'm wrong, or have disagreed with me, or made critiques of things I've said, and put them in a big spreadsheet. You've reviewed them, and today we're going to go through them. For each, obviously we're not going to be able to cover every critique that people have given me, but you got to pick the ones that you thought were most interesting, and we'll go through them. For each, I'll attempt to steelman the case for the critique. In other words, I'll attempt to make the strongest argument that I can for the argument against my position. Once I've done that, you can feel free to comment or jump in if you have anything you want to add or say. Then I will say what I actually think now, having reflected on the steelman of their position as well as their original critique, I'll give my point of view.

JOSH: Sounds good.

SPENCER: So what's the first critique of things I've said that we're going to do?

JOSH: Some people have critiqued your views about cults. First of all, I guess I should ask, are you anti-cult?

SPENCER: I am anti-cult, but I will clarify that not everyone uses the term cult the same way. I may not be anti-cult according to someone else's definition of cult.

JOSH: How would you define cult?

SPENCER: First of all, I think of cults as being on a spectrum, and I do not think you can apply binary to cults. Yes, there are some things that are so far on the spectrum that any reasonable definition would call a cult, but I think we have to accept that what we're talking about is a number of related, kind of correlated properties that a group can have, and you can have more of them or less of them. We can talk about what those properties are. I would say that the most key characteristic properties have to do with a group that has one or sometimes two or a small group of leaders. Sometimes the cult will continue existing after those leaders are dead, but at least there have to be initial leaders that are thought of as having special information or special knowledge or special abilities. The group separates itself from society through its ideas and sometimes physically. The group typically discourages outside information seeking and becomes its own kind of information ecosystem. The group requires self-sacrifice of its members, and the group members sacrifice for the benefit of the group, but much of this benefit often accrues not to the group itself, but to the leader, although some of the benefit might also go to the group itself. The group also typically has high levels of conformity that are demanded, whether it's conformity of ideas or conformity of dress or conformity of behavior. Finally, typically, group members will be asked to behave in ways that are generally considered unethical, in some manner, or group leaders will behave in a way that's generally considered unethical. That's kind of how I think about it. Again, it's a spectrum concept. If you take a group like the People's Temple that was run by Jim Jones, that was very clearly a cult; it checks every single box. Then you have things that are much less clear, like a multi-level marketing group, and maybe that might be 60% of the way to a cult, but not actually fully 100% a cult.

JOSH: One of the critiques, or disagreements from listeners, is the idea that there is such a thing as a good cult. From the way you're describing it, it sounds like maybe you generally think that most cults, or maybe all cults, are bad, and the disagreement is the belief that some cults are good. One listener said cults are an essential part of the genius of America. Cults have tremendous potential to address the deep problems of modernity, like obesity, addiction, loneliness, fertility collapse, etc. What do you think of that critique? Is there such a thing as a good cult?

SPENCER: Yeah, to steelman it, I could see a few different reasons you might think that cults could be good. One thing is, you might think they can create a really strong group identity, which, while it could have some harms, you could say, maybe that's really nice to be part of a tight-knit community of people that think similarly to you. Cults give people a mission, a lot of times, which could be nice. If that mission is a good mission, it could be beneficial to the world, potentially. Some cultiness might get people to work together really well, take on ambitious goals, and maybe create things together. Also, people may just enjoy being part of cults; they may find that it's an exciting thing to be a part of. It gives them companionship, meaning in their life, and so on. I could see those cases made for some of those things. But this part might partly be definitional. I don't know exactly what definition of cult this person wants to use, but what I would say is a lot of those benefits are really about a tight-knit community. I would like to separate cults from tight-knit communities. Yes, cults are tight-knit communities, but most of the good things I would put in tight-knit communities, especially ones that have a shared sense of meaning and values, and separate that from cults, which is a very specific type of tight-knit community that has things like separation from the outside world, conformity, and self-sacrifice that ends up accruing to one or a small number of leaders who benefit a great deal from the self-sacrifice, while the members in the group pay a price for it, and where unethical behavior is typically inculcated either among the members or among the leaders.

JOSH: As you were reading your first definition of cult and the properties that many cults have, I was thinking of governance in general, and there seem to be some similarities between cults and governments. It makes me wonder if that's true. If cults are a kind of governance mechanism, there could be potentially good cases. For example, often businesses, unicorn startups, are referred to as cults if they have a very charismatic leader and a strong cultural identity within the company and have to adhere to very strict guidelines. A lot of people will praise that if they are producing a good product or have a good business model. If that's true, if cults are a kind of governance structure, does that make them morally neutral, or do they inherently have something corrupt about them?

SPENCER: When I think of the word cult, I think of it picking out a pattern of the way things are structured. If we look across history, we see this pattern arise again and again. If you think about companies, most companies are not very cult-like. The behaviors in the group are not that unusual. Okay, maybe people wear business casual, but so do thousands of other companies. They don't behave that unusually from other companies. There may be some conformity among people at the company, but it usually is not such extreme conformity. They might conform around agreeing to proposals, but they're not usually conforming around every element of their life. Usually, they don't have significant levels of isolation; the company is not telling them to isolate from friends and family or to not seek information sources outside the company itself. Most companies don't ask people specifically to engage in unethical behavior, although obviously some companies do, and that's a problem itself. You might say sacrifice is maybe the more common element that companies might share with cults. Some companies do ask for significant sacrifices in time, and some companies ask you to work really hard, but they also pay you for it, and you could quit and get another job elsewhere. That is quite different still, where very often cult members are asked to make tremendous sacrifices and don't even get paid for it, or get paid the absolute minimum, like just food and a tiny bit of spending money. I'm not saying there aren't any companies that are cult-like; there are, but most companies are not very cult-like. Let's talk about more cult-like companies. Often people would say Apple is a cult, Apple Computer. If you go through these things, you actually see it's only a bit cult-like. What makes it cult-like is people's dedication to it. People get really excited about it. They want to be part of it. They attribute all these great properties to its leader. Steve Jobs was a charismatic leader. Still, I think most of these traits don't apply even to something like Apple, which is one of the more cult-like startups. On the question of countries, that's an interesting one because I think there truly are cult-like countries. I'm sure cult-like startups exist too; I just think they're really rare. If you think about countries, North Korea is incredibly cult-like. I would argue that North Korea, in fact, is a cult. It's one of the largest cults in the world. If you look at the indoctrination of the people and what they teach the people about the leader, the extreme efforts they go to prevent people from having external information, the way they commit extremely unethical acts against their own people, like locking people up for really minor things and making people inform on each other, and getting people to engage in extreme conformity as well. I think North Korea is genuinely a cult. It's not a metaphor.

JOSH: One group of critiques that was really interesting to me, and I thought would be interesting to discuss on the show, especially related to Clearer Thinking's mission and tools, is that Clearer Thinking makes a lot of rationality and self-improvement tools. One critique from a listener was that they would probably focus on other kinds of things in somebody's life, like their physiological needs, eating the right foods, getting the right amount of sleep and exercise, before they would focus on their personality or psychology. Other critiques are that these tools leave out some of the things that give people real meaning, like religion and spirituality. So what's the steelman of those critiques?

SPENCER: Yeah. So I take it to be saying, "Look, do your tools really help people? You're focused on these really abstract things like how to think more clearly, how to make better decisions, how to have nuanced thinking. What really matters to people are not these kinds of abstractions. What really matters are things like healthy eating or spirituality, and these are the things that actually change people's lives for real." And I certainly agree that those things are really important. Eating healthily really matters, and people's religion or spirituality is some of the most important things to them. People who resonate with that. I think there are lots of useful things to teach that we don't teach. I think that's absolutely true. There's tons of useful things you could teach, and maybe to steelman further, there is a lot of abstract material that probably doesn't change people's lives. You learn about a concept and you never use it. I think that's a very standard thing that happens. Or you just have trouble connecting that abstract concept to your daily life, so you don't even know how to apply it. Or maybe the information is just not that useful. We have all kinds of tools on our website. I don't think every one of them is that useful. Obviously, we try to make stuff that's useful, but I don't think we always succeed. It's okay. So what do I think about this? I do think it's really hard to change people's lives. I think it's a big challenge. I completely agree that some of the things that the people who gave these critiques mentioned are important things. Why do I think what we teach is important? Well, let's go through a few of the things we teach. One of them is critical thinking, including things like nuanced thinking, seeing the world probabilistically rather than true-false binaries, seeing things as having both good traits and bad traits, rather than seeing it as all good or all bad. Nuclear power is good or nuclear power is bad; thinking about what are the benefits and weaknesses of nuclear power, and comparing that to the benefits and weaknesses of other things. So that kind of material we teach. The reason I think it's important on a personal level is that the better you understand the way the world works, the better you can make decisions about the world. If you have false beliefs about the world, that actually does impair your decision making, because when you're trying to navigate the world, you actually just believe false things. Really clear-cut cases of this are things like around health, for example, which one of the critiquers brought up. But take health, how do you know what to believe about health? You've got all kinds of podcasts saying all kinds of things that are completely contradictory. One popular health guru says you've got to just eat meat, and another says, "No, you've got to go vegan." You have a third that says, "Neither of those things matter; actually, you just need to take these supplements, and that's the key to longevity, etc." So I think critical thinking is essential for making these actually important life choices. It also affects society, because we make decisions on behalf of society by voting. If we run an organization, we might be influencing society, or just even the way we impact other people in our lives. I think critical thinking actually helps with that too. It helps leaders make better decisions. It helps people vote better, so I think it has societal implications as well. I also think sometimes falling into false beliefs can be just incredibly detrimental, whether it's certain conspiracy theories or false beliefs about the way the world works, where we think, for example, that one ideology has the answer to everything. When we start to think that, we start to rely on that ideology to answer all the questions in our life, and that often goes really horribly, because, in fact, no simple ideology has all the answers. So I think there's a lot of benefit at the individual level for deeply understanding critical thinking. Now, I think it's a hard topic to teach; we try to teach it really well. But of course, it's an ongoing thing. We're learning to do it better and better over the years. I think our modules today are better than the ones five years ago, and hopefully the ones five years from now will be much better still. It's still ongoing learning about how to best teach these things, how to make it relevant to life, how to help people apply it in everyday situations, and teach it in a way that sticks. So that's some of the kind of material we teach. Other material we teach, I think, is actually almost easier to make the case for it being beneficial. We have a module on how to form healthy habits, and we actually ran a randomized control trial testing the exact interventions we teach in that module, showing that they actually do indeed work compared to the control group. The people that used our module had stickier habits over the coming weeks. I think that's just a direct benefit because I think healthy habits can make your life better, so that one's a relatively easier one. Or, material on goal setting. If you think about some of our content, it's trying to teach you to understand your values better so you can set goals better, or it's trying to teach you to think about how to set goals more effectively, so you can achieve the things you want. That has a really direct benefit. I think it's a reasonable critique, and I would definitely not say we're doing perfectly at it, but I do think that the kind of stuff we teach has a lot of value, and I also think that I expect we'll continue getting better at it.

JOSH: So let's assume for the moment that all the tools we have are really effective and useful and interesting. Why is the strategy of putting them up for free on our website better or more effective than, for example, trying to take them directly to, let's say, politicians or school boards or things like that, to try to work them into educational decision making, political decision making, things like that, more directly?

SPENCER: If you're thinking about making educational content to try to improve the world, I would say there are three broad strategies that folks take. One is you try to go mass market, usually focusing on adults, trying to reach as many people as you can. That's really what we're trying to do. We're trying to reach, I mean, we're not trying to reach literally everyone in the world, but we're trying to reach people who are in the top percent, let's say 25% of reflectivity and interest in understanding themselves, their own mind, and making better decisions. We're trying to reach them with our content. The second strategy is you could go to kids and children, try to educate them, get them early, and hopefully change the trajectory of their lives. The third category is you could focus on people who have higher levels of influence, whether it's focusing on politicians or focusing on business leaders, with the idea that even if you reach many fewer of them, maybe you can have an outsized impact because each of them affects a lot of other people. I think those are all completely viable strategies that some people can do, and each of them could be great. I think we are best suited to do the broad-based adult strategy. I think it's the best fit for our skill set and our team. I would not at all knock the other strategies; they could be great in other contexts. I just think we're not as well suited based on our own expertise and skill sets to address those other markets. For example, we don't have expertise in pedagogy for children, which I think is not the same thing as pedagogy for adults. We also don't have deep ties to politicians. If you want to do a strategy where you're trying to influence politicians, you would want that ideally to be something you have a background in or have deep connections in, etc.

JOSH: Another kind of critique made by a listener is that the Clearer Thinking tools and the studies behind them don't go through any kind of academic peer review process. We may do our own "studies," but how legitimate are they really? How can people using the tools know how legitimate they are if we didn't have them published in a major journal or have them reviewed by well-known academics, etc.?

SPENCER: To steelman this critique, I do think that peer review catches some forms of bad work and prevents it from getting out. Of course, someone could always submit to other journals and keep going for journals if someone publishes, but at least if you're focused on higher-tier journals, peer review catches some serious issues. It blocks some really low-quality papers from getting into the literature, which I think is a good thing. Also, I'll add that sometimes peer review just makes papers better. Even if it doesn't block a bad paper, it could make a mediocre paper better because the reviewers have really great comments and suggestions. They ask for more rigorous work, and so on. I think that, to me, that's the kind of steelman of the case, and by not using peer review, we're missing out on those benefits; we might be producing really poor work. If we went through peer review, they would catch those issues or suggest ways to make it better. Anything you'd add to that?

JOSH: No, I think that summarizes it.

SPENCER: What I think about this is actually reflected in an article we put out about the strengths and weaknesses of peer review. We talked about this a bit, but I'll go over my thoughts now for those who didn't read that article. First, I just want to say we do sometimes publish in peer-reviewed journals. In fact, just recently, we've published two papers in peer-reviewed journals. This is something we do sometimes. We don't do it with most of our work, but we do it with some of our work. Why is that? Why don't we just try to put everything we do through peer review? First, some of our work just isn't relevant for peer review. If we're making an interactive module trying to teach a concept, it's just not the sort of thing peer review exists for. That doesn't really apply, but when we conduct studies, many of them would be appropriate for peer-reviewed journals in the sense that there are peer-reviewed journals we could submit to. My experience having published a bunch of papers is that I very rarely feel that my papers are better off having gone through peer review. Oftentimes, reviewers will have some minor helpful comments, like, "Oh yeah, that would be a little bit better. I'll tweak this, that, tweak that." I've not had an experience where I felt like they found some devastating problem or made a huge difference in the quality of our work, so I found it kind of marginally useful. That's not to say it never happens. I think it certainly does sometimes happen. I do think peer review blocks some really low-quality work. I do think occasionally peer reviewers find some devastating critique or make something much better with their suggestions. I just haven't had that experience in my life. Additionally, I will say that I think peer review just doesn't work nearly as well as a lot of people think. I think it lets in a tremendous amount of bad work. In fact, with our project Transparent Replications, we replicate random papers from top journals in psychology, and we only do the very top journals, and we find lots of very serious problems in those papers that clearly peer review is not catching. I think people who believe in peer review, I'm not saying they're wrong to think it's a good idea. I think it has a lot of benefits, but I think they're really wrong if they think it is a very effective method of weeding out bad work. I also find that we get lots of feedback on our work. When we put our work out in the world, we generally get a lot of comments, suggestions, and critiques, some of the things we're addressing in this episode, and I find that incredibly valuable. A lot of times I don't agree with those critiques, but I try to take them seriously, and we definitely learn from them. There have been significant critiques people have given us that have been really helpful and helped us make our work better. We actually get critiques a lot. It's not by three academics in the field, as would happen with peer review, but it is usually by a lot more than three people, depending on the case, with a wide variety of different backgrounds, some of whom probably are experts in the topic, although many of whom are not. I find feedback incredibly valuable, and I think our work would be much worse if we didn't get regular feedback and critiques. I think critiques are really important. I just don't think they have to come from peer review. Now, why does peer review miss a lot of bad work? It's an interesting question. You may not agree with me; maybe you think it does a really good job. I would point you to the replication crisis in psychology, where I think 15 years ago, about 40% of papers in top journals wouldn't even replicate. If you tried to redo the experiment from scratch, you wouldn't get the same answer. I think that right there shows that peer review is missing a lot, but I think it's more than that. Peer reviewers generally are not paid, so they're volunteers. They're generally anonymous, which creates a very weird incentive structure where they don't have that much time to work on the peer review. They don't really have that much benefit from doing a really great job, although some of them do a really great job. It's a pretty thankless task, and they generally don't reanalyze the data. They certainly don't redo the study; that would just be way too much work. I think it misses a ton of really important problems. We try to create feedback loops. For example, when we put out new tools, we generally run a study on 40 random Americans and have them try the tool, give their critiques, and tell us what they liked and disliked, and we'll use that to improve it. Then we'll generally run a second qualitative study where we'll have a bunch of our beta testers, who are real users, try the tool and critique it. We get tons of critiques that way that really help make our work better. When we put things out in the world, we get additional critiques. Critiques are awesome; peer review can also be good, but I don't think it's essential. You might ask, "Why not just put it through peer review? Why not just do it?" We would if it were a really simple and easy process. The reality is, it takes so much more time; an unbelievably large amount of extra time to go through the process, and so on a cost-benefit basis, it just doesn't make sense to us. Our general policy right now is we sometimes do publish. As I mentioned, we've recently published two papers in peer-reviewed journals, but we generally do it when we've found a good academic collaborator. If we're working on a research study and we know someone who might want to collaborate with us, we'll pull them into it. It'll be their job to write up the paper and submit it for peer review. Usually, we'll do the bulk of the research, although with their input and ideas, which can be very helpful, but usually we'll do most of the actual execution. It's a great collaboration. That's something we've been moving towards to get the best of both worlds, but still, with most of our work, we don't do that.

[promo]

JOSH: On a different topic, one listener critiqued your view that AI companies should roll out AI text watermarking. Can you tell us a bit about what that watermarking is and why you think these companies should roll it out?

SPENCER: Text watermarking refers to techniques used when a large language model (LLM) generates text. You might want to know, was it generated by an AI or not? And that's what the text watermarking does. Essentially, during the generation process, it hides information in the text that makes it much more reliable to tell whether it's generated by an AI.

JOSH: So one of the critiques around this, of your views, is that, "Maybe companies didn't roll this out because it's just not technically feasible, not that they're being negligent or focusing on other things, but just that maybe there's not a way to do this reliably." Is that the case?

SPENCER: Yeah. And so to steelman their view for a moment, I will say that with any kind of watermarking technology, there typically will be counter technology that helps you remove the watermark. I wouldn't expect this to be any different. If someone is really intent on removing a watermark, they could probably do it. Eventually, some software will spring up that helps you do it more automatically if there's a demand to remove it. Additionally, there may be ways to get around it, like maybe you can have another AI rewrite it in slightly different words, and maybe that removes the watermark. Or maybe, if some of the companies deploy the watermarking technology but not others, then people will just move to the ones that don't watermark. So I think that, to me, that's kind of the steelman perspective of what this person is saying. What I will say is that I was told by someone who was involved that at least one of these companies actually has a technology that really works, and the approach is actually quite fascinating. You might think, "How on earth can you generate just regular text in such a way that you can tell whether that particular AI generated it or not? It's just text. What could you possibly look at?" With an image, at least you could say, maybe you could hide some details in the image, like some slight pixelation that the human eye can't detect that indicates whether the image came from an AI or something. Indeed, you can do that. But you might think, "Well, text, how on earth can you do that? All you have to work with is letters and words, and we're going to see all the words. There's nothing you can't put a hidden word in there." So the basic approach, at least for this technology, is that whenever it's generating the next token, or the next word, it will slightly raise or lower the probability of some words being generated. Just to keep it simple, imagine half of all the words in English are going to get a lower probability, and half of the words in English are going to get a higher probability for that next token. Then for the token after that, again, half the words in English will have a raised probability, half will have a lower probability, but it's a different set of words every time. The set of words for each token is chosen in a really clever way so that you can then reverse engineer it and say, "Okay, was this set of words generated by this AI?" We can go back for each word and say, for token three, we know what set of words it was that was supposed to have a lower probability. Did it generate that word or not? In fact, this seems to be quite effective. You can go back, and by comparing which words or tokens appeared at which points and comparing it to the word lists of which words were lowered or increased in probability, you can actually get a very reliable signal without that much text of whether it came from that AI. Now, I think that AI companies should roll this out. I think it's bad when we can't tell what's generated by AI. For example, a recent thing came out where it turned out people were using AI to generate lots of responses on Reddit. I think this is a bad thing when AI can start appearing everywhere without us realizing it. Now, those steelman views I mentioned, I do agree with those that yes, someone can get around the watermark if they really wanted to. Yes, someone could switch to another AI. But I think really, the way to handle this is that each AI company, every large AI producer, should roll this out. There aren't really that many of them. There's Google, there's OpenAI, there's Anthropic, there's Meta. If they all just collaborated, they could roll out this technology, or they could each roll out their own technology, and it could just make it much more reliable. One of the reasons to do this is not just for the sanity of people knowing what's AI and what's not and potential abuses of AI, but also because massive cheating is now happening where lots of people are using AI for things and pretending it's their own work. You're seeing an enormous amount of this in schools, and it would be really nice if teachers had some way to reliably tell what's generated by AI because right now they don't have very reliable ways, or things like job applications that are entirely written by AI. I'm not talking about people using these as aids, where it's helping them brainstorm and things like that. I mean when someone puts in a prompt and then the entire text is AI, and then they just spit it out as a result. I think also the amount of friction matters. If all the major AI companies implement this kind of thing, it just becomes much more difficult. Even if it's not that difficult, you at least have to think about it. You're like, "Okay, I have to try to remove the watermark." If you really want to cheat the system, you have to actually try to figure out a way around it. Then how do you know you've successfully figured out a way around it? It's not going to still get detected. It just creates a barrier to using this stuff in an abusive way, which I think is important because it's not like everyone has infinite intent to do bad things. Many people will be deterred by small chances of being caught. But if there really is no chance of getting caught, then there's very little deterrence.

JOSH: Let's take the narrow domain of just education. Do we have any metrics on how prevalent this is, how frequently students are using AI to just sort of copy and paste answers?

SPENCER: Yeah, I'm not aware of any good studies in this or anything. But part of the problem is, how do you even know? Even if you had all the essays, we don't have a good way to tell if they're AI. There are tools that claim they can do it, but what I've heard is that they're just not accurate. That's my understanding, and I think a big part of that is because AI companies don't provide really good tech to help you. They could, but they don't, and so companies try to make their own systems to detect, and I just don't think they're that reliable. Anecdotally, I've heard of teachers being absolutely inundated with AI-written essays, constant AI cheating, and a lot of sense that they don't know what to do. "Okay, maybe we just need to have students write essays in class." But how do you do that for your regular homework assignments? Other people are saying, "Well, screw it. Maybe we should just accept that they're going to use AI and just assume everything's AI." But then, "Okay, how do you teach critical thinking if someone could just outsource their thinking to the AI?" So I think it's a really big problem, and I don't think there are good solutions right now.

JOSH: Most of the technical bits of this are going to be way over my head, but one limitation that does occur to me is that the length of the text matters a lot when you're trying to determine whether or not something was written by AI. For example, if a student gives a three-word answer to a question, it's going to be very hard to tell whether that was written by an AI or not, whereas if they give a three-paragraph response, then it's much easier to, assuming we had these detection tools in place, detect a watermark on three paragraphs than on three words. Is that a potential limitation of this technology?

SPENCER: Absolutely. And that's definitely true. The shorter the text, the harder it is to detect. My understanding is that the best versions of this technology get exponentially more confident as the number of words increases. I don't know what that exponent is; it might depend on the technology. But suppose it gets 5% more confident with every word, or whatever it is, because it's exponential, it goes pretty fast. As you get to longer and longer blocks, it can achieve a very high degree of certainty.

JOSH: You write a lot about critiquing yourself, taking real stock of how you think about things and how you can think better about things. One person wrote that they're concerned that the smartest people already tend to doubt themselves the most. Wouldn't we do better to err on the side of encouraging scientists, rationalists, and really smart people to speak up rather than contributing to a culture of excessive hesitation? Because it does seem like the loudest people in the world right now are also the ones who are most confident about things that they are probably clearly wrong about.

SPENCER: Yeah, it reminds me of a Yeats quote, which is, "The best lack all conviction, while the worst are full of passionate intensity." I think there's definitely something to this. The people that know the most, you want them speaking out. You don't want them doubting themselves too much. Excessive questioning is not necessarily healthy, and it may lead people not to take action in the world or not to express their viewpoints. So I think that's a fair point. However, what I will say is I think there's a bit of an assumption baked in there, which is that looking to improve, questioning yourself, becoming aware of your mistakes, and trying to work on them means that you shouldn't be contributing to the world. You shouldn't be speaking out; you shouldn't be offering what you know. I think that's not true. I don't think those things are incompatible at all. If you take the view that to make any mistake is a huge problem, or to have a chance that you're wrong is a huge problem, then, yeah, the implication would be, don't speak out, don't contribute. But I don't think that's the case. Making mistakes is just a human thing. It's something we all do. Being wrong sometimes is a human thing. We all are wrong sometimes. Those are not good reasons not to try to do things, not to contribute. What I would say is we want both. We want self-questioning. We want people to acknowledge their mistakes, try to understand mistakes they've made, try to improve based on them, but also not let them hold them back from doing things in the world. If you wait until you're perfect to do things, then you'll be waiting forever. If you put a lot of ideas out in the world, you'll be wrong sometimes, and that's just something you have to accept, and that's okay.

JOSH: How do you think about decision-making when it comes to the probability that you're right about something? For example, let's say I was a policymaker. When I'm young and naive, I believe 100% that some economic theory is right, and so I base all my policies around this. But as I grow and learn and read more and understand the world better, I come to conclude that, well, I adjust my probabilities to say, "Maybe I'm 75% sure that this theory is right, but now I have some doubts or concerns. It's not as though I can write policies such that 75% of the economy is this one economic theory, and the other 25% is something else." These decisions collapse into binaries; either I implement this, or I don't, or I implement something else, or I don't. How do you think about converting those uncertainties into decisions?

SPENCER: Well, I think for most decisions when there's not a lot at stake, like not a super high risk, or a ton of stuff is being put on the line, you should just try to maximize expected value, given the probabilities. You can prove mathematically that if you have a fixed good that you're optimizing for, like, let's say in this situation, you're just interested in money because it's a money decision, or something like that, and that's the only thing on the line. If the risk is not that big, you're not risking most of your savings; you're just risking a small amount of your money. Then you can show that, in fact, the expected value is the optimal strategy. It will dominate all other strategies in the long term. On average, it doesn't mean it does every single time, but on average, it will dominate all other strategies in the long term. So I think we have a nice, clear answer to that. Now, if you're making big risks, or there's a lot at stake, you have to also take into account risks such as the risk of ruin. You have to think about that separately as well. But I will also say that I think a lot of things that seem binary are not necessarily binary. Yes, if you have to vote and there are two options in front of you, you just vote. You're just going to vote for the one that you think is better. That's not incompatible with being uncertain because you could just say, given an expected value sense, "I think this option is better, so you're going to vote for it." But many things that seem binary are not. Let's say you're a politician and there's a policy; maybe you can influence the implementation of the policy. If you're only 75% sure it's going to work, you could build some things into the policy that help in the case that it's wrong, or you could build in some monitoring to tell if it's not working, etc. I think there are often other options.

JOSH: So on Facebook, you made some claims about how people might interpret the hand gesture that Elon Musk made on stage at the inauguration. Some people had feedback and critiques in that thread. Can you summarize for us what the thread was about?

SPENCER: Okay, so I said in the comment that I thought that few people, if they independently came across the video of Musk, having never heard anything about it, would independently think on their own that it was a Nazi salute. So that was my claim. I wasn't talking about whether I thought it was a Nazi salute. I was simply stating my perception of what other people would think if they watched it with no prior priming. Someone very strongly disagreed with me to the point where they made a bet with me about it. They said, "Hey, I'll bet you you're wrong." Not only did they bet me I was wrong, but they offered me 10 to one odds. In other words, I would have to pay $10 if I lost, against their $100 they'd have paid me if I won. So I was like, "Sure, I'll take the bet." The way we decided to operationalize the bet was that I would run a little study where I would recruit people who had never heard about this video before. They would watch it, and then they would be asked about what they thought, what they saw happen in the video, and if they independently mentioned it was a Nazi salute. We would count how often that happened, and if at least eight out of those 25 mentioned that it was a Nazi salute, they would win the $10 of mine, and otherwise I would win the $100 of theirs. So I won the bet. It turned out that only somewhere between four and seven of the respondents said that it was a Nazi salute. The number, whether it's four or seven, depends on how you count what they wrote — are you more lenient or more conservative in counting it? But anyway, that was what happened. Some people, I guess, strongly disagreed with me and critiqued me in different ways.

JOSH: So one critique was that there's only 25 participants. That's hardly enough to run a robust study. What's your thought about that?

SPENCER: I agree with that. I think with only 25 participants, it's hard to generalize to the whole world. But basically, it was the terms of a bet. It wasn't designed to be some really rigorous study. It was more like, "Hey, I'll bet you, you're wrong, and I'm really confident." I was like, "Okay, and let's say that's the term we agreed on." It was actually such a big news story that it was pretty hard to even recruit people who hadn't heard of it. That was also part of why we didn't have that many people we tested. But if you find that it's four to seven out of 25, we don't know what the number would be if we tested 1,000 people, but it would be pretty surprising if 70% of people responded this way if only between four and seven out of our 25 did.

JOSH: And how exactly did you filter out people who had already seen or heard of the video?

SPENCER: So we screen people, and we only included them if they said they had not heard any news about Elon Musk in the past two weeks. If they hadn't heard Elon Musk news in the past two weeks, we knew that they hadn't heard about the video because we didn't want to ask them about the video because we were worried that would prime them as well.

JOSH: So another critique is that the gesture for many people is clearly a Nazi salute, and the question of whether naive viewers label it as such is sort of beside the point. It's like asking whether a five-year-old child labels it as a Nazi salute. It doesn't really matter; they don't know what a Nazi salute is anyway. So what does their opinion matter?

SPENCER: Yeah. So if I try to steelman this, it's basically saying, what do we really learn from this? Because maybe a lot of people don't even know what a Nazi salute looks like. It could just be that if people don't know what a Nazi looks like, then them thinking it's not a Nazi salute may mean nothing. I agree that some people don't know what one looks like, and certainly if they don't know what one looks like, then they wouldn't identify it as one. I think it's a reasonable point, but I don't think it really has much to do with my post because my post is not about whether it's a Nazi salute. It's funny; it's hard to talk about a thing when you're actually talking about just one aspect of it. People want to drag in other aspects. The post is not about whether it's a Nazi salute. It's about whether random people think it's a Nazi salute who hadn't heard about it. Yet people want to make it about whether it's a Nazi salute. I understand that impulse, but that's not really what it was about.

JOSH: Do you think, though, there's a possibility that you could have over-screened by asking people whether they'd heard any Musk news in the past two weeks when Musk is in the news every single day, every hour of every day? It's like selecting the most media illiterate people, to put it frankly.

SPENCER: Yeah, I think that's a fair point. Maybe the people who hadn't heard about Musk's news in the last two weeks are less likely to recognize the Nazi salute. Maybe they're less likely to know about history, et cetera. I was really surprised how hard it was to find people that said they had not heard any Musk news in the last two weeks. We had to screen tons of people to find 25, so I guess Musk was so in the news that it was hard to even not hear about him in some way or another. That being said, I don't know that that's true. I'm not confident at all that that's true. A lot of people know what a Nazi salute is. It's in tons of movies. It's one of the most important historical events in our grandparents' lifetimes. I think most people have probably seen it, at least in some form of media at some point. Probably most people have seen it quite a number of times in the media at some point. But yeah, it could be the case that people who don't read the news much or don't follow the news much are less likely to have seen it.

JOSH: And one final critique about this was just the overall framing. Treating this as an interesting experiment sort of detracts from the moral and historical gravity of the whole problem of Nazism and the symbolism used. What's your thought about that framing device?

SPENCER: On the object level question of whether it was really a Nazi salute, I would say that the evidence I brought from my study was only very slight evidence against it being a Nazi salute, because whether you know what percentage of people interpret it that way on their own, I don't think is a super strong indicator of whether he intended to make a Nazi salute or not. I just don't think that little study I ran has much bearing on that question of whether he intended to do a Nazi salute. Although I could see why people wanted to make it about that.

JOSH: Feel free to not answer this question if you don't want to. But are you willing to share with us your own views about whether or not it was a Nazi salute?

SPENCER: I feel pretty uncertain whether he intended it. I think the question of whether it was, there's different ways to answer that question. One way to answer that question is to say, was it the same physical motion as a Nazi salute? There you can compare it to Nazi salutes and say, "There are some Nazi salutes that are very similar to the physical motion he did." So in that sense, yes, it is technically a version of a Nazi salute. The other question you might ask, though, is did he intend to do a Nazi salute, or was it just an accidental hand gesture that happened to resemble a Nazi salute? That's a question we can't really know without getting inside his mind; you can't know for sure. However, you can look at evidence, and there are different pieces of evidence. One piece of evidence is that he supported a right-wing party in Germany, so that's some evidence. Another thing is that he's known to do things to troll other people, so that's another piece of evidence. He's even tweeted about how he likes to troll people. You can imagine doing a Nazi salute because he believes in Nazism. You could also imagine doing a Nazi salute just to troll people even though he doesn't believe in Nazism. Either of those could be true. There's also evidence against it being a Nazi salute. One piece of evidence is that very few people actually believe in Nazism; even most far-right people don't believe in it. They think it's a stupid, hateful, or horrible ideology. So it's actually an extremely unpopular ideology in the world. That does lower the probability that he believes in it. So, yeah, I think there's evidence on both sides. I think we have to be pretty uncertain, at least. I don't have enough evidence to be confident one way or the other.

JOSH: So another criticism of your work, and I think this probably applies to some of the things you've said on the podcast and some of the things that you've created through Spark Wave, is that you often coin or popularize terms for concepts that already exist. For example, is "importance hacking" not just another word for hype? Isn't that already a known thing? Why come up with a new word or phrase for this? Maybe the claim is that you're sort of reinventing wheels that don't need to be reinvented or just repackaging things that are already known. We also looked at AI in some of your blog posts and writings to see how it could critique you, and one of the critiques it made is that you often coin new terms, like importance hacking or "light gassing," that are maybe just repackagings of known concepts. What's your take on that critique?

SPENCER: To steelman the AI for a second, I do sometimes try to coin terms, and sometimes those terms I coin have related meanings. You might argue I should just stick with a common phrase or whatever the closest phrase in English for what I mean and not come up with a new word. Just use a phrase to describe it. Or pick the closest word in English, even if there's no word that's that close. I actually pretty strongly disagree with this critique. I would agree if there was really a word that captured the meaning, then I would say, yeah, just use that word, of course. However, generally speaking, when I try to coin words, I do so in situations where I don't think there's an adequate word that already exists. While yes, you could explain the concept in a bunch of words, it really would require a bunch of words. I think it's valuable to have a single word for things when you want to think about them and converse about them fluently. For example, imagine there was no word for derivative in calculus. Every single time you talked about a derivative, you'd have to explain what you meant. It would be cumbersome to talk about calculus; it would be cumbersome even to think about calculus. By packing it all into one word, it just makes it much more fluid to work with, think with, communicate, and reason with. That's what I'm trying to do. I'm trying to give words to things when I think it's useful to compress information into a single word so we can talk about it more fluidly and think about it more fluidly, and also hopefully to help enable people to talk about the concept because now they have a word to refer to it by. Okay, so let's use those two examples that the AI gave. One is the importance hacking. Importance hacking is the name we gave with our project Transparent Replications to a strategy that some people use to get published in academic journals. The strategy of importance hacking is to try to make your study seem valuable, novel, or interesting enough to deserve publication when it's not. Specifically, we mean it as a strategy to get published much like p-hacking is a strategy to get published. In p-hacking, you use fishy statistics or throw away outliers or check lots of outcomes to make it look like a result has a p-value less than 0.05 for statistical significance so that you can get it published. Importance hacking is not about changing the statistical result because with importance hacking, your result probably would replicate; if you redid the study, you'd get the same answer. It's about misleading reviewers about the nature of your result so they misunderstand it and think it's worthy of publication when it's not. Now look at how many words it took me to explain that. It's actually not that easy to explain. It takes quite a few words to explain it. I don't even know if I did a good job, but we're packaging it into one word. I think that's pretty powerful. But why this word? Why bother having a word for it? Just like p-hacking, imagine trying to explain p-hacking before there was a word. p-hacking captures something really important about the way science often fails. It talks about something that's happening that's corrupting science, and just like with p-hacking, in importance hacking, we introduced it for exactly the same reason. We think this is a big problem occurring in science. We think it's making the academic literature significantly worse. We want to introduce a term that points out this exact problem. You could say, why don't you just use a related word, like hype? Hype is a word that everyone knows, and there is a sense in which importance hacking is a form of hype. The thing about it, though, is it's much more specific than hype. Yes, it's a form of hype, but hype refers to all kinds of things. We wanted to call out this very specific behavior. It's a specific strategy people use to get published when a result really shouldn't be published. That's kind of my response.

JOSH: Another critique that an AI made is that your work, especially with the research you do that backs Clearer Thinking tools and things like that, relies too much on self-report.

SPENCER: Yeah, Literal Banana, whom I've had on our podcast a couple of times, also has made this critique not necessarily about my work, but about self-report broadly, and probably about my work too, if I were to ask Literal Banana. To steelman this view, self-report can be very unreliable. There are a lot of reasons for that. Let's go through some of the reasons it could be unreliable. So the first reason it can be unreliable is that people sometimes have an incentive to lie. If you ask someone a question and it's about something really embarrassing, they may not want to tell you, and they might lie. I think this is a really big problem, especially if you're interviewing someone in person about something sensitive, illegal, or something someone might be embarrassed about, etc. A second reason self-report can be unreliable is that people are not actually trying to give you a real answer. Maybe they're just clicking as fast as possible. Maybe they're just messing with you and trying to corrupt your survey data. Maybe there's an AI bot that's trying to take money out of the system if you're paying for a survey. So that's the second reason. A third reason that self-report can be unreliable is that people actually are not able to provide that information. This can come in different forms. One form is that sometimes people don't remember things. For example, if you ask people, "How many times did you such and such over the last year?" they may literally just give you a bad answer because they don't remember or they can't give a good estimate because of limitations of memory. Another reason people can have limited ability to answer something is that they just don't have an accurate self-view. For example, if you ask people about their positive qualities, people tend to give exaggeratedly positive answers about their positive qualities, even in situations where there's no incentive to lie. I think this is because, due to the way the human ego works, we kind of have an internal thing going on where we tend to see ourselves, on average, as more positive than we really are. I don't think everyone does this. For example, depressed people can do the opposite, but on average, people tend to do this, and so this limits their ability to report accurately. There are also subconscious processes that we have trouble accurately reporting on because they happen subconsciously. Those are all valid points. It is also true that we do a bunch of work with self-report. Not all of our work is like that, but a bunch of our work is like that. So what do I think, all things considered? I think self-report really does have all these issues. But first of all, there are smart strategies to get around a lot of the issues. Second of all, the issues don't always apply. So how do you get around people having an incentive to lie? You don't give them an incentive to lie. If you're asking about sensitive subjects, you're going to use anonymous reporting. You don't have it be someone asking them because then social biases come into play, and you have to really assure them it's anonymous. There's no way to trace it back to them. I'm not going to say that eliminates all lying, but I think it vastly reduces lying. Just to give you a data point on this, in one anonymous survey, we asked people if they had ever cheated on a romantic partner in their life, and more than 20% of people said yes. That's a lot of people saying yes to something very sensitive. I think a lot of people are honest in those situations, even about sensitive material. Another thing is you have to be really mindful of what people are able to report on and what they're not. We try to avoid asking people to report on things that we don't think they're capable of reporting on, either because they're subconscious processes or because they're unlikely to remember them. On the other point that people tend to have biases in their reporting, like they might think they're better than they are, we just try to take that into account in terms of the meaning of those self-reports. If we do have people self-report on something like, "How rational do you believe yourself to be?" we're not going to treat that as how rational they actually are. In other words, we know that what we're actually measuring is how rational people report themselves to be, and we're not going to mistake that for the actual true measure of rationality, which is definitely not. We have actually studied that exact kind of question, and we know it's not a good measure of rationality. Self-report can really just be a powerful and useful tool, and perhaps surprisingly to many, in many cases, it's literally the best tool there is. You might think, "Can't we use some cool measuring device?" But it turns out, a lot of times, there is no cool measuring device that measures the thing accurately. Even if there were, it would be ridiculously expensive, and it would be almost impossible to run large studies with it in a reliable way. Take, for example, how happy people feel. There is no measuring device that's better than asking people a bunch of questions about how happy they feel. Maybe one day there will be, but until that day, I think we're stuck with self-report for a lot of things. The last thing I will add is we've also tested our self-report a lot, and we find that it does quite a good job for a lot of things. We show that we can predict some objective things from self-report, and we can show that we can predict some self-report things from other self-report things that are seemingly unrelated, and so on. We do see that there's quite a bit of signal there in self-report.

JOSH: A critique given by an AI about Clearer Thinking tools is that they don't sufficiently account for individual differences. A tool might be effective for an average user or typical user, but might fall down at edge cases or cases where people vary significantly, that kind of thing.

SPENCER: To steelman the AI's view, different people are different. A tool that helps one person may not help someone or might even be a little bit harmful to someone else. A tool is not necessarily going to be able to meet people where they're at or adapt to their situations. That's all true. What I would say is that the main thing we try to optimize is helping people on average. If we put something out in the world and it creates an average benefit, I think we've done a good job. Of course, we also don't want to harm people, and we definitely will try to avoid things that we think are going to harm people, for sure, but the main thing that we are optimizing for is average benefit. I don't think this is really a critique that cuts to the heart of what we're doing. It's certainly true that by adapting to the individual, you can sometimes do better, but it's surprisingly hard to do that. Even the best of the best things that have been developed in many domains, all you could say about them is that they work on average. There are still people that don't respond to them, and there are still people, maybe even some people, that get a slight negative effect. The best medical treatments usually don't work for every single person, even though they're really effective. The best psychotherapies ever created, the most effective ones in the world for specific disorders, still don't help everyone, and they might even make some people slightly worse. It's very hard to ever avoid that critique. Of course, you could try to make things more adaptive, but often it's really difficult to do so because it's often very hard to even find things that work on average well. You kind of spend all your optimization time focusing on that, and then to make it a bit better by adapting to the individual might take ten times longer than that. It's not necessarily always a good use of time to even investigate that. Mainly, such attempts fail to make things better. If you try to make things super adaptive, I think most such attempts fail to actually do a good job on average for the bulk of people.

[promo]

SPENCER: Josh, do you have any critiques of our work that you want to bring up?

JOSH: You've talked a lot about intrinsic values and valueism. My understanding of valueism is the idea that, basically as a life strategy, you're trying to bring more of your intrinsic values into the world. Feel free to correct me on that if I'm wrong, but that's my understanding of it.

SPENCER: That's right. I think of it as having two parts: one, to try to figure out what you intrinsically value, and two, to try to take effective actions to create what you intrinsically value.

JOSH: Okay, so I think my question about this is sort of two-part. One is, is this a normative claim about morality? Is this what people should do, try to bring more of their intrinsic values into the world? And if so, how does that work in cases where a person might value something that's very harmful? Maybe they get real pleasure out of harming or killing other people.

SPENCER: Well, so valueism I do not propose as a normative system. It's not saying you should do it. It's just saying, here's a life philosophy. I'm putting it out there. If you're looking for a life philosophy, for example, let's say you fell out of religion, you no longer believe it, and you're like, 'What do I believe? What should I do to guide my life?" Or, let's say you don't believe in objective moral truth, and so you feel like, "I wish I had a framework to think about what I'm doing in my life, but there don't seem to be objective answers to that. What do I do?" I put it out there as a potential life philosophy to consider, which I think is a life philosophy I find valuable, and I think other people will find it valuable too, but not everyone. That being said, I don't think that fully gets around your critique. I think that partially addresses your critique, but I don't think it fully gets around it, because to steelman your critique, I think you're saying, "Well, if you're saying, figure out your intrinsic values and pursue them, maybe some people's intrinsic values are to cause harm or create suffering or chaos, and maybe in the pursuit of those intrinsic values, they create harm, even if their intrinsic values are not harmful. Intrinsic values are not harmful." Does that accurately describe it? Yeah. So one thing that I found really fascinating, we attempted to discover the different types of intrinsic values out there, and as part of that, we had people submit their intrinsic values. We got, I think it was literally thousands of submissions of people's claimed intrinsic values, and we went through them, categorized them, and deduplicated them. We also looked at different perspectives on intrinsic values from others as well. From this, we created 22 categories of intrinsic values. One thing that was really interesting to me is we didn't see much evidence at all that people have intrinsic values that are inherently harmful. In other words, we didn't really see evidence that some people's intrinsic value is just to create suffering or just to create chaos or something like that. I think that is pretty nice. You could imagine a world where people do have an intrinsic value like that, but I don't think we live in that world. I'm not going to say that there literally are zero people like that, but if they exist, I think they're incredibly rare, which is really good. So I think no intrinsic values I am aware of are bad in and of themselves. In that sense, they're kind of okay, at least okay. Many of them are actually actively good, not just okay, in the sense that they're things like forging connections with others and reducing suffering and increasing happiness, et cetera. They're things that almost everyone would agree are good. That being said, it still could be that someone in their pursuit of their intrinsic values creates harm in the world. I do think every life philosophy, no matter what, has a danger of this. I don't think any life philosophy is immune from the question of what if someone does harm in pursuit of that life philosophy? It could always happen. I think one thing that helps this with valueism in particular is that most people actually have at least some intrinsic values around not causing harm to others, not causing suffering. That kind of helps rein people in, to some extent, in the pursuit of their intrinsic values, as they actually actively don't want to harm other people. There are some people who probably lack these intrinsic values; they just don't have one around. It's not that they actively intrinsically value creating harm, but they may lack an intrinsic value of preventing harm, and in that case, I think they're at greater risk. I think those people would be more likely to create harm in the pursuit of their intrinsic values, even if their intrinsic values are not bad themselves. The other thing I'll say is that, according to my own life philosophy, I would oppose someone who, in their pursuit of their intrinsic values, created a lot of harm because I value a lack of harm or a reduction of harm. In other words, it's not to say that everyone who pursues their intrinsic values will all have compatible efforts in the world; some of them might oppose others. I think that's just natural. If someone's creating a lot of harm by following their intrinsic values, it still works against what I intrinsically value. So I would oppose their actions, if that makes sense.

JOSH: You had a blog post about whether magic is real or not, and apparently there were a lot of critiques about this. So maybe the first place to start is for you to summarize that post for us, and then we can look at the critiques of the post.

SPENCER: Yeah, so before we get into the critiques, of which there are multiple, let me try to express briefly what I was trying to say in that essay. Basically, the argument is that there is a sense in which magic exists, but it does not exist in the real, physical world. Here I'm not talking about magic as in Harry Potter or Lord of the Rings or even Wicca magic. I'm talking about situations where we view things as having a property, and the actual property is psychological, but we think of it as being more than psychological, either as existing physically in the world or having non-psychological properties. It's a little bit abstract, a little bit hard to understand when I describe it that way, but I think it's a little bit easier if I give some examples. For instance, dead bodies after a person has died; many people perceive the dead body as not merely symbolizing the living person, but still being the living person in ways that I think don't correspond to physical reality. Another example would be sex. Many people view things involving sex as carrying special, transformative properties. While that may be true psychologically, I think people view it as though it's something more than psychological. An example would be that people put a tremendous amount of weight on virginity, and in some cultures, no longer being a virgin could be considered incredibly bad, maybe even banish-worthy in certain contexts. People view it as something it's not; it's not just that the person was changed psychologically. It's as though they were fundamentally changed by losing their virginity. I think there are a bunch of other examples of this. I'll give one more. It's something like body energy healing, where someone holds their hands over another person without touching that person, and the person might feel like they were healed. I don't believe that they're actually being healed through that process. I do believe it can create psychological effects; the person might feel something, they might have an experience, but I think it's a psychological experience, whereas some people would say, "No, there's something else, a non-psychological experience occurring, some physical experience, some magical experience." I think there are a bunch of cases like this where we view things as being magical, and we impart psychological properties into what are actually psychological phenomena. To wrap up, the point of the essay is that it's not that these things don't exist. They do exist, but they exist in our simulation of reality that our brains create. The analogy I make is to virtual reality. In virtual reality, suppose there are zombies running around. There's a sense in which those zombies really exist, but they exist in virtual reality. They don't exist in the actual physical world. The way we experience reality is not by experiencing things directly. You can't actually see a photon of light. What you see is the experience of, let's say, redness that the red photon creates when it hits your eye, and then it creates an electrical signal that goes into your brain. You're seeing the experience of red. Redness exists, but it exists in your simulation of reality. I'm saying that these sorts of magic exist, but they exist in our psychological experience of reality. It's not that the person laying their hands over someone actually healed them; it's that they created a psychological experience for that person.

JOSH: One of the main critiques is that you did this redefinition of magic where most people have this definition that corresponds to something supernatural or ritualistic, and you have redefined it to be something psychological. In doing so, there are sort of two side effects that have occurred. One is that you're, in a way, no longer contending with the definitions that people normally use, and the other is that you are potentially ruling out real things that have occurred in the physical world that are measurable but that we don't yet understand scientifically, even though people have labeled them as being magical.

SPENCER: Just to make sure I understand those critiques, I'll try to steelman them. The first aspect is that I'm kind of redefining the term magic because what I'm calling magic isn't really magic the way it's broadly used. I would agree with that. I did try in the essay to explain what I meant by magic, but I think I could have done a better job of really clarifying. You might say, maybe I shouldn't have named it magic. Maybe I should have said something else, but then I didn't know what else to call it. If I call it something else, then people may not know what I'm talking about. I think it's a bit tricky, but I do kind of agree with that critique that what I'm referring to doesn't refer to all of magic, and I could have done a better job of explaining that and exactly what I meant and ruling out the kinds of magic I don't mean. The other critique is basically that maybe some of this stuff actually does have a non-psychological presence, or it's doing something that's beyond the psychological, and I'm kind of ruling that out. I think that's a fair point. I'm making a choice there and saying, "Yeah, I don't think those things are real, but for someone who does think they're real, then, yeah, absolutely." It's a very fair critique, but that's just my opinion; they're not real. My piece does hinge on my belief that those are not real because if they were real, then the critique wouldn't follow. That being said, the point could still stand about other things. Maybe you think that a dead body doesn't actually carry the properties of the living person anymore, most of the interesting properties of the living person, but you disagree about Reiki. Fine, that's no problem. Then you just say my essay may apply to the things that you agree are purely psychological.

JOSH: One analogy that I think you used in your blog post is about redness and how it's a perceptual phenomenon that occurs inside our brains. If we had a different kind of brain or a different kind of eye, then redness would be a different phenomenon, or maybe we wouldn't see red at all. You compare this to magic, and the idea that magic is something that happens inside a person's brain rather than out in the real world. One of the critiques of this is that magic has a sort of meaningful element, or purposeful element to it, intentional element to it. That's not the same as redness. There seems to be a significant distinction there, and perhaps using redness is not a good analogy, is the critique.

SPENCER: Yeah, I agree with that, in the sense that I think it's far from a perfect analogy, because indeed, the way we experience magic, as I'm calling it, is quite different from the way we experience redness. What I was hoping to do in that analogy is just to point out how people generally think of something as existing or not existing. I'm trying to draw a distinction there. Let's talk about another category. There's existing in the physical world, like atoms. There's not existing, which is totally made up. Then there's this other category. I'm trying to use redness to get across what that other category is, which is things that exist in our minds, but aren't out there physically in the world. Redness is an example of that, even though it's not a perfect analogy.

JOSH: Another potential critique is that the stance you're taking here is very physicalist, very materialist. I don't know if you say it explicitly, but it sort of rules out anything supernatural. It also neglects potentially broader perspectives on what consciousness is, like panpsychism, because panpsychism might be the kind of view that could account for more mystical or numinous experiences.

SPENCER: So panpsychism is a view that says that maybe everything is conscious, maybe consciousness is everywhere. I think there's a variety of related views. One view says everything is conscious; maybe an atom is conscious too. Another view says maybe consciousness is actually the fundamental thing, and the physical material world is secondary, and consciousness generates everything. There are other views around that. I agree with the critique in the sense that it's true that if, for example, consciousness was fundamental and the material world didn't actually exist, the material world only exists insofar as people imagine it, or consciousness generates it, then I think that would really change the paradigm of what magic is, what it could be, and what's possible. I don't believe in that paradigm, but I do think that is a valid critique, in the sense that if that paradigm were to hold, it would change the nature of magic a lot and how you might interpret it.

JOSH: One final critique comes from a reader of the One Helpful Idea newsletter. They said they were reading through this particular newsletter and saw an image. The image had a picture of 10 people in a boardroom, with seven or eight men and only two women. They were saying, "This is an AI-generated image, so surely you could control what things were in the image. Why were there so many men compared to so few women? Is this not a problematic way to generate content?"

SPENCER: So if I understand the critique to steelman it, I would say, "Yeah. Putting out representations of boardrooms where there are only two women and eight men might reinforce stereotypes against women, that women are not suited for business or not suited for positions of power, and these are stereotypes that are harmful. They've been around for a long time, and it's good to try to seek gender equality and not spread stereotypes." Do you think that's a fair ceiling of it?

JOSH: Yeah.

SPENCER: My response is a fewfold. First of all, I will say that there are certainly times when we might accidentally support stereotypes, and in doing so, we might be contributing to problems, and I think we should try to avoid that. Now, do I think we were doing that in this case? A kind of funny thing about this example is, actually, I believe it was a newsletter about how groups can be biased, and I think the lack of representation in the image actually kind of worked synergistically with the idea in the newsletter to reflect that. So I don't think it was necessarily inappropriate for the context of talking about how groups can be biased. The second thing is that, suppose one were to produce a bunch of newsletters, and sometimes there are more men than women in positions shown in the images, and sometimes there are more women than men. It's sort of random. Suppose a group is writing many newsletters, and in a bunch of those newsletters, they're talking about people in positions of power, and half the time in the images, they have more men than women, and half the time they have more women than men. Would it be fair then to critique them every time they have more men than women and say that's representing a bias? I think not in that case, because unless you're going to perfectly make it 50/50 each time, it seems a little bit unfair, given that they're actually balancing it out. My point is that it's a bit hard from any one example to say that it's biased because you have to look at the larger picture, and anytime there's an unequal number, I don't think that automatically means it's biased. I'll tell you another example where someone gave me a very similar critique. I had an essay, and there were three examples given, and two of the examples were about women and they were longer, and one of the examples was about men, and it was a shorter example. A person said that I was biased. I don't think there's anything inherently biased about having two women and one man. In fact, if you have only three examples, unless you're going to include non-binary or other genders, you're going to have to have it be imbalanced. But I suspect what made them call me out on this in particular was the feeling that by the nature of the article, it could be associated with stereotypes of women, and therefore, by using two women examples and one man example, maybe it was reinforcing that stereotype overall. I think these critiques are worth keeping in mind. I do think we should try to avoid reinforcing unfair stereotypes. I think that's a worthy goal. That being said, I also think it's pretty harsh if you're a writer who writes a lot and you sometimes use examples of men and sometimes examples of women. I think asking that every single one should be perfectly balanced, that every image should have half men and half women, is going too far. Or that every essay should alternate men and women examples is going too far. I think it's something that's worth keeping in mind. We should be mindful of it. We should try not to reinforce stereotypes. But I also think we shouldn't be obsessive about perfectly gender balancing every single essay, every single image. What do you think about that?

JOSH: I think that all makes sense. I think it probably matters how the image is being used, whether you're referencing it or endorsing it.

SPENCER: Because sometimes you want to show something that is the way that people see it, or is the most common thing. It is true that CEOs of Fortune 500 companies are less likely to be women, and if you were depicting CEOs, would you want to gender balance it, even though that's not the way it is in the real world, or would you want to depict it as it is? It gets a little tricky saying, "What is reinforcing a stereotype versus what is representing the way things are, even if that's a bad way that things are?"

JOSH: How much effort do you think should be put into, let's say you go to ChatGPT and say, "Please generate an image of ten people in a boardroom for me," and it generates this image of eight men and two women. Clearly, this is probably something it's learned from its training data, that this is the way things are, on average, likely to be, or something like that. How much effort would you likely put in to say, "Well, actually, I'd prefer to have a more even balance of men and women," or something like that in the image?

SPENCER: Yeah. I think that the effort really should just depend on the level of harm that you anticipate. If you think in this particular instance you're trying to, let's suppose it was an article about gender bias among CEOs, where women are woefully underrepresented, you might purposely choose an image that has fewer women to illustrate your point. So, I think it's very context dependent how much harm it causes. And I think you just want to be mindful of the level of anticipated harm.

JOSH: But you would agree, I think that AIs do have these kinds of biases because of the way they're trained, right?

SPENCER: Yeah. Well, interestingly enough, Google got in hot water for trying to correct these biases but overdoing it. A bit like when people would ask for examples of Nazi soldiers, it would show Asian women, which we know that Asian women were not Nazi soldiers. And it was because they were trying to get the AI to be more gender balanced and so on. But it kind of went too far and sometimes made these errors. Yes, I think that AI being trained on images on the internet, being trained on text on the internet will tend to reflect the biases of the internet, which is the biases of people in general, to a large extent, at least people who are internet users, unless we do something to try to correct that. And there are different approaches trying to correct that. There are many different approaches AI companies have used, and I think some of them have made some progress. Some of them have gone too far and caused other problems. But yeah, you will often find biases in these AIs.

JOSH: So as we wrap up, are there any critiques that have stuck with you over the years? Maybe one that you had when you were younger that is still very formative to your work today?

SPENCER: Yeah, one critique I got when I was in college, and then again later as an adult, is that sometimes people can't tell how much I like them, and this can actually be a problem. If I like someone, I want them to know that I like them. I realized that sometimes created insecurity in other people. Multiple people expressed that to me, and something I've tried to work on is trying to communicate that to them. As we get to know each other, obviously, it's not something you do if you just met someone, but if you become friends, you want to communicate your feelings towards them. I'm trying to be better at communicating positive feelings towards others. Another is that I tend to have a not very expressive face, so just trying to compensate for that, if I feel warmly, trying to express that in my face more, to express really what I'm feeling, even if my face isn't naturally always expressing it, and just kind of communicate that warmth in other ways. I think that one was definitely a critique that was important for me to hear. Well, Josh, thanks so much, and thank you all for listening as well. If you have other critiques of me, send them to me, and maybe we'll do another episode at some point of additional critiques. You can just send me a critique, or you can say you're submitting a critique for a specific episode that you'd be excited to have me respond to. So yeah, Josh, thanks.

JOSH: Yeah, thanks for having me again.

[outro]

JOSH: A listener asks: "How can we best and most quickly tackle the climate crisis? What's the most effective thing for each of us to do as individuals?"

SPENCER: In my view, there's only a few strategies for working on reducing man-made climate change that are realistic. And I think people often try really unrealistic strategies. So the three realistic strategies in my view — and by realistic, I just mean that they seem much more plausible that they could work, but I don't mean that they're easy, right? So the first is government collaboration. If major governments around the world took climate change really seriously, they could basically make a pact to do something about it. But it would essentially have to involve China, the US, India, and Europe. They have to all be involved, and then other countries obviously would be great if they were involved too. But without that, I think it's really, really tough. And of course, very few people have the power to make that happen. But by electing leaders that are interested in that, then that does increase the probability that that could happen. So leaders that are willing to create a kind of multi-country collaboration. The second thing that seems like a plausible strategy is better technology. I think so many organizations have tried to get individuals to act against their own selfish interests to help the climate by getting individuals to try to pollute less and stuff like that. And I think those strategies just really, really don't work on average. I think there's abundant evidence of that. But if technology improves, it may be in people's interests to reduce their greenhouse emissions. And when it's in people's interest to do that, I think they will do it. And so that is the second plausible strategy is more investment in technology. What that could look like at the individual level: it could look like starting a company. It could look like joining a company that already exists or playing around with new ideas for technology, learning about new technology, et cetera. And so I'm not an expert at all in this area, so I don't know exactly what technologies are needed. But I do know that there are a bunch of technologies that, if developed, could really help because they could shift it so that it's actually more in people's interests to pollute less. And also, you can talk about carbon capture technology and the potential benefits of that. The third main strategy that seems plausible to me is increasing pressure on mega corporations because there really aren't that many corporations that account for a lot of the greenhouse gases. I think it's just a few hundred that account for a surprisingly big proportion of it. And so if there was a way to influence those companies, for example, to get them to agree with each other to do things differently or investor pressure for the people that own their shares, perhaps that could actually move the needle and make a difference. And then finally, I'll just add there are meta strategies. Meta strategies are things like getting other people interested in working on these existing strategies that might be promising or creating more momentum towards those strategies. So that's what comes to mind. Obviously, it's hard to know what to do at an individual level. But if you really want to do something that's plausibly effective, I would urge you to think about those strategies and think about where your personal actions might fit into those. Of course, I might be wrong. There might be other strategies that are really great. But I'm pretty skeptical of all the other strategies that I've seen.