SPENCER: Hello, everybody. Thanks so much for coming. This is a live podcast taping of the Clearer Thinking Podcast. Raise your hand if you've ever heard the Clearer Thinking Podcast before. Okay, we've got some people who have listened before in the audience, which is awesome. Because it's a live podcast taping, it would be great if the people listening at home could know that there's a live audience. So I just request that when I say, "Welcome Dean," if you could just give some sound or clap or something to indicate that there are humans here listening to this, that would be awesome. Just as a little bit of background, I'm so excited to have Dean Karlan here. Dean's career is staggering. It's really incredible. I'll just give you his quick bio. He's Distinguished Professor of Economics and Finance at Northwestern, co-director of the Global Poverty Research Lab at Northwestern, former chief economist of USAID, and he's also the founder of Innovations for Poverty Action (IPA). They've done literally hundreds of randomized controlled trials across the world figuring out what works for development. I'm so excited to talk to him today and to have you here to listen. Just a little background on myself. I'm the host of the Clearer Thinking Podcast. I also run clearerthinking.org and I'm the founder of Sparkwave, where we do psychology research, and our aim is to try to make psychology into a more robust science. We do a lot of different research in that area. If you enjoy this conversation or love it, check out the Clearer Thinking podcast later, we've got lots of interesting guests on there. But with that, let's get going. Dean, welcome.

SPENCER: So in 2022, you became the chief economist of USAID, which was literally the largest agency in the world for doing foreign aid. I want to talk to you about how you approach being really effective with that work. But before we get there, when you arrived at USAID, to what extent were they really taking into account evidence about what really helps and what's really cost-effective?

DEAN: It was a big place, so the answer is mixed, as you can imagine. There were some areas where the work was more proximate to the evidence base. Global health, as a general rule, was closer to that space. The medical field, as a general principle, is more evidence-based, so that's a natural area, but in other areas, they were further behind. What was striking upon getting there, though, is this wasn't something that I concluded from afar. This was something that they told me when I got there. There were a lot of willing and eager people at USAID when we arrived, and we were creating this office to help do this. They knocked on our door to say, "These are the kinds of changes we would like to make, that we've wanted to make, but we couldn't for various reasons." So great. Let's get going.

SPENCER: What did they say to you about that?

DEAN: What did they say?

SPENCER: Yeah, in terms of what they thought you'd have progress?

DEAN: Good question. Put them in two buckets. In a sense, I'm kind of describing what our strategy was. It wasn't that there was someone there who could have spelled out the strategy, but one was recognizing that the work we were doing academically wasn't actually taking that final step needed to inform the person with the pen who was writing a procurement about what they needed to put into the procurement. There was still a gap in that final step for how to translate the evidence going into academic papers into the kinds of things that needed to go into a procurement. The first recognition was, "Hey, we've heard of these papers. We've heard of this space." But meanwhile, that doesn't tell me how to do this procurement and what I need to do over here. The other thing that was loud and clear was nobody was going home at three because they were finished with all their work, so the idea of just adding a step for them, "Hey, now you have to have this extra section of your whole procurement process in which you do this evidence review, figure out what the best evidence is out there for how to address this problem, and how to adapt this into your procurement." With what time were they supposed to do that? That needed some changes in the process and direct support. We had effectively what can be described as a two-pronged strategy. One was being in-house consultants, where we were also a bridge to academics, bringing in the academic work, but then doing that final stage of mapping the work to figure out what that tells us needs to actually happen in a call for proposals for implementation. The second were things that were more institutional enablers, regulatory documents within USAID about how procurement works, how strategies are set. We needed to make some changes at that level as well, and we were making them little by little.

SPENCER: And why do you think that didn't happen much earlier? Because you could imagine, people knew about the evidence. People wanted to do good, presumably.

DEAN: Two reasons, I think. One is there was an effort about 20 years ago to bring more evidence into what USAID was doing, but more on the evaluation side, not on what we would call the evidence use side. We actually have two teams in my office literally called just that, evidence use and evidence generation. Twenty years ago, there wasn't enough evidence out there that was concrete enough. You might be doing a program on education and working with primary school children recognizing there are children lagging behind. Someone might say to you, "Oh, that's interesting." I heard about a program in India that did that, and they ran a really nice randomized trial. Here it is, and you're in El Salvador. Should you really just take that study from India and say, "Great, let's go implement this?" Probably not. You should learn from it. It might motivate you to set up another test in your context to figure out how to adapt it. But you don't want to move policy globally just because someone, somewhere did one study. Twenty years ago, that's more of what we were looking at. We were looking at a series of one-off studies. We didn't have a collection of evidence that was dominant enough across the board of the things USAID was doing where it could systematically say, "Great, what's the evidence based on this? Let's use one other example." Cash transfer is something talked about a lot here and very supported by many at this gathering. Twenty years ago, there were roughly zero randomized trials testing the impact of unconditional cash transfers. We put together a meta-analysis that had 115 randomized trials of unconditional cash transfers. We can now say a lot more about what to expect from that. If someone wants to use that evidence to inform how they're going to roll out a cash transfer program, they can do that with a lot more confidence now than they ever could 20 years ago, so that evidence base has changed radically. The other issue, I think, is just best described as nobody is opposed to being more effective. But if you ask someone, "What changes USAID needs to make, you're going to get a longer list of things?" It was just one of many things, and that's a question of strategy and focus from people in leadership and what they choose to approach. The administration that asked me to come in, I was not a political appointee, but it was the political leadership that said, "Hey, please come in and help do this." This was something they were very committed to, and so it worked. It's not that I wouldn't say that anybody prior to Mark Green, who was under Trump, was opposed to evidence, but there were just other choices made in terms of what to focus on regarding any reform and changes.

SPENCER: So getting concrete, when someone's going to sign their name to that money being moved, how did you change the process? What did they then have to do, or how did the evidence get brought in?

DEAN: So we were still in the middle of making changes, to be clear. We definitely had some plans for the longer run in terms of what we wanted to see, and based on the fact that we had to build our team. We started off with one person in the office, and by the time things fell apart, we were around below 30s in terms of the number of people in our office. So it's a bit of a slope, right? What I can describe is where we wanted to be, where we got in terms of the types of changes that we were able to make. USAID, at the end of the day, is a very decentralized organization, and the way the money flowed, basically, was Congress, in theory — I have to add that now — has the power of the purse. Congress would tell us, "USAID, we want you to spend this much money on this cause area, and we want you to spend this much money on these countries," and some grid from that gets put together. But for the most part, there were very few conversations among senior management about those high-level allocations. Their hands were tied. There was no ability to take the health money and just move it into agriculture and things of that nature. Those allocations are done via Congress, in collaboration with the leadership to kind of make that grid work. But then what ends up happening for the decision on what to actually fund? Most of that is made by people living in the countries, the Foreign Service officers and the Foreign Service nationals who are then told, "Okay, in Tanzania, here is a budget. You have $50 million over the next five years. You need to spend this on primary school education in accordance with this congressional act. Your main goal is to improve test scores for children." They actually have a fair amount of power for what goes into that procurement. That's the part that we needed to work with. That's where we basically opened up shop and announced to all the various offices around the world, "Here's what we can do, here's how we can help. If you want to bring in more evidence for what you're doing but you haven't been able to, that's what we're here to help do. We'll do the heavy lifting, we'll do the evidence analysis. We'll share it with you. We want you to be a part of that process, and then we would actually help by making the edits to the right documents to get that evidence in and then have the key proposals include what we wanted to do." A lot of times, there were some proposals that were very prescriptive for a product or a service. But a lot of the proposals that we saw at USAID stated the problem without saying what the solution should be. They would state the problem. They might have what they would call illustrative examples, and it would be a little box. It wouldn't have too many details. For the most part, they were pushing it to the contractors, maybe large nonprofits or for-profit firms, who were bidding on these awards. When they would do that, they were handing the decision-making as to what happens to the contractors effectively. You could argue it's a little bit of a collaborative process, but it's not in the procurement. That's the key thing. The proposal that goes out the door just says "Food security, northern Kenya. Go," what are you going to do? The proposals that would come in often wouldn't even say what they're going to do. They would talk more about the process of what they're going to do. Yet, when you look at what they're going to do, what they actually do, it's not that different from year to year. The difference that we wanted is to say, "What's the problem you're trying to address? Food security in northern Kenya? Great. Here's the analysis of the top 10 ways that people go about dealing with that. Here are the two of those that stand out as being the most cost-effective ways of dealing with that problem over and over again. Here's the evidence, here's the sites, here's how we're coming up with that." That's what we want you to do, but in any one of those, there are always going to be questions you need to figure out about how you're going to do that in that country. That was some of the work that we needed to do, and we were doing with those offices because it's not as simple as just saying, "Go do this program." If it's a cash transfer program, even one of the more simple programs, which is why it's popular, you still have decisions you have to make: how much are you going to transfer? Who are you going to transfer it to? What's the modality, frequency? Are you going to tag it with any sort of messaging or even just a one-hour session? What are you going to do? All these are decisions you have to make to contextualize it to what you're doing. Part of what we wanted to see the procurement do was say, "Look, here's your best buy. This is what we're thinking is the most cost-effective. Here are the six questions you need to answer as someone bidding on this in order to show USAID that you understand what this intervention is, and you know how to implement it, and you know how to think about what needs to be done in this country to do it well." That's what you can judge on, or you could judge on how much they are going to charge to do it right. That is obviously a way of helping to get more scale. If someone is just able to do it for cheaper than someone else, and it's the same exact outputs they produce, then you can reach more people. That's what we were doing when working with procurement and getting into the details. Now, the one change we wanted to see happen that we hadn't made yet, this was on the plans for 2025. We never got there. The ideal procurement, as far as I'm concerned, is one that says what I just said, but then says, "By the way, if you think you can beat the thing that we think is the smart buy for this, bring it on. But now please do an impact evaluation along with that so that we can learn. You can learn, others can learn whether that's actually better than the thing that we thought was going to be the best thing to do here." I wanted to see that tagged in, and this is a good example of where the rules of the bureaucracy and the procurement would get in the way, and we needed to do more institutional work because the response that I got when I proposed that in a couple of instances was, "Oh, that's an evaluation strategy decision. We do that later. We figure that out after we have the contractor in place." My attitude was, "But that should be part of the selection. If they can bring an innovation to the table that they think is worth testing, I want to hear that before we choose the contractor, not afterwards, and I want them to commit upfront to doing a good evaluation." That was going to require that second category of work because the reason we were told no to that wasn't an opposition to doing that. It was literally saying the rules don't allow us to do that, so we have to change the rules. But that's a longer game.

SPENCER: It reminds me of this idea that you have of "use it or produce it." I imagine almost everyone in this room is convinced that you should be using evidence to figure out what to do to help the world. But you kind of say, "Well, you should be producing evidence." Sometimes that's as helpful, or maybe even more helpful.

DEAN: That was definitely one of our slogans. I still dream of that phrase in my mind, "use it or produce it." It refers to the cost-effectiveness evidence, and that was always our mantra. It also realizes it's just a short phrase. There's a lot baked into it. One of the other ideas that's very much baked into it is that if you don't think there's enough evidence out there to use, that's a moment that says, "Okay, great, then I guess you ought to produce some because we ought to learn more." Likewise, one of the other things you want to think about is when do you decide not to do impact evaluations? Take the cash transfers as an example. As I mentioned earlier, there's been 115 studies. This happened at USAID where someone was doing a cash transfer and they said, "Great, let's do a randomized trial to measure the impact," I was like, "Why? We have 115 studies studying this. What's different about the 116th that is going to change what we think about cash transfers?" Please tell me something different about what you're doing. If you're trying something a bit more innovative in how you do the cash transfers, then I'm all ears. But if you're just doing it for the sake of accountability, I would actually call that unethical because that's money going to measure something to draw an inference on something that we already have a really good number for. It's not going to change your opinion too much at all, and yet that's money that could go to help more people. You're literally taking money out of people's accounts that they would otherwise get in order to collect data. That's not ethical. You have to have a lens when you do the "use it or produce it" slogan, it is serious about it being an "or." Sometimes, if you're using it and it's good, then don't produce it because you could then just use that money to do more of whatever it is you're doing and save more lives, transfer more money, whatever that is. We need to think hard about when we spend money on research and make sure that it's being done in a way that we're going to learn something. It's not being done on something that we already know, and it's not being done strictly for accountability purposes to look backwards and say we did what we said we would do. If that's all you need to do, you don't need to do randomized trials. You can check that programs were delivered, they did what they said they would do. That's what you need to do if you actually know that the program works. It's no different than medicine. Imagine taking a pill like Tylenol. We don't need to run randomized trials to know that Tylenol can reduce headaches. It doesn't mean it's always going to solve your headache, but we don't need a randomized trial to know we're going to do that. You do immunizations on diseases. If you already know how immunization works, we don't continuously run more and more randomized trials. We just make sure that the shots are getting in our arms. The same thing applies to lots of areas.

SPENCER: It's hard to find a meta-analysis that doesn't conclude in the end, "more research needed." But sometimes no more research is needed.

DEAN: That's excellent. I think there are a lot of spaces like that. I agree it's a problem that we have this knee-jerk reaction. There are definitely things that are well past, particularly when you think about the scarce resources and the overall allocation of how much we should be spending on helping versus learning. The learning needs to be motivated by how it will change how we're doing things in the future. If it doesn't have that path to changing things in the future, then we should just spend more money helping people.

SPENCER: Now, I've heard it claimed that even prior to your tenure there, USAID was saving millions of lives a year, for example, through the HIV and AIDS programs. Do you think that's accurate?

DEAN: Yeah, absolutely. It's not an easy number to estimate. I don't know, but there was an article in The Lancet that came out earlier this year that got a lot of press, and even if it was off by half. I'm not claiming it was much. Let's say it is. It was still a big number with a low cost per life saved. It's exactly what you would expect from a program delivering life-saving medicine. When you're addressing that last mile issue on diseases that actually kill people, it's not that hard to have that path to saving lives. Those numbers seemed reasonable over the decades, given the nature of the work. The harder numbers to put a pin on are things on the economic side. There were programs leading to important changes in people's lives in terms of economic activity, agricultural returns, things of this nature. Those are harder to aggregate because it's not as mechanical as a pill. When you're providing treatment for HIV, you know exactly the efficacy of that drug. You can multiply it by how many people received the drugs, and voilà, you get some good numbers.

SPENCER: And during your time there, to what extent do you feel you were able to shift the effectiveness of the organization?

DEAN: We had a bean counter for ourselves. We tried to be true to holding ourselves accountable with goals, but at the same time, we were also well aware that we couldn't. We weren't randomizing which countries we worked with and which staff we helped.

SPENCER: And roll of dice when you're talking.

DEAN: No, I talked and I joked about it, but I knew that it would have been a poorly designed randomized trial. It also would have been a poorly designed study. Our bean counter was at 1.7 billion. That is the magnitude of the awards where we were heavily involved in helping to shift what the money was going to something that has a strong evidence base, and that was exhilarating. The 1.7 was just getting going. It took six or seven months just to create the office and grow the team. The 1.7 is a little bit of the gift that keeps on giving because of the bureaucratic nature. Once you change something, there's a certain inertia that kicks in. The renewal of the award would likely do more of the same. Another country that's doing the same has the same problem, sees the solution over here, and figures out how to morph the procurement just like I described before. You might actually have the same five questions you want to ask about how you're going to adapt this, but it's a different country. The answers may differ by country, but the underlying questions are often quite similar in terms of what you need to ask about how you're going to implement this program in that context. If we did an award for 150 million on a program in Kenya, and the next year, the Tanzania country office was doing a similar thing, they see the procurement and can use it. The 1.7 would have grown more.

SPENCER: What were some of the specific shifts in how you were using the money that you think had the biggest bang for the buck?

DEAN: So there were, the biggest piece was a program called the Graduation Model. And this is a model that is, and I think I could be off on the exact number, but I think it was right around four to 500 million of the 1.7 was this particular model.

SPENCER: And when I was talking to you the other day, you mentioned that you felt that was one of the most impactful areas of research you'd ever done within the Graduation Model that really fascinated me.

DEAN: I was involved as a researcher. Our role with it at USAID was a bit interesting in the sense that some of the decisions to move resources into it were made not attached to any particular award, but the decision made from the humanitarian affairs team, recognizing the evidence base, was actually made before I got there. But that was just more like a strategy level. "Hey, we ought to do this more. There's strong evidence, not this other program that we're doing in rural areas that does not have strong evidence of working." And then when we got there, we were basically able to help them actually execute that because it is complicated. There's a certain element of the program that is a little bit more on the complicated side, somewhere in the middle of the spectrum of how complicated things can be. It's certainly more complicated than a mere cash transfer program. There was a lot to help do in terms of figuring out which questions to ask, as I was describing earlier in the program. What do we need to help design the program in a given context, and elicit from potential contractors and nonprofits who are going to bid to do this, to help judge and guide them in what it is that we're looking to do. And so there was a lot of work involved, but the actual decision to do it was something that happened basically when we got there. And basically, what this core program is doing is you can kind of think of it as a cash transfer on steroids, although that's not a totally fair description, because there are some iterations of it that actually don't include cash transfers and have proven to be effective that way. But it's trying to address economic activities for low-income households, usually in rural areas — there have been a few efforts in urban areas, but that's separate — so it's trying to help rural households that are very poor develop their own stable source of income, develop a productive asset base so that they can absorb shocks better and also have a stable source of income. And so it usually has at its core some sort of lump sum cash transfer or asset transfer. In the original iteration of the test that we ran 15 years ago, it was often animals that were transferred, goats and sheep or guinea pigs in Peru and things like this. Over time, as mobile money has become more popular and the programs have gone to scale, it's easier to procure buying a thousand goats; it's much harder to procure buying 20,000 goats. And so as programs scaled and got folded into government programs, there has been a shift towards cash rather than assets. And then there's training, usually, in what those assets are, how to raise guinea pigs most effectively, or beekeeping in Ethiopia. Often there's an element that's referred to as life coaching, which is some sort of engagement from a field agent to provide support to the household. Some of it is about information, but some of it is about setting goals and aspirations and helping them stay on track to build this asset base to help support their longer-run income aspirations and access to savings, sometimes that is a formal account or mobile money accounts. In other cases, it's an informal savings club, but that's exactly what I mean by that contextualization. What is that savings going to look like? Just because you see a randomized trial from India that had a particular type of savings as part of this program doesn't mean when you go to Kenya and you roll out this program, you can just say, "Oh, do the same thing." That savings institution might be structured differently. Travel time might matter, rules for how you withdraw and things like this might matter, and so you need to adapt the savings aspect to figure out what makes sense to do in the Kenya context. And that's part of the contextualization step. So that's what the core program is. And we've now seen about 35 randomized trials on this program, and again, finding very consistent and longer-run effects compared to other kinds of interventions, typically on income, well-being, and consumption for the households.

SPENCER: It's interesting. In the EA community, effective altruism community, there's a lot of talk about cash transfers, but I've heard very little, if any, talk about the Graduation Model. And you might think that in a certain sense, the Graduation Model is trying to do what cash is doing, sort of the best-case scenario where we give people cash. You want that cash to benefit them for many years, not just be a one-time consumption thing, but something that creates investment and creates long-term benefits. Is that a fair way of describing this program?

DEAN: Sort of, yes, everything there. I wanted to add a couple bits, which is, there are times when cash wins. From a policy perspective, for instance, yes, we like long-run impacts, but if there's really an immediate short-run crisis, then you need to address that. It's not as simple as just saying the long-run is always going to be better. The other is that graduation has more implementation fidelity risk. There are more moving parts. It's not as complicated as some things that were out there. I think this is part of the big change that we helped make. The prior efforts to address the problems that these awards were trying to address were much more complicated. They would be doing 10 or 15 different things, and it led to no average impact from the few impacts we saw. It was also harder to evaluate because of the kind of shotgun approach that it was taking. Those were harder to do and keep implementation at a standard. Graduation is harder than cash to do. If you're in a setting where that implementation risk is real, then go with cash; that's a clear advantage. The other thing I would add to what you said is there's definitely an in-between that we're seeing that is very exciting. There are efforts to strip down graduation and say, "Okay, how do we get it cheaper? How do we get it simpler?" That's a very good iterative learning process that that space is going through. In its simplest form, it is a lump sum cash transfer with a little bit of a nudge attached to it. Some of those have been done effectively, are really cheap, and yet still generate big impacts. We don't have quite the simplistic formula to say this is the way to do it everywhere. Every time I see a result, it's difficult to draw perfectly consistent conclusions about when cash is needed and when not. We're seeing some variation. The simpler versions actually look more like a cash transfer than a complicated program. They look more like a cash transfer with a training program attached to it.

SPENCER: Besides these caveats you mentioned, such as implementation risk, if you think about hitting the Graduation Model head to head against cash transfers, what are the differences you see in outcomes?

DEAN: The biggest one is going to be long run versus short run. More of a cash transfer gets used for immediate food and immediate housing. There is an argument to say, "Well, that's fine; that's what people are choosing, and let them choose." The other doesn't quite give them that same choice. It does in the sense that you can transfer for animals; they can turn around and sell them and then buy a roof, but it's making it a little more costly for them to do it. You do see a bit more of the money being used for productive assets versus durables for household construction and immediate food. You do see long-run impacts from cash. I want to be clear; I'm not saying it's an all-or-nothing situation. In fact, one of the things I found striking about the cash analysis we've done is how often the long-run results are there. There's more of it in the graduation space, which I think is more consistently seeing the long runs kick in. In terms of outcomes, when we see the studies that measure them, they often look at the same exact outcomes. They care about what the income-generating activity the households are doing is. Does what was transferred lead to a substitution from one income-generating activity to another, or is it adding more? That is one of the other differences we have seen. In most cases with cash, we always see an increase in labor supply and work investment, even from straight-up cash. This is one of the most important results from the randomized trials of cash. If there's one thing to worry about with a cash transfer program, it's that you transfer money and that leads people to work less. It's probably the worst thing we've ever done as economists: teach economics 101 with nice little charts about labor leisure, telling people that there's a trade-off. If you transfer money, you work less. That is a nice little simplistic model that ignores the reality of low-income households who are capital constrained and need money to make money. It completely misses the boat in being a predictive model that in any way predicts reality in developing countries.

SPENCER: So the data is pretty clear that people don't actually work less.

DEAN: Oh, and they work more. They need money to make money. So you want to start a business, then here's some money, and then they work more. Without the money, they couldn't start the business. So we either see zero effect or positive. I think we have almost zero, maybe one, hopefully zero. Studies we've seen where it's negative. There are probably some that are negative, but very few and not in small samples, and it's a very unambiguous story of zero or positive from the cash transfers in low-income countries for working more. But with the Graduation Model, one thing that we have seen in some cases is that with the cash, you might work more, but you do see some substitution. You work less on one thing and more in another. In aggregate, you're working more, whereas with graduation, one thing we have seen, at least in one study that we did in Ghana, is that the graduation made it so that all that happened was they worked more. There was no substitution away. Part of the logic of that — I'm a little hesitant to really firmly claim this — but the logic that we think is supportive of it is that the spirit of the graduation program is providing more life coaching support to the household. The idea is that with that life coaching aspect, it provided increased capabilities for the household to expand their income-generating activities and help them see this new opportunity as something to do as well as what they're doing, whereas without that, it was difficult for them to add an activity. They did a bit of substitution, so they did end up working more, but with substitution from other things. That's not something that we have super robust results from, like 30 places to be able to make that comparison, but I have noticed that in a few.

SPENCER: So the Graduation Model can be an alternative to unconditional cash transfers. If you look at GiveWell's ratings, my understanding is that they put other interventions that are more cost-effective than unconditional cash transfers. Where do you see unconditional cash transfers fitting into global development? Where should they be used?

DEAN: I actually see these often in the same conversation. To be clear, even graduation, there's a world in just thinking of graduation as a lump sum cash transfer and then some with a bit more. At USAID, we were very active in helping and trying to encourage more cash transfers to be doled out. There is less implementation risk. There are a lot of complications as a government agency in rolling out programs, and when you can do things in simpler ways, you will often get better outcomes. You don't have that ability to be as granular and fine-tuned. We had actually released — in government jargon, or USAID jargon — a position paper. It had a little bit less onerous requirements from a clearance perspective in terms of the various parties that needed to clear. The funniest advice we got at some point was not to call it cash transfers because the fear was that it had this image issue, and we were told to call it market-based assistance. This was advice that came from a Republican in Congress. I thought this was brilliant, and we started doing that because I just wanted to see good policy happen. At some point, we changed it because we realized that was a bad phrase; market-based assistance tells you so little about what it is. Every time you said that, we had to follow it up with an explanation of what we were talking about. We called it direct monetary transfers. There's also a different issue, which is that cash transfers in USA nomenclature often refer to government-to-government cash transfers. Every year, we gave a lot of money, hundreds of millions of dollars, to the Jordanian government, just straight-up budget support unrestricted. But it's government-to-government; that's not for a program to give to people, that's just support to the government, and that was called a cash transfer. It also helped separate, so we wrote a piece about direct monetary cash transfers. This is still out there and publicly available. It was intended to be a piece that helped explain some of the mechanics of what needs to be done to procure more cash transfers. What are the problems that it's good for addressing? What are the problems it's not to be used for? There was a much longer list of the things to use it for. For example, if your goal is to improve test scores in school, no, don't do cash transfers; the evidence does not support that. If your goal is to improve school attendance, it has some impacts, but it's really much better in other areas. It was a guide for when to do it, when not to do it, and some of the decisions you have to make to do it. We were seeing traction from this. This position paper is intended as a document that doesn't have any "shoulds" or "shalls." A position paper doesn't make anybody do anything within USAID, but it opens doors. It makes it so that doing that is now more acceptable. There's now a guiding document, you're following that, you're breaking down some barriers, and that's the role of that kind of document bureaucratically. We were seeing that happen, and I expect us to be able to come back to that as well. We are seeing huge movement from other donors, other European donors, World Food Programs. The idea of getting cash into what they're doing is also succeeding. Some of the work we were doing has shifted over to working with them on those same problems.

SPENCER: With an organization like USAID, there's this question: why does it really exist? Some people will say, "Well, it just exists as a form of soft power. It's just the US trying to control other countries for the US's benefit." How did you think about that aspect of it?

DEAN: So I have a couple of thoughts, and it's obviously something that, for what it's worth, I've thought a lot more about this now than I did when I was there. When I was there, my mantra was, "Congress was deciding the why, in a sense, where they were deciding the sector, and embedded in that is something about the why." My role was to accept that and just say, "How do we help? How do we achieve that goal as best as we can with the resources we have so we can help as many people as we can?" And that's it. That allowed me to not actually have to think about those kinds of questions as much because I accepted the goal as it is. Since we've been working on thinking about how to restructure and think about plans for the future, this has obviously become more straightforward. One of the things Rubio says, I found very useful. I don't know if the way I interpret it is useful or if he even heard what I had to say. I did write an op-ed on this with a Republican congressman, Ted Yoho. His three questions for the why are: does it make America safer, more prosperous, or stronger? Embedded in that is soft power. The main thing to realize is, when you think about those three terms, safer is a little bit more straightforward. There are diseases that USAID was working on to help prevent the spread of those that can lead to diseases coming to America. Safer could be about crime in Latin America that affects immigration. Safer can be about reducing risks of terrorism by addressing poverty issues in countries where poverty can lead to increased violence, terrorism, civil war, and conflict. There are paths where you could argue that, and that's not necessarily soft power. I would call that more like strategic development. Soft power is more about when you're providing support in a country that then sees you as their friend, and you receive other benefits. It could be flyover rights, votes in the UN, things like that. More prosperous is straightforward in the sense that it could be about trade, access for investment, or buying wheat from farmers in Kansas. There are concrete things that make us more prosperous. What is stronger? The way I think about being stronger is you can't answer this question by naming something that makes you more prosperous or safer because there are three questions, not two. What makes us stronger is if you're living by the golden rule. The golden rule does not say, "Do unto others as you would have them do unto you," but "help others." You're strong when you can see past your own nose and support those around you, and that makes you a stronger person. It's a manifestation of being strong, but it also does make you stronger. Another way of saying this is that our US government should represent our people. If our people think of ourselves as generous and as people who want to live by the golden rule, what better way than to have our government do the same? That puts us into the space of saying, if that's an acceptable why for doing foreign aid, there are now no boundaries. Now it's just helping people, and that's full stop. Good. You still need some disciplining device to add to that to say, "Well, okay, fine. Where do we stop? How much?" But it does tell you if that's an acceptable definition of what makes us stronger as a nation, and that's the why of foreign aid, then that's a win, as far as I'm concerned. That's why I've always liked that way of thinking about it. There are quotes from Rubio that are very supportive of that from more than a year ago, not in the most recent year. There are definitely quotes from John F. Kennedy, when he started USAID, that are consistent with that logic, and Republicans and Democrats for the entirety of USAID have made statements along those lines. That's my preferred way of thinking about the why that has bipartisan support over the years in different ways that it's been said. The one main problem I'll point to is it doesn't give you a disciplining tool for saying how much, what countries, things like this. That does force you to think harder about what that means. I put that into the how, and then I get into the space of cost-effectiveness. What are the most effective ways, where can we pull levers, and what types of things can we do that are more effective?

SPENCER: That reminds me of the hilarious statistics about what Americans think in terms of how much foreign aid we actually give. Do you remember the numbers there?

DEAN: Yeah, it's something like they think 25% of our budget goes to foreign aid. They think it should be radically lower, like 10%. My response is, "Fine. Let's do it." The right answer is about 0.6%.

SPENCER: Yeah, it's wild. You can see why that could create a sense of people being against foreign aid because they think we're doing such an enormous amount.

DEAN: Yeah, so that's consistent with what we were saying in your question earlier about what makes us stronger. They thought it should be 10%, but actually, fine, let's ignore that part. The point is, they do think it should be a decent number. Most people, Democrats and Republicans, are actually supportive of the United States using its money to save lives overseas. And by saving lives, I want to be clearer. I mean today's lives and tomorrow's lives. Sure, some people might have different preferences about how that plays out and different emotional salience that today's lives versus tomorrow's lives bring, but that support for being a moral leader and for being generous as a nation is something that most people consider a point of pride in being able to describe themselves and their country that way. And that's great. So we should do that. It's not happening right now, but we should see that happen. I think there are a lot of people on both sides of the aisle that are supportive of that, and I am hopeful that we'll get back to a day where that is the way of thinking about foreign aid.

SPENCER: Well, that brings us to the elephant in the room, which is, if you go to the USAID website right now, you get some kind of message about it doesn't exist. What the hell happened?

DEAN: Yeah, it was definitely, if you asked me to make a set of predictions back in December, post-election, this was definitely the very extreme left. I'm not sure that total annihilation would have been a probability of this happening, but it was definitely not what we were hearing. It did not align with Project 2025. The first thing that happened after the election, I was not a political appointee, so I was prepared to stay on and do things I wanted to do just to help make things more effective. I knew they were going to come in and turn off some programs that I may or may not think are good ideas, but they don't like. There was a lot of stuff left.

SPENCER: You didn't expect the wood chipper?

DEAN: No, there was a lot of stuff left. My attitude was that I would work with you to help make those more effective: agricultural policy, food security policy, cash transfers, like we were talking about earlier. There are a lot of things that achieved the goals the Republicans were saying they shared and were there to do. I knew they were going to turn off LGBTQ support and gender programs, but there were a lot of things that remained. If you just read Project 2025, which is basically what everybody dusted off, I had read it six months earlier, but I definitely got that out on the Wednesday after the election and started reading it in detail. And that's what we were expecting.

SPENCER: It didn't say in it let children die.

DEAN: Oh, absolutely not. Some of the things they wanted were exactly what we were trying to do. Some of the things they wanted we actually had already done in the past year. There is one item in particular about having a more centralized process for reviewing the cost-effectiveness. I don't think they used the term cost-effectiveness in that bit, but it is part of what they were asking for in Project 2025: that things above a hundred million dollars went through a more centralized review to make sure that it was aligned with strategy and likely effectiveness. We did this; that was actually put in place. My office was heavily involved in that process. Some of the things were just done. Some of the things were exactly the kinds of things I thought needed to happen. That was our key expectation. Obviously, we didn't know, and even in December and early January, the rumors we were hearing about the people who were going to come in were people who were involved in Trump one, and some of them were people for whom I had a lot of respect and I was looking forward to working with. I thought we could make changes to help continue the effort that we had already started to make things more cost-effective. None of the people I was familiar with, that I interacted with, got invited to come in, though.

SPENCER: So do you know how it got to where it is now?

DEAN: My understanding is that it's best described as DOGE. I mean, this was the world's richest man taking it to the world's poorest people. And why? I don't know. I don't know how to say what goes into his heart. I can't speak to that. I don't know.

SPENCER: What was the justification given?

DEAN: So the justification given has actually been admitted to be wrong. Jeremy Lewin, who is now effectively running it, did a New York Times interview about a month ago in which he openly stated that the claims of fraud and waste were simply not found and didn't happen. I wish he said that louder. I wish he said that really clearly to people but regardless. He has very clearly stated that no, we never found those things. When they said fraud and waste, what they really were saying is things they don't like. Now, why can't you say it that way? That's politics. But it's really shameful, but that is how they said it. What they meant was, we don't like LGBTQ. We're going to call that fraud and waste. We don't like gender work. We're going to call that fraud and waste. We don't like things that are climate mitigation. We're going to call that fraud and waste.

SPENCER: But presumably, even that was a very small percentage of USAID activities.

DEAN: Those were tiny, yeah. Gender was not. Gender permeated lots of programs because it's more of a crisscrossing kind of concept, but the others, yeah, were fairly small in terms of the size of USAID. So look, I think there was an element that had nothing to do with USAID. That was more of a demonstration. It was what's a program that we can take to the wood chipper to use Elon Musk's language as almost a demonstration exercise, to say we can just annihilate something and pull it off. That was vulnerable, in a sense, because even though Congress has the power of the purse, they have to actually exercise it and back that up. It's not affecting checks to Americans in the sense of, if they tried taking a wood chipper to social security, that would have been a different scene. This was the kind of program that in an agency that Americans might be supportive of, but they're supportive of it on a philosophical level, on values, thinking that it's an appropriate and good thing, but not in a way that actually affects their checkbook or the services they receive. So it doesn't have that same level of defense. That's one thought, but maybe you should invite them on the podcast, see what they say. Why did he do this?

SPENCER: How did they even have the authority? Because, as you say, Congress writes the checks. How does that even work?

DEAN: So this is actually still playing out in the courts. Vought, when he got his nomination to head OMB, explicitly stated in his nomination hearings that he wanted to test the Impoundment Act. The Impoundment Act is that he didn't think that was a constitutional act, and he wanted to challenge it. Impoundment is when you ignore Congress and say, "You treat with Congress." Congress says, "Thou shalt take the money to Jordan, thou shalt spend $400 million and send it to Jordan." He wants to be able to interpret it as, "Thou shalt spend up to $400 million, but anything from zero to 400 is fine." That's a very different sentence. Early on, it sounded like that was going to go to the Supreme Court. I don't know the status of those lawsuits, but I think they have mostly gotten derailed on various standing issues. The other thing they did is literally just started canceling awards. If you don't have Congress standing up and exercising its muscle to stop it, they just stopped spending money. They stopped paying bills on things that were already done even back in 2024, and there were organizations that were hurting financially because they wrote money, and this had nothing to do with the election. When you dismantle everything in that way and put everything on hold, there were a lot of awards put on work stop orders. You can't just tell an organization to freeze and not do anything. "We're not going to pay you now for the next two months, but please hang out and be ready so that when we push a button, you can be there and come back." That's just not how it works. There are people who have to eat and have to do things, and that really causes them to lay off people. Then two months later, if you call and say, "Okay, we're ready to go now," it's like, "Whoa, we don't have our staff anymore." One of the things my team built was a website called PRO that went through a really amazing, granular job, took the things that we knew were saving lives that were very specific and focused, and went through as many awards as they could identify. They worked with the implementers to say, "Okay, you had a $50 million award, but we want to know exactly which part of that is delivering this medicine, and then we're going to raise money for you for that." Cash and Graduation were part of that, but a lot of it was more on the health side. I don't know what it tallied up to, but I think around a hundred million was raised to go to implementing parties who were in this moment of dire need. They were able to cut to the chase and say, "Look, of course, we'd like to have a program back, and we think it was doing good, but if we have to reduce it down to just the bare bones, what's the absolutely most cost-effective way of saving lives?" It was this piece here, and that got funded.

SPENCER: Most foundations?

DEAN: Those foundations were high-net-worth foundations. There was an event about two weeks ago in which it was announced that someone bought up the rest of them. Now, I think there's a new wave coming because it succeeded beyond our wildest expectations, but there's more need. The team is continuing, and just because it was bought out doesn't mean it's over. There's going to be another round of identifying further things.

SPENCER: Well, that's fantastic, but isn't USAID like $10 plus billion? So that's the tiniest little amount.

DEAN: It is and it isn't. It isn't to the people who got their lives saved.

SPENCER: Of course. Well, I guess that removes a few of the dead children from the conscience of the people that shut the program down.

DEAN: But that assumes something. I leave that for you.

SPENCER: Before we shift topics, just a couple logistical things. We're both going to have Office Hours in the room over there at the end of this. I just want to let you know that. Also, if you have questions, go on Swapcard and send in your questions. We'll be taking questions in a few minutes, and we'd love to hear your most difficult questions for Dean. You can do that now. One thing we were discussing today is this idea that goals and aspirations are really not enough. You have to think about strategy. In your organization, IPA has conducted hundreds of randomized controlled trials. I would love for you to talk a little bit about some of the meta patterns you've seen about what actually works in practice when you're going to implement something, versus things that may be a good idea, but kind of don't work.

DEAN: In that, I think there's a great way of putting it in the sense that we have some principles, some theories about development. You might know that this theory is true, but then it requires a certain implementation fidelity to get it right. There's always that nexus where we need evidence on both. We need evidence on the underlying theory to validate that this idea has the right legs and what the contextual factors are in which this theory is appropriate. But then things can be implemented well or poorly. We need more work on that implementation side. When you have things that have more implementation risk, it means that even if it's the right theory, it might not be the right answer. You have to know the context. You have to know who you're working with and what's the scale in which they can work. It might be that you want to reduce to something simpler to increase effectiveness. Even if there's a theoretically better idea out there, if it's not implementable, what does it matter? It's an interesting point to know there's a better program in theory, but it doesn't actually help. One of the other insights that I think has been really important, that we've seen run out, and this is actually one of the things we were going to focus on in USAID, is that, as you said, it was the largest bilateral aid agency in the world, but it was really small compared to the partner governments' budgets. If it was spending 50 million on education in a country over five years, that's nothing compared to what the Ministry of Education was spending. How do we use what we're doing as a large bilateral donor to improve what the government is doing? That kind of system strengthening work is absolutely key, and you can build the same exact logic of cost-effectiveness into that. What that looks like is, instead of setting up a bespoke process that is kind of separate, almost running as a separate program that is maybe complementing but not really integrated into the government program, work with the government to help that evidence review I was describing. Do that with the government. So they're asking these same questions. They're trying to optimize their agricultural extension programs, infrastructure for farmers, access to markets for farmers, improving schools, and health clinics. Work with them to identify the problems they're facing, what the evidence base says from elsewhere that can be used. That's the "use it," and in that process, you're probably going to identify some questions and some ideas, and then that becomes the "produce it." You can use the bilateral money to say, "Great, we'll pay for the pilot." Maybe "pilot" is actually the wrong term if it's a large-scale effort, but we'll pay for the innovation as the bilateral donor, but the government is actually integrated into what they're doing and becomes baked in to be part of their policy. That's a much faster road to impact at the country level because you're not just sharing results with them. You're not just having a meeting, inviting them to a conference, and showing them some figures. It's actually their program that you're helping them implement, and you're helping them evaluate and understand whether it's working. If it works, it's much more set up to be the on-ramp to policy. You want to do that in a way that you also share with other countries. You don't want to just be consultants to the government and leave it at that. You want to take a lens of saying, "Okay, what are the questions you're asking that others have that question too?" This innovation is going to be helpful for others to learn from. That's where the dorky academics come in because that's where they thrive, wanting to write the research up in a way that others in other countries can also learn from.

SPENCER: Is there any way to think about it that these local governments are spending a lot of money, they already have really clear objectives, and you're able to come in and say, "Hey, we can help you achieve those objectives better." You're not trying to change the objectives.

DEAN: That's exactly right. And so, go back to the graduation example. We've done a lot of work in the Philippines with the Philippine government, and they have a program that actually looks and smells a little bit like the Graduation Model. When we first met with them, sharing results from the graduation, their exact question was great. It was, "Huh, that's a little bit like this thing we're doing. What are the differences? Let's get into the weeds and figure out those differences." Some of that is not about setting up a new randomized trial. There might be some management data that's actually helpful that we can work through with them to help understand what that program is mechanically doing and what the Graduation Model is doing, and then help see, "Okay, if it's different, then maybe there should be an evaluation. Maybe we should try a pilot, implement the graduation, compare it to what you're doing, and understand what the optimal model is in your context." That kind of integration comes about by them posing questions and saying, "We have a fixed budget. We want to do the most good we can with what we're doing." In a lot of those cases, when we're working, the goal is usually to work with the highest level of civil service. It's not to say you don't want to engage with political aspects, but you don't want to be subject to the whims of the election cycle, for perhaps painfully obvious reasons these days in America. The point is that these things happen in developing countries too. Each one is a little bit different. There's not a set formula for how to do it, but the goal is usually to work with the highest level in the civil service because they're the ones who are most knowledgeable about the details of the programs. They know the questions they need to ask, and it gives it legs for long-run impact.

SPENCER: Before we wrap up, I asked you, if you had a critique for the EA movement, what would it be? You mentioned how you feel there can be an overemphasis on measurability. I definitely don't think we would make that critique of this sort of AI part of the EA movement, which is really focused on more theoretical arguments, but you can make that argument in maybe other areas of EA. I'm just curious to hear you talk a little bit about that, the role of measurement and when measurement can be taken too far.

DEAN: We had a phrase that we used called cost-effectiveness thinking. The position papers I referred to earlier, we actually put out two. One was on cost-effectiveness, and one was on cash transfers. The point we were making in the section about cost-effectiveness thinking, and this is work that is now carried on, and we're doing work on this with other bilateral agencies, is that a good example is what we were just talking about a moment ago. Would that work if you were going to work with a local government to help improve their processes? Some of them might be helping them install better software for what they're doing. You should still be able to do some back-of-the-envelope calculations to think about the likely cost-effectiveness, but it might actually have a number in there of likelihood of success, 20%, because maybe there are some regulatory changes or policy or legislative changes that have to be part of that to make that work, and you're going to take some guesswork as to the likely effectiveness. Usually, when you do those back-of-the-envelope calculations, if it's a good program, those numbers look way better than direct program services, but with a huge risk and a bit of uncertainty on whether it's going to work.

SPENCER: And so, 20% is sort of a principle, You have to make it up.

DEAN: You have to make it up. The most important thing, in a sense, is that you're consistent in how you do it. You want to impose the same kind of discipline on those types of choices you're making, so that when you're actually deciding what we mean by system strengthening, what we are actually going to do, and how we are going to choose across these approaches, you're being quantitative, but you're doing it with a strong qualitative sense and recognition and humility. You're trying to be as consistent as you can about it so that you can make better choices. You're not dismissing something just because you can't measure it, and that's the real key. You're not dismissing something because it's not as concrete or because it's not done with a randomized trial. You're just doing the best you can to estimate it and recognizing the uncertainty of it, but not dismissing it out of hand because it's difficult to measure. The same is true for things that are difficult to measure as an outcome. If your goal is changing public opinion or improving women's power within the community or in the household, these are more difficult concepts to measure than consumption and income, and we shouldn't avoid working towards them just because they're difficult to measure. That may be a less good example, because they might be harder to measure, but you could still set up randomized trials when measuring impact. I think the more interesting case is on things that are outside the space of what you can use randomized trials for. They should not be dismissed out of hand because we don't have a randomized trial for them. You just need to think hard about what the path of impact is and what the ballpark estimate that we think is reasonable is. You might want a higher bar because of that uncertainty, but a lot of those calculations look really high. We can look at our office as an example. Our office is a good example. We couldn't randomize what we were doing, but we estimated a 40 to one return on investment from our office, and that's just the 1.7 billion I referred to, and the change in the impact that was projected from those programs divided by the cost of our office.

SPENCER: Another critique you mentioned about effective altruists is that you feel they can underestimate the value of meeting people where they are. You actually did work that ended up being absorbed into Charity Navigator, so we could just talk about that for a moment.

DEAN: This was referring to Impact Matters, which I started with Elijah Goldberg about 10 years ago.

SPENCER: Your resume is too long; it doesn't even make it into the list of things I mentioned.

DEAN: And so we always saw Impact Matters as both a compliment and a complement to GiveWell. Compliment with an I in the sense that we admire what they are doing and we respect their analysis. I'm sure we might quibble on little things here and there, but I have a lot of respect for the level of rigor and detailed analysis they do. Complement with an E in the sense that we were not interested in ranking the top six. What we were interested in doing is meeting the donor where they are and recognizing that while there are a lot of donors who are content to go to the GiveWell website, identify charities, and then support them because they read enough about the process that GiveWell went through and they liked it, there are a lot of other donors that say, "No, actually, I care about poverty in Cleveland," and "What are you going to do?" Or "I care about education," which doesn't make it into the GiveWell list in Sub-Saharan Africa. I don't know if this says something about me, but my experience in an audience with donors, whether large or small, is that when they say, "My passion is education" or "My passion is Cleveland," I have 0% success in convincing them to change that. That cause area is, in a sense, chosen because of their heart, because of some emotional attachment to the cause area or the area geographically. Maybe it's a family thing, whatever it is, why would we want to do anything to discourage that? So that's why our mantra was, "Choose your cause with your heart and your charity with your mind." Our mantra was very much about accepting the donor where they are and then helping them find the effective charities to do what they want to do, and not risk coming across as judging someone and saying, "Oh, your cause area is not that important," which just makes them not feel good. It doesn't convince them to join your cause. That's always been, I think, an important way of thinking about philanthropy. Of course, if someone asks me what cause areas I think are important, I want to look at the math and say, "Okay, well, here's the money, here's the good you can do," and it would lead to certain conclusions that are more consistent. But most people don't do that. Most people have a passion. They start with that passion, and I think that's important to embrace and respect.

SPENCER: It seems like a really unique aspect of the effective altruism movement is that there are a lot of people in it who try not to choose their cause area with their heart, but they try to choose it with their mind. It is a very unusual feature.

DEAN: I have a lot of respect for that openness that comes with that. I think that's great. When someone does that, those are fascinating conversations and fascinating struggles. Some of the conversations can be very heavy, but they're important and thoughtful. I respect that as well, and I'm not critical of that by any means — that cause agnosticism kind of mantra. I recognize that when I give talks in broad crowds. If I give a talk at an EA crowd and say, "Raise your hand if you've heard of GiveWell," everybody raises their hand. If I say, "Raise your hand if you've heard of Charity Navigator," no offense, not as many raise their hand. If I go into a non-EA crowd, I get the exact opposite. That's where we are as Americans. So meet people where they are, help them make better choices in terms of choosing effective charities, and of course, let's do as much information as we can to try to inspire people to give towards the causes that are most impactful. But let's not do it in a judgy way that makes someone who is passionate about something else feel unappreciated for what they're doing.

SPENCER: Dean, thank you so much for joining us today. I really appreciate it.

DEAN: Thank you very much. Thanks for having me.

SPENCER: And now we've got questions from the audience, which I have not seen yet. Thank you, Jacob. So this is a question from Jack: the vandalism of USAID is a moral travesty, but does it offer the opportunity to build something better to replace it in the future? What would your priorities be for USAID 2.0?

DEAN: Fair question, and I am for this word. This is what has absorbed a fair amount of my time that was USAID, and I am now spending thinking about that and also working with other bilaterals and donors. I think there are a couple of principles that we're working on, and I'm part of a bipartisan team that is doing this, but we're not thinking about the next three years. We're thinking about past that. One is identifying a core set of things that are of common agreement that Rubio, a year ago, would have said he supported, as well as many other Republicans and Democrats. Identify that core. It's a very large proportion of what USAID was doing. Identify it. Let's protect it. A second is obviously, perhaps not, but based on our conversation, perhaps obvious: let's put evidence at the forefront here, in terms of making sure that it's baked in from the very beginning, a process of looking at the evidence of what's most effective for addressing the problem we're trying to tackle. Use it, identify the settings where you don't have enough evidence to guide you, and produce it, and bake that into the actual process of thinking about how things should get done. I also think this work with the government is really important and a different change, and that's something that we're seeing in conversations, not just as part of the conversations I'm immediately in and helping to organize, but the overall global challenge that we're seeing, not just from USAID, but from other players. A lot of the transformation we're seeing has a large emphasis on how do we strengthen government, how do we strengthen the delivery of government services, the setting of government policies, recognizing that that is absolutely necessary for the long-run development of countries? That kind of emphasis on putting those governments more at the forefront of the process is absolutely critical. I think an aid system that takes that priority is absolutely critical. That's part of the thinking about how to differently structure a process.

SPENCER: This is a question from Nadav. I'm going to modify it slightly. Do you think that USAID being shut down really was a partisan thing, fundamentally? Is there anything that could have been done to help prevent it? Or do you think it was just one of these out-of-the-blue events that nothing could have stopped it?

DEAN: Yes, it definitely was political. It was definitely MAGA DOGE coming in. There was no pretense of it. As I said, even in December, no one was talking about it. Maybe somebody was, but they were not talking.

SPENCER: But is this in your radar? Is there anything that could have been done? How do we depoliticize things?

DEAN: I don't think so. People often made that claim, and here's the thing: they said, "Oh, maybe we just didn't make the case enough. We needed to make the case for what USAID was doing and why it was important." But here's the thing: if you looked at the talking points that they gave, which were written straight from the White House, most of them were not true. So what does it matter if the people that are taking it down are just lying about what's going on? It doesn't really matter what the truth is. They're just going to say whatever they want. One of the accusations was that $50 million worth of condoms were distributed in Gaza to Hamas. There are a few problems with that. One is, there is a region called Gaza — in Mozambique, not the Middle East — that did receive some condoms and it was awarded $50 million of an award, but it was to provide support to the health clinics for all sorts of sexual health as part of PEPFAR to reduce the spread of HIV. One of those things is a box of condoms that goes in the health clinic so you can get some free condoms. If it was $50 million for the condoms, take a guess what that would have been in terms of the number of condoms: 1.5 billion, or three cents each. Imagine what Mozambique would look like with 1.5 billion condoms. It wasn't even all of Mozambique. It was a region of Mozambique. There was no — it didn't take long to realize this from anybody reasonable, and this was pointed out, and they kept repeating the line. So I don't think the rhetoric was the problem. If you're just going to name things that are not true to take it down, and they're going to survive, and you're going to repeat them, what does it matter? They made a decision to do that, and then they put in their social media machinery to make it happen.

SPENCER: This is a question from Jacqueline. She asks about when you have to make these big decisions, but there's just not evidence available. How do you approach that?

DEAN: So in a sense, that's a hard question to answer in the true abstract, but my answer goes to what I was referring to earlier as cost-effectiveness thinking. You're still going to spend a certain amount of money, and you should be able to, in some sense, write down your theory as to what's going to happen: the first thing, second thing, third thing, write down a chain of events, and then what's that final thing, and why is that better for the world? Usually, each of those steps, even if you don't have strong evidence that goes from the full set of five steps, you probably have some way of thinking about the magnitude of each of those five steps. Each one of those steps might have some good back-of-the-envelope estimates.

SPENCER: It's almost like a Fermi estimate, kind of where you're breaking it down into these component estimates.

DEAN: Yes. And so that should give you some sense of whether you have a reality test: does this seem like a good idea to do or not? Does it have potential to pay off?

SPENCER: Okay, this one's from Mashoud: what do you think are innovative or emerging strategies that advocacy groups should deploy to improve or influence aid policies in high-income countries?

DEAN: In high income, my mind goes exactly to the kinds of things that we were trying to do, which is how to help the aid agency. I would put it into two buckets. One is the "use it or produce it" slogan: how do we help the aid agency use evidence of effectiveness or produce it when it's lacking? The second part is about how to work better to strengthen local government, which is, in a sense, taking that "use it or produce it" mantra and helping the governments in our partner countries integrate that thinking into how they design their policies. We can fund some of that work. We can help fund the innovation. We can fund those tests. That's the short answer, and we are seeing demand for that exact thing from other agencies. The best email by far that I've gotten in the past year was one that said, "Hey, you came after I resigned." It said, "I heard you might have some spare time," and it was from a European aid agency that had already been talking to us before. They said, "We want to mimic some of the things you're trying to do with USAID. We're facing some similar issues. Please work with us and help us implement some of the changes you were trying there. And try to help us do the same." That was music to our ears, and that's part of what we've been working on.

SPENCER: A final question. This is an important question from Michael Thatcher. Please say more about the future of U.S. foreign aid and the best ways to fill the gap.

DEAN: I don't have much to say about the next three years. There's going to be some new spending. They haven't yet spent new money. We don't know how it's going to play out. We have lots of reasons to think some of it is going to go to trade private enterprise, not even in developing countries, in places like Greenland. There's been a lot said about this. Past that, we need to see who comes into power next and whether they're supportive of some return to working in low-income countries and taking it seriously, not just doing it for public image. I think that's the key. Some of the things that have remained, some of the mantras and priorities, are clearly more about a public image issue or very deeply transactional, like, "We will give you aid if you give us these mining rights." This very transactional way of thinking about foreign aid is dangerous. You get exactly what you negotiate or bully for and nothing more. It's not a relationship. Imagine having a marriage in which every single thing you did for your spouse was contracted and negotiated. That's not a healthy marriage. But that's the way they're thinking about development, and that's a serious long-run problem about our role in our relationships with other countries. Hopefully, we can see some reversals to that. Here's one reality to perhaps close with: in my limited two and a half years there, I didn't meet a single person at USAID who said, "We're the most amazing, efficient organization on this planet. There's nothing to change here. We're perfect." Everybody was passionate about helping other people, had some area of expertise, and faced tons of bottlenecks to make changes. Some felt empowered to make changes, and some did less. The point is, everybody was able to point to issues, and that's true of anyone in the workplace. I don't think anyone in this room can think of their workplace and say, "That's a bastion of efficiency." We are all aware of those kinds of issues. USAID was no different. There is a blank slate. There is a moment here when serious people with experience in development, who understand more about what works and what doesn't, have a chance to seize that opportunity and try to do something good that has a long-run impact. That's why I'm working with this bipartisan team, hoping to see some of that happen and play a long game, investing in the three-year and seven-year effort. We have to do a lot more evidence work to do that in the short run. That's one of the big gaps. The academic community has been great at producing something like 5,000 randomized trials and horrible at producing meta-analyses and synthesis and mapping of what the lessons are to actual procurement needs. We're working really hard on that aspect so that in three or seven years, when we put things in place, we're not starting that process. That's part of what we're trying to do, and that's at the core of a foreign aid system.

SPENCER: Dean, thanks again for joining us. I really appreciate it.

DEAN: Thanks a lot. Thanks for having me.