SPENCER: Kanika, welcome.

KANIKA: Thanks so much, Spencer. I'm so excited to be here.

SPENCER: Do you think that we're missing out on a lot of really effective ways to help the world because people don't want to use the simple solutions?

KANIKA: I think that's exactly right. Spencer, I've spent nearly a decade at my current organization, Evidence Action, and I have been so surprised at how some of the simplest and most effective solutions just go underfunded.

SPENCER: What's the psychology of that? Why would people underfund those?

KANIKA: Those are really great questions. Spencer, I think it's a combination of things. Partly, a lot of these solutions are actually pretty invisible. One of the things we work a lot on is safe water. Safe water seems so intuitive, but it's actually invisible. You make water safe by chlorinating it, and people don't necessarily realize whether their water is chlorinated or not. I don't know, Spencer, when you go to the tap and turn on the water, are you ever thinking about whether it is chlorinated or not?

SPENCER: Nope. I'm just hoping that someone's done their job to keep it safe.

KANIKA: Exactly, exactly. So that's exactly it. Safe water actually saves one in five child lives. There's a meta-analysis done by a Nobel Laureate quite recently, and it showed that safe water is one of the best things you could be doing to save a child's life. In our programs, that costs about $1.50 per person per year, but it's invisible, and so people actually don't even know that they should be asking for it in rural Malawi or rural Kenya, or wherever we work. That's one part; people aren't asking for it. Governments don't necessarily prioritize it. There are other reasons as well, but that would make sense, Spencer?

SPENCER: Absolutely. But I guess this makes me wonder how much of these interventions are driven by what people are asking for versus donors or governments coming top down and saying, "This is what people need."

KANIKA: That is such a great point. That's another big driver of why programs actually get neglected; they can kind of fall between the cracks of global development strategies. One of the ones I think of a lot here is maternal syphilis. That's basically women in pregnancy who have syphilis and don't know it.

SPENCER: When you say it's fallen through the cracks, could you elaborate on that?

KANIKA: Every year, maternal syphilis — because it's untreated — causes 200,000 stillbirths and newborn deaths, and then over 100,000 cases of lifelong disability. That's things like blindness, deafness, neurological damage that's lifelong. Just to put that in context, that's a really big number. It's more stillbirths and neonatal deaths than all deaths from HIV/AIDS in children from zero to 18. In general, we're seeing about $300 million historically being spent a year on HIV/AIDS, but less than $50 million on this disease. It really seems to be driven by what's on strategy for the donors.

SPENCER: What does that mean, on strategy? Why would one of those two fall into some strategy, but not the other?

KANIKA: It's such a great question. Partly, it's that things like HIV/AIDS are funded by the Global Fund to Fight AIDS, Tuberculosis and Malaria and by PEPFAR. There's a lot of metrics around treating AIDS in kids, and there has historically been less attention. But Spencer, I want to highlight why I find this so heartbreaking. I want you to take a guess. I talked to you a little bit about the numbers of kids who are affected by this, and I want you to take a guess as to what it will take to prevent so many kids from having these lifelong difficulties, being born, having stillbirths, and newborn deaths, which traumatizes the family as well. What do you think it would take to prevent this, and what would you be willing to pay to prevent it for a kid of yours?

SPENCER: I'm guessing you're gonna say some incredibly simple intervention that is widely available if we just give it to people. Obviously for your own child, you'd be willing to pay a huge amount of your own savings to protect them.

KANIKA: Yeah, so it turns out that, and you're right, of course. I guess that was a leading question. Sorry, Spencer, it's a dual test. Right now, most women who are going to antenatal clinics, pregnant women go to antenatal clinics in rural Africa, and they are typically getting tested at pretty high rates for HIV/AIDS. There's a rapid test, it's a lot like a COVID rapid test. You get results very quickly, very easy to read. It turns out that you can actually use a dual test, which tests for syphilis and HIV/AIDS. You just need to switch to that, and it costs 35 cents more per test. When a woman is found to be positive, they need a single shot of penicillin. That shot costs less than 50 cents, and that prevents 80% of these really terrible outcomes that I mentioned. You get the results in minutes, and it's same-day treatment. It's about as turnkey a solution as you get in global health.

SPENCER: That's pretty amazing. So basically, they're already getting the HIV test. You just add this extra test in, if they test positive, then you give them penicillin, and 80% of them are fine. It seems like an incredible opportunity.

KANIKA: It is an incredible opportunity. It's incredibly cost-effective, and it's actually really easy to make the switch when you have support from the governments. In Liberia, we started working on this a little over three years ago, and when we started, 7% of women were getting screened for syphilis. Two and a half years later, we worked really closely with the government on training and getting them the supplies, and basically just the basics, and the screening had gone up nearly tenfold to 68% nationwide, which meant that we were able to save 1,300 infant lives in Liberia and prevent over 1,000 disabilities. It's really these kinds of opportunities where when we think about the resource-constrained environment we have right now in international development, that I think we really want to be focusing on and getting to massive scale. They're easy, they're cheap, and they save a lot of kids' lives.

SPENCER: Now, how do we know that this is effective? The argument you made seems very convincing. We know about how often syphilis occurs. We know how effective penicillin is, etc. But are there studies that actually test this and measure outcomes?

KANIKA: So, there's a lot of literature and a lot of medical literature about penicillin preventing these outcomes, and so that's really well established. My organization, if there's sound medical literature or health literature, we're not going to redo it, but what we do is go to the health clinic and ensure that these women are actually getting the tests. We examine the records. We go really deep to ensure that they have, in fact, been tested and also are getting the treatments. And so for us, there really is a big focus on gaining confidence that that last mile delivery is actually happening. It's a great question, and it's actually quite hard to do, I have to say.

SPENCER: So maybe one way to think about this is, in order for someone's life to be saved or for them to be delivered, the help that you're trying to give them, a whole series of things has to happen. They have to be going to the clinic, they have to be getting the test, they have to have the add-on test. The add-on test has to be accurate, then the treatment has to be administered if they test positive, etc. So it's a whole series of things, and some of those you can get straight out of the medical literature. You can say, "Well, if someone has this disease and you give them this treatment, what's the percentage chance that they recover? What's the prognosis if they don't get the treatment," and so on. But then it sounds like you're able to rely on the sort of basic literature for some of those answers, but others, you have to actually go test yourself. Are people actually getting the penicillin? Is the proper test actually being administered and so on?

KANIKA: Exactly right. That's very well said, Spencer. And because practically, we think about this a lot with things like safe water. Fortunately, even with safe water, deaths are quite rare, and so we're not going to redo studies that require a massive footprint to make sure that in our footprint, when we deliver chlorine, it's having the same effect as well-established literature. What we really want to be measuring is how many people actually have chlorinated water at levels that can help them. So you said it really well; we're thinking about how to model where we can and where to use the literature, where that's really sound.

SPENCER: It seems that a significant part of this is actually behavioral. You have to actually get people to come into the clinic. You also have to get the nurse or whoever's at the clinic to behave differently than they did before. There might also be behaviors around tracking; you have to get the data back and get it reported. Do you consider that a key aspect of your work is behavior change?

KANIKA: So Spencer, we really try to think about the low-hanging fruits, and in some senses, the more steps and the more places where there can be drop-off, the more likely you are to have it just be lower efficacy. When I looked at maternal syphilis — we first started looking at this five, six years ago — one of the things that really struck me was how little behavior change was needed. You had women already coming to antenatal clinics; we were only looking at the population of women who were already coming to clinics. These women were already getting tested for HIV, and the healthcare workers were comfortable with the test. Switching to a dual test really just required reading a few more lines. There was some training, but it was relatively easy, and sort of the scope of things we can be doing. I've done a lot of work with pediatric HIV/AIDS, actually, before this. There, you had to get kids and parents to adjust to taking, sometimes not great-tasting medicines daily. What was really appealing about this was you had few drop-offs, and it required very little behavior change. I think that was a big part of why it was so successful and why we could get a tenfold increase in two and a half years. Part of what we're doing at Evidence Action, when we're deciding what interventions to do, is actually really looking at this sort of tractability: how hard of an implementation lift will it be? By spending a lot of time selecting that, it actually allows us to find things that can have a pretty massive impact with relatively low cost and lift. I can talk a little bit about deworming, but let me pause there and see if that resonates, if that makes sense.

SPENCER: It does resonate. Looking at the literature on behavior change, it's really remarkable how often behavior change efforts, even if they succeed, don't move behavior that much. Success might be going from 5% to 10% or something like that, and that's a huge win. But, "Oh, that's only 10% of people doing the thing." So it makes sense to me that you actually view it as kind of a bad thing if you have to do big behavior changes in order to make the interventions succeed.

KANIKA: Yeah. Because ultimately, the way we think about it, there are limited dollars in the world. African governments, Asian governments, donors have limited dollars. And what we want to be doing is saying, "What is the best way to spend every one of those dollars to save lives, to improve lives?" If there's a lot of what we call drop-off in an intervention. If delivering safe water one way reaches 40% of people because it requires behavior change, and another version of delivering that water, which we've actually been working on right now in land chlorination, which hits in pipes, requires very little behavior change and reaches 9,800% of people, the latter one is going to be more effective and a better use of resources. So, yeah, that's right. We think a lot about effectively how much of the available market can we reach pretty easily.

SPENCER: I believe it was Evidence Action that was implementing a chlorine program in water. And I read a report about the challenges of changing people's behavior. First of all, it was like, well, people didn't really understand what this chlorine thing is they're putting in their water, like, maybe it changes the taste of the water. And so there had to be educational campaigns. And then it turns out, if the chlorine dispensers get empty, then people fall off the wagon and stop using it. Even if you get refilled, you have to make sure there's really good logistics so that the chlorine is always there. And still, people weren't using it enough. So you created an ambassador program of people in the community going around, educating people and so on. Am I right that that was you?

KANIKA: That's indeed us. That's exactly right, yes. And that was one of the programs where we do behavior change. And we do behavior change because our intervention, this dispenser intervention, which you described well, is at the community level. And so we can make a lot of investments because that sort of investment that we're making is amortized over a village with hundreds of people. But even then, it really requires this very tight operational system because, as you said, if fluorine is out for a few days, people are less likely to use it. If it's down for a few days, people are less likely to use it. So we've created systems so that we can refill, we can repair within 24 hours. It really requires real operational excellence. It's also harder to scale. And so a big other lever that we use, in addition to simplicity, is using government systems. And I think the Safe Water works are a really interesting example of that.

SPENCER: So how does the government come into play?

KANIKA: Recently, we've started working with the Indian government. Our dispensers program, which you described, operates in Africa. We've been able to reach 9 million people, but it's not done in partnership with the government, we operate it. In Africa, we're reaching 9 million people, and we're doing that through direct work with dispensary-shaped water. It's saved a lot of lives. We're really proud of that work, but it is limited in its scale because we're delivering it directly. In general, we think governments are an amazing platform to scale, and there really is no better example than India. India has a mission called the Jal Jeevan Mission, which is intended to reach all rural households with household water. It's a $44 billion national program, and they've installed water in 150 million households, and they're moving fast. They've asked us to work with them to make sure that water is safe to drink. We're combining two things that we just talked about, one of which is more automated technologies. We're putting effectively what we call inline chlorinators, chlorine devices that sit within pipes, into the Indian system. Then, we are leveraging the scale, political capital, and funding of the Indian government to take that to a really large scale. That's been a really exciting approach where we're combining multiple things. One is an intervention that we know works, which is chlorination and saving lives, the scale of the government system, and an approach that doesn't rely heavily on behavior change to take the scale. We're still in the early days, Spencer, but hopefully I can come on in a few years and tell you that we've reached tens or hundreds of millions of people with this approach.

SPENCER: It seems that often people want to work outside of governmental systems. I imagine that might be because governments move slowly. If you don't have the right contacts, how do you even work with the government in the first place? The government might not have the same priorities as you and so on, but it sounds like you've had a lot of success interacting with governments. How do you think about that decision of whether to work with a government or whether to build a parallel system?

KANIKA: To be honest, Spencer, I think the narrative that low-income country governments can't scale or are too corrupt to scale isn't just wrong. It's really holding back meaningful progress in this era. When I look at where we have achieved the biggest scale, it's because we are willing to roll up our sleeves. How do we make the decision? Our first stop is governments, and because that's our first stop, we've been able to reach half a billion people globally. There's really just no substitute for it, in my experience.

SPENCER: That is because governments are willing to put in much larger sums of money, or because they have infrastructure already in place? What's the fundamental reason why the government enables you to reach more people?

KANIKA: It's because they are delivering these services. They have the clinics, they have the infrastructure, they have the supply chains, they have the maintenance systems, and so it really provides the platform to get to scale. Again, I'm really proud of what we've done with dispensers, and I think it was and is the appropriate thing in places that aren't being reached by the government. But if you can tap into the scale and the infrastructure of government, that's where you're going to really see impact.

SPENCER: Does that mean when you're designing programs, you're thinking about how it can attach to what the government's already doing, like the example you gave with adding on to an HIV test, this other syphilis test?

KANIKA: Yeah, we're thinking about a couple of things. We are thinking about where there's already political will and a program. We're also thinking about what are the best platforms for scale. Another program that's had a lot of success for us is deworming. Deworming is another one of these no-brainers. Kids have parasitic worms. They feel sick, they often don't attend school as frequently, and their nutritional status suffers. Research shows that 20 years later, kids who are dewormed earn 13% more than kids who weren't dewormed, and it costs 50 cents per child. It's just this tremendous boon to kids if you can get them dewormed. But when we started, it had just been immensely neglected. There weren't many kids, this was over a decade ago, who were getting dewormed. So we asked ourselves, where can we reach the hundreds of millions of kids who need deworming? We actually said, "Well, the kids are where the kids are at, which is in schools." We worked with governments to use public schools initially to start delivering deworming treatment. Part of what we're looking at is not just what programs are there, where there's political will, but also what is the simplest platform to get the kids where they're at?

SPENCER: I see a theme in this conversation, which is trying to find the simplest way to achieve the goal, whether it's adding it under an existing service, or meeting the government where they are with something they're already offering, or trying to avoid changing behavior if you don't need to.

KANIKA: Exactly. We sometimes say that the best breakthroughs aren't what we invent, but how we reach people. We think deeply about what are the simplest, most straightforward, fail-safe ways to reach people, because that's how you get the scale. That's how you get the cost-effectiveness. That's how you get the impact. In deworming, I would argue it was that simple breakthrough of starting with schools that led us to have this huge impact. At the end of the day, we've delivered over 2 billion deworming treatments, and we recently did the math on that, combining the cost of our programs and the research I referenced before on income gains. We've helped increase productivity by $23 billion, Spencer, and that's just because of the sheer scale we were able to reach because of that school-based platform, combined with the evidence that I mentioned.

[promo]

SPENCER: My understanding is that GiveWell has recommended deworming, and you actually fostered that program of deworming recommended by GiveWell? Is that right?

KANIKA: Yes, yeah.

SPENCER: And obviously, GiveWell has a really high standard of evidence. One thing that surprised me is that a lot of the effects seem to be more general effects over a longer period of time, rather than the kind of immediate, acute effects you would expect. If I imagine someone who has parasitic worms, and then they're given a treatment for it, and the worms die, I would expect to see those benefits right away, in days or a couple of weeks. In fact, as far as I recall, at least at the time, a lot of the evidence on the effectiveness of deworming was these long-term, general things, like over years, people had higher incomes, or over a number of years, people had a little bit better educational outcomes. I'm just wondering, why is it that a lot of the effects seem to be general and over a long time, rather than really acute?

KANIKA: Yeah, we do find really big benefits over the longer term. We see that kids who are dewormed 20 years later are earning 13% more. For every dollar you're putting in deworming, productivity is increased by $169 over time. That's based on the evidence. Those are the longer-term impacts. In the nearer term, my hypothesis is that it's likely an effective interaction among different factors, rather than one single one in the short term. We see improvements in nutrition. There are studies showing a 25% reduction in school absenteeism, improvements in school performance, and so on. My hypothesis is it's really a composite of these sort of weaker, if you will, impacts. Kids are healthier, they're going to school more, they're learning more, they're better nourished. That ends up causing those longer-term benefits. I think it's just a great example of how it can be hard in the short term to really see impacts that are cumulative over time.

SPENCER: Parasite worm infections were going down a lot, but as far as I remember from the evidence, you weren't seeing this massive immediate impact. As soon as people no longer had the parasite worms, it was more like the longer-term, more general outcomes, like income, where you were seeing it.

KANIKA: It still remains a little unclear the mechanisms by which deworming increases adult outcomes. But there's some literature that looks like it's a combination of effects between improvements in nutrition, reductions in absenteeism, and improvements in school performance. These things are subtle. Sometimes they're subtle just because it's hard to measure these things and sort of separate out what's an interaction of a kid just going to school more frequently for some other reason versus the deworming. Some of it's just the limitations of what's easy to study, given correlation and causation, and sometimes it's just subtle effects. I think for me, we have seen some of those effects, and it just becomes a common-sense case. For any of us who have kids, if your kid is nauseous, they're unwell, they have worms in them, which not many of us, fortunately, experience, it's hard to get them to school. For me, the intuitive case combined with the evidence is quite compelling.

SPENCER: I don't think I've ever had parasitic worms, at least not to my knowledge. What is it actually like to have them? Is it the kind of thing where you just feel a little bit worse, you may feel tired, or is it more of an acute effect where you're really dealing with intense negative symptoms?

KANIKA: I have not had worms either fortunately, despite many years as a kid going to India and getting other bacterial illnesses. My understanding is that kids just feel kind of nauseous. They feel unwell. For those of us who have school-aged kids, when your kids feel like that, it's really hard to get them to school, and it's harder to learn. Yes, that's my understanding.

SPENCER: How do you think about combining, let's say, randomized control trials with these arguments? They're more based on first principles, like, "Okay, we know that parasite worms make you feel ill. If you remove them, it's probably good." Do you think of it as you really want both, to be able to trace out the sort of intuitive case and then also back it up with randomized control trials or do you think about it differently?

KANIKA: That is the question, Spencer. I think it's a combination: a randomized control trial combined with evidence that you see reduced absenteeism, that you see improved nutrition, some of that passing just the sniff test of, "Okay, yes, that makes sense. That seems feasible." And then we also look at the strength of the evidence base. Something that has five different randomized control trials across five different geographies, we feel more confident about than one randomized control trial. So it's really, I would say that I'm not a purist in these things, but we really do look at the composite of a number of these factors together. I don't know. How do you think about it, Spencer? Does that make sense to you? There's no clean answer. I'd be curious, what's your take? What was your take reading the deworming literature?

SPENCER: Well, I try to be Bayesian about these things, where you think about each piece of evidence shifting your probabilities. You come into it with some sense of how likely something is to work. And then you get some evidence, let's say it's medical evidence about the acute effects of deworming, and you're like, "Okay, that changes my mind a little bit," and then you get some other evidence, let's say a randomized control trial in one country, and that updates your probabilities a bit. And then you get another randomized control trial. So really trying to work with all the evidence you have. A pet peeve of mine is when people say, "Oh, we have no evidence of this thing." And what they really mean is that you don't have a certain study of a certain type proving the thing. But it's like, "Well, that's not the only form of evidence. We have to incorporate a wide range of evidence to make sense of the world. It would be wonderful if we had 20 randomized control trials on every topic," but that's just not the way the world works. A lot of times we have to synthesize different types of evidence, and even the most rigorous randomized control trials almost always have some limitations. "Okay, well, you've only shown it worked in this country. Well, what about this other country? Or, okay, well, this was administered in this particular way, but that's not feasible here. We have to generalize it, right?" So I don't think you can get away from trying to combine multiple types of evidence and doing it ultimately somewhat subjectively, but hopefully guided by principles of how to evaluate evidence.

KANIKA: I think that's exactly right. Yeah. I think that's really well said. We think randomized control trials are extremely valuable. But the other evidence that we see, and again, my husband once said, "It's funny how often the studies end up converging in common sense." And I do think there is some of that, that as we find out more about the world, oftentimes, it's perhaps not that surprising that we didn't have as much data that not having a belly full of worms originally, we didn't have as much evidence about what a belly full of worms did to a kid in terms of their nutrition. It's not all that surprising that, as more evidence emerges, we find out that it's not great for their growth. And so, I agree with sort of combining a lot of the evidence and thinking about where it lands us.

SPENCER: And I think that it makes sense that studies often converge in common sense. However, I think that the converse is often not true. In other words, there are a huge number of common sense things that actually don't work. I've seen this again and again. I don't work in exactly the same area as you, but we do behavior change stuff, and we read a lot of papers on behavior change, and it's amazing how often papers in behavior change do incredibly common sense stuff that totally bombs and doesn't change behavior.

KANIKA: Yeah, I think that is because behavior is really complex, and that's often why, at Evidence Action, I talked a little bit about these interventions that are pretty straightforward. We have seen that behavior changes over time. It can change across geographies. We don't always understand it really well, and so in general, a lot of our solutions, at least in recent years, haven't tended to focus as heavily on behavior change. I think an interesting example of one that did was No Lean Season. Are you familiar with our No Lean Season program in Bangladesh?

SPENCER: No, I'm not.

KANIKA: So this was something that we had done. I think it was six or seven years ago, and it was based on work in Bangladesh, which showed that small $20 grants, when given to agricultural laborers in Bangladesh during what was called the lean season, where there wasn't a lot of harvesting going on, meant that those laborers went to urban centers. They found jobs, they sent money back home, and it boosted caloric intake by between 550 to 700 calories per person per day, and household consumption went up by 30 to 35%. So folks were really excited about this evidence. We made a bold bet. We said we're going to scale this up. This project was profiled in The Economist. It was made a GiveWell top charity, and so, very exciting. We then said we really believe deeply that programs can change as they scale. This gets to the complexity of behavior change that you just talked about. So we scaled it to over 40,000 laborers. This was in 2017, and we tested it again, and it showed pretty limited impact. I think this was a blow to everyone who had been following it, including the leading economists who were working on it, our funders, and for us, we said, "Well, this doesn't seem like it's potentially not the great use of resources we thought it was." So we paused fundraising. We started digging in where there are implementation challenges. We invested in refining the program. We did another rigorous trial in 2018, and we told all of our donors about it, which kind of surprised them. Then in 2018, we looked at the results, and they were disappointing again, and there were other operational complications, and we ultimately shut down the program. It was really sobering for a lot of our funders and for the community, but I think people also just really appreciated that we had continued testing at scale, that we were transparent, and then when the program didn't have the results that we would have expected, to your earlier point, Spencer, it was surprising to a lot of people. We shut it down. To me, I think that's just such a great example of how behavior is complex and quite hard to model.

SPENCER: I think that's really fantastic that you shut the program down. Honestly, I think there are very few organizations that would do that. Sadly, I think the vast majority of organizations wouldn't even know whether it worked or not, so they wouldn't even have the data to show them it didn't work. Even if they did start to collect the data that it didn't work, I think a lot of organizations would find some way to rationalize or continue on anyway. I think that's hugely important, and I suspect that part of what enables you to do that is that you have a bunch of success under your belt. You don't need every new thing to work, right? You can say, "We're going to try things, we're going to take risks, and some of them aren't going to work out."

KANIKA: We work really hard to establish that trust with our donors, and with No Lean Season, it was funny, Spencer, I sweated that. I didn't actually sweat the decision. To me, the decision was incredibly clear cut. There was no world in which I thought we should be operating a program that wasn't having the impact that we expected, and where we could be using our money much more effectively to save or improve lives. I really sweated about what funders would think, and to your point, it was actually really well received because people said it's exactly like we're effectively living our values. It was a really interesting case. But I will also say this is another thing that I think a lot about, which is how, as a global development community, can we be better at incentivizing high-risk, high-return efforts? I'm on the Trust for the Anthropic, that's an AI company, and I watch the amount of money that flows to what is arguably a higher-risk investment. I think a lot about why that happens there and it doesn't happen in development. I have a hypothesis. Do you see that, Spencer, and what's your hypothesis on that?

SPENCER: It's interesting because I think there's a lot of lip service to, "Oh, we're willing to take risks, and we're willing to hit space giving," but I suspect that it's not done nearly as much as it's claimed to be done.

KANIKA: I find it's quite hard. We have some investors who will do that, but I think a lot want to see the programs working. I reflected on that; we have had a program recently in which I talked a little bit about this work with the Indian government, and I can explain with Safe Water why it's a very high return, but I also think there's some risk to it. It has been interesting how the owners have reacted and what it took to sort of bring some along. I thought a lot about why that was, and my theory is that when you make tech investments, the upside goes to the investor and the downside goes to the investor. It's a very symmetrical equation, but with development, if a program fails, really the NGOs and the funders have to go to their boards and stakeholders and say, "Hey, we made this investment, and it didn't work." Everyone likes that in theory, but it can damage reputations; it can be perceived to damage reputations. The upside when it works accrues to tens or hundreds of millions of people in the developing world. But there's an asymmetry to who gets that benefit, and that's my theory. I don't know; it's the one I sort of knew a lot. I haven't had many people react. I haven't told too many people, but I don't know. What do you think, Spencer?

SPENCER: I think people often view companies with skepticism and say companies have bad incentives because they're just trying to make money. If they can make money by harming people, maybe they'll do that. To an extent, this is true. There's an extent to which companies have an incentive to help you if they can make money doing that, but there's an extent to which they might have an incentive to harm people if they can make money doing that. I think people are much less likely to think about the fact that charities also have really weird incentives. You'd like to think we live in a world where the incentive of a charity is just to help the world, but that's really not true. Charities have to raise money, and if they do things that allow them to raise money but don't help people, some charities are going to say, "We're not going to do that because it's not helping people," but there are going to be plenty of charities willing to do that thing that gets them money, even though it's not helping people, usually not so nefariously, more like they're just going to convince themselves that they actually are helping people. I think the incentive misalignment for charity is way underestimated. As you point out, if you think about it from the point of view of what benefits the person running the program versus what benefits the donor versus what benefits the recipient, they're quite different things. It really takes leadership that says, "No, what we really care about fundamentally is the benefit to the recipient," to push back against that natural incentive.

KANIKA: I think that's exactly right. I think there's a ton of misaligned incentives in the development sector. There's even misaligned incentives for donors. Many donors can aggregate across funders. They may have principals who have certain viewpoints on things; they may be pulling in money from public stakeholders. I think they also have their incentives. Another one that's often really hard is that most nonprofits don't have a lot of unrestricted funding. When we shut down programs, there's a cost to that. You have to have the flexible money to adjust fixed costs — I don't want to get into sort of arcane NGO finances — but there are costs associated with shutting down programs. We were really fortunate that we both have flexible donors who are willing to allow us to do that responsibly and thoughtfully and had some of the flexibility of funding to make those hard decisions. It's a funny thing that there's often mistrust of NGO incentives, and perversely, that often makes having the flexibility to make principled decisions that much harder. It's a strange thing.

SPENCER: I know exactly what you're talking about, and I've seen this happen a number of times, where donors want to have a higher impact, or they want their donations to go to a particular thing they especially care about, and so they put extreme limitations on the funds. They say it can only be used for such and such programs, but it ends up being incredibly distortionary to the behavior of the organization, because the organization thinks, "Well, that's not really the best way to use that money. Maybe we don't even need more money for that thing right now." And they end up just saying, "Well, I guess we have to spend it on this, or we have to try to move other money around." I definitely think that if you really believe in an organization fundamentally, you should try to trust them to do the right thing with the money, rather than saying, "I'm going to force you to use it in this way," because it's a very weird kind of trust. You really trust this organization, but yet you're trying to force them to use the money in a particular way that you preset, and I think that often actually causes more harm than it does benefit.

KANIKA: That's exactly right. I also think a lot about the transaction costs associated. If you can't tell Spencer, I'm a math econ major, I think I spend a lot of my day thinking about efficiency in different ways and the amount of transaction costs that occur for an NGO in the field to be spending time answering a lot of queries from grant officers who may not have worked in development and providing that context. In some senses, I both completely understand why that happens, and it also can come at a pretty high cost to nimble decision-making, just simply leadership, time, and attention. I don't know how to fix it, but it's one of the other things that I think is a real challenge in the field; we spend a lot of time administering that money rather than doing the programs.

SPENCER: So how do you make it okay to take risks?

KANIKA: In my sort of perfect world, you would have donors who are truly willing to take high-risk, high-return decisions. Who can do that nimbly, who works with trust and collaboration with NGO leaders to say, "Okay, let's think about this as high-risk, high-return. Let's think about frameworks that actually allow you to take those risks and at the same time, allow us checkpoints to say, 'Okay, now we actually think this is a genuine implementation problem,' but giving enough latitude for the sort of natural learning up and down, just structure it like you would more of a private sector, high-risk, high-return. We have clear milestones, but there's a lot of latitude for that up and down." So I think that's one thing. Two is, in my mind, to build really trusting relationships between donors and implementers, and three is to give the NGOs the flexibility and the flexible funding so that they can afford to make tough decisions without coming as an existential risk to the organization. I don't know, just for a thing Spencer, but that's what comes to mind for me.

SPENCER: Yeah, I think that last point is especially important, that if it's sort of do or die, it's very hard to take real risks. There aren't that many people willing to do it in that situation. If you're talking about, "Oh, if we screw this up, then our organization is dead, and we all have to go get other jobs," that's going to be a much harder place to take risky bets. Whereas if you think, "No, we can sustain this, we can take a certain number of risks, and even if they don't all pan out, that's fine. We can keep doing what we're doing." I think that may be a point of strength for your organization where you have enough credibility and a track record that you can get away with some failures, and that's fine.

KANIKA: Yes. We also really try to signpost that we may fail when we think we might, which helps. The fact that we've made the decision to exit programs also adds to that credibility. We also save money. I feel this strong sense of responsibility in that our work is saving lives. When you look at banks, they need to hold deposits to be responsible, and individuals are supposed to save money. I often find there's this aversion to NGOs holding unrestricted funds or having reserves. I find that really odd, because ultimately, if you don't have risk capital, if you can't make tough decisions without it being existential, you're going to make non-optimal decisions. A big part of how I've approached this in my leadership team has been to make sure that we do have the savings and we do have the capital to make both strategic bets and really tough choices.

SPENCER: Right. Obviously, there's an amount of capital that would be kind of ridiculous for a nonprofit. If you have huge excess capital, the funders won't give you money. But there's a certain amount of excess capital that lets you take bets, and if it doesn't work out, you can still sustain things.

KANIKA: Exactly. Yeah, that's right.

SPENCER: The way that you operate is so different from how much of the charitable sector operates. From my point of view, a lot of donors don't seem to be following the evidence at all. How do you think about that? Do you think of it as you're just going to look for a target donor base that does care about evidence, and you're just going to say, "Forget the rest of the donors." Or do you try to also appeal to donors that are not really evidence-minded? And if so, how do you do that?

KANIKA: We really try to appeal to a broader range of donors, and that's for a few reasons. The scale and the scope of what we want to do is so massive, and we really do want to widen the base of individuals who want to give. When I look at what we want to do in Safe Water, there are over 2 billion people who need safe water. If we're only preaching to the converted, we're not going to get to that scale. I don't think we've done a great job of this yet. It's one of the things we're really giving a lot of thought to, but it is how to make the work that we do that much more accessible to a wider audience. I'd be curious. Do you have ideas on how we should be, either as Evidence Action, or just more broadly, how we should be doing that?

SPENCER: I think there's a fundamental challenge there, because if you're just trying to pull on people's heartstrings, you don't have an advantage in doing that. In fact, the organizations that are willing to exaggerate, cherry-pick, and focus on one child that has a really compelling story are going to have an advantage over you.

KANIKA: That is really interesting. I still think that even those who want to pull on the heartstrings can get behind real stories of impact. It's a little bit of a test case. With Safe Water, we're giving a lot of thought to it. It is such a visceral thing. If you can deliver it for $1.50, and there is strong evidence that it saves one in five lives, it's not just that it saves one in five lives; the kid who avoids getting diarrhea actually saves more money for their families. They avoid having to go to the clinic. It saves up to $44 per case, so you have real income benefits. I'm hopeful that we can present that in a way that can be appealing to a wider audience. I really think that's something I would like to see for the donors and the community that's more evidence-minded. How do we take what are really great stories of impact and make those more accessible? I think a lot about deworming, and I don't think you need to be deeply in the evidence to say that it's pretty great if you can spend 50 cents per kid per year, and that every dollar invested generates $169 in economic benefits. I feel like that's a stat that should be compelling, but maybe not.

SPENCER: Well, I think it is a compelling statistic. But the problem is that statistics are not primarily what move people. We're talking about most people. So it's a great statistic, but if you're just trying to convince your average donor, the incredible story of that one child whose life was transformed might be more effective than showing statistics at all.

KANIKA: Do you think that's the case? What would you do in my shoes, Spencer?

SPENCER: I think if you had to pick between the story and statistics, the story would win. Obviously, you don't have to pick; you can do both. I think doing both is a value add on top of just doing one. But even when it comes to the statistics, I think there's a really big problem, which is that many groups use misleading statistics. They're not based on randomized control trials. They're based on some back-of-the-envelope calculation that doesn't even properly attribute what they're causing to occur. You could find all these organizations claiming they save a life for 30 cents or something, which is complete nonsense. As far as I'm aware, there's no way to save a life for 30 cents. On average, there's no way to save a life for 30 cents. But there are lots of organizations that claim it, and then they run TV commercials, and people think, "Oh, yeah, you're saving a life for $5,000; that's ridiculously expensive."

KANIKA: I don't have a great answer to that, and I will say I am actually a big fan of a lot of NGOs that do good work. But it is a challenge, I will admit. At the peak, we were reaching 20 million treatments with governments, which was actually 25% of the world's children in need of deworming treatment. But when you stack that against other organizations that might be claiming similar things but are not doing the school-based validation that we did and the prevalence surveys, it just doesn't sound nearly as impressive. So yeah, that is a challenge. We haven't cracked it yet, but I welcome ideas.

SPENCER: And I think the less you care about valid evidence, the more wiggle room you have to make a compelling story. That's unfortunate. I do think that your approach gives you a huge advantage when you're talking to evidence-minded folks. When you're talking to GiveWell, GiveWell would never give to the organization that claims they're saving a life for 30 cents, right? So it does give you a big advantage, but I do think that it gives you a disadvantage with your average donor.

KANIKA: I think that's right. I think that's right.

[promo]

SPENCER: You mentioned a couple of times the economic impacts of some of these interventions, like deworming. Before we wrap up, I want to talk about that a little more. Often we think about interventions as helping at the individual level, but there can be positive second-order effects too. How do you think about that in your work?

KANIKA: We find that health investments don't just save lives. They really can unlock human potential, and we see that because of this linkage, oftentimes between health, education, income generation. Rather than treat them as silos, we really do try to look at the holistic effect. Safe water, we've talked a lot about the impacts on mortality, but again, when a kid is sick, their parents will oftentimes take them to whatever clinic or hospital is needed to save their life. When my kid was sick, especially when kids are under two, they get dehydrated so quickly. In the US, I used to just fly my kid with Pedialyte and milk, and I knew that if there was ever a problem, I could just pop over to the pediatrician. In a lot of these countries, that's not available. Families face dehydration risks that can escalate quickly. These families end up having to go to clinics and spending a lot of money. Sometimes hospitals charge $14 for a clinic visit, $44 if they're at a hospital, and in places where families are making $5 a day, that's an enormous cost, as well as the lost wages. That's just one example of where with Safe Water, we think about the benefits, not only of the lives saved, but also the cost of illness averted, making up a relatively sizable amount of the benefits. With deworming, we were talking about the evidence there. I think that's another one where the nutrition benefits and the income benefits are really intertwined. We were talking about the impact on absenteeism. Work by Michael Kremer and Ted Miguel found that kids who received deworming had a 25% reduction in school absenteeism. Those mechanisms end up meaning that kids earn a lot more. With deworming, every dollar invested, I mentioned this before, creates $169 in economic benefits, and that's what aggregates up to that $23 billion that we've achieved by getting all those deworming treatments out. That's a couple of examples. There's also anemia.

SPENCER: How do you measure the economic benefit of deworming? How do you actually figure out the increased economic activity for each dollar you put in?

KANIKA: That was done by combining a couple of different data sets. I do have to give a disclaimer that I was not the one who did this. If I get this slightly off, I apologize, but fundamentally, we were looking at the work that had been done by Miguel and Kremer, which showed that the children in Kenya who had been dewormed as part of an RCT, 28 years later, when they went back and looked at the cohorts, the kids who had been dewormed increased future earnings by 13%. Interestingly, that actually even reduced child mortality in the next generation. That's just a side point, but I think again, getting to this intertwining of health and economic benefits. We then used the data sets that we have on what it costs to deworm in Kenya. How much are we spending there? A lot of work was done from an epidemiological perspective to think about where kids had been dewormed twice and how to consider that, etc. We sort of factor out and address those things. Ultimately, that's where we found that it was the dollar invested that was generating that $169.

SPENCER: Is it fair to say that those estimates are incredibly uncertain, and that really what we're talking about here is sort of your best guess or expected value estimate? I imagine something like, "Okay, you're tracking them for many years and seeing how much money they're making many years later." But there are very complicated questions there. Are they just now outcompeting other children who would have gotten the wage because now they're a little bit healthier? It seems to me the second-order effects are really difficult to pin down.

KANIKA: As a general point, I will say there is a lot of uncertainty because the world is complex. What we try to do is, when there's uncertainty, for better or worse, take the more conservative estimate. To give you an example, I mentioned that we did modeling, and we had it checked by leading epidemiologists and economists to make sure we didn't miss anything. We asked, "What has been the impact of deworming globally, and of those 2 billion treatments that we've delivered?" We came up with an estimate of $23 billion in productivity gains. In some of the countries we're operating in, particularly in some states in India, prevalence has fallen so low. In fact, in two states in India, for example, our partnership with governments has meant that we've reduced worm problems by over 97%. That means we're now supporting the states to shift strategies, including suspending large-scale deworming and monitoring for resurgence.

SPENCER: You're putting yourself out of business. You're too effective.

KANIKA: We put ourselves out of business, actually. What my board said to me when I showed them that $23 billion — this is a board that has leading economists on it — they think deeply about these issues. They came back and said, "Well, we really think you're being too conservative about this because you're actually only looking at the present-day benefits. You're only looking at the benefits to the kids who were dewormed. You're not thinking about the benefits to kids who will no longer need to be dewormed in the future." That is true, Spencer, but that is a hard thing to model because it hasn't happened before that worms had fallen this dramatically in a geography. We have our detailed models as to when we think resurgence might or might not occur, but we don't know. In that uncertainty, our approach is to zero that out. It's possible, for this example, that we're off by an order of magnitude. I won't deny that, but the truth is, I think what you're saying is fundamentally true. There's just a lot of subjectivity that one needs to operate in, and so we use the best evidence and are also, at the same time, humble about the limitations of that evidence.

SPENCER: I think you're right that it's just an inescapable fact about the world that the world is ridiculously complex. You think of almost any topic that you don't know much about, and then you start learning about it, and you're like, "Wow, there are all these layers to it, and then each layer has layers within it, and so on and so forth." I think this is something that people tend to underestimate because they know their own field. They know how complex it is, but they haven't grappled with the complexity of other fields, and so they have simplified models of how it works. In fact, many of these things are just layer upon layer upon layer of assumptions that we have to make. You do your best job you can, but at the end of the day, a lot of these are irreducible.

KANIKA: Yeah, I think that a lot. I can make a case for any one of our programs to be the most cost-effective based on just key assumptions I know we're making within any given program. You could toggle those based on which data set you use, what subjective assumptions you make. I think there's a wide range for us. What it means is that we accept those error bands and try to really look at what is in the top quintile of things we could be doing based on the available evidence.

SPENCER: What do you see as low-hanging fruit that hasn't been tapped yet? Where you say, "Oh, wow, there are some programs that could be implemented in the future, maybe by you, maybe by others, that currently haven't been pursued to the extent that they should be?"

KANIKA: I'm really excited about anemia prevention and treatment. Anemia may seem like a minor issue, but its impact on children is actually pretty profound. This happens when kids aren't getting enough iron, typically in their diet, and it reduces the ability of the body to carry oxygen, which means just at the most fundamental level, children who are anemic struggle to breathe. They feel constantly fatigued. They find it difficult to learn and focus. Imagine going to school and feeling breathless. It costs pennies to help prevent this. If you give kids iron and folic acid supplements and these simple vitamins, it boosts their IQ scores by four or five points. Then you have countries like India and Malawi, where more than half the girls are anemic. In those contexts, it's a low-cost intervention, and it's a game changer. You can address a large percentage of the anemia cases, and it ends up breaking the cycle of poverty because you're boosting cognitive function and future productivity. That's something we've had a lot of success with in five states in India, and we're able to help reach 35 million kids. There is a huge need in a lot of the geographies we operate in; there's just a huge burden of anemia. Addressing that, both for kids and for pregnant women, can have real impact. We're excited about that as the next frontier.

SPENCER: That's really interesting. We talked about opportunities, but on the flip side, are there certain interventions you think are way overhyped, where too much money is going into them? Even if they might seem promising, do you think they're actually not as promising as they seem?

KANIKA: I think it really is. The scale of need and the things that we're covering. In Safe Water, there are 2 billion people who need safe water. Deworming, I was really proud that we were able to reach 25% of the kids who needed it, but there are a lot of other disease areas where we just haven't been able to get the resourcing or the partnerships to really make that kind of a dent. I think about maternal syphilis, and we talked about what a no-brainer that is, how easy it is, and how cheap it is. Still, after our years of doing this, there are hundreds of thousands of kids who are needlessly dying from something that would cost pennies. The biggest challenge is how to get the resourcing we need for stuff that is, in my mind, kind of criminal that it isn't happening. That, I think, is the thing that keeps me awake: the fact that there are so many opportunities where we could be making such a massive difference, and the scale and the scope we can do, given the resources we have, is just not allowing us to really do as much as I think we would like to.

SPENCER: Is that mainly bottlenecked by funding, or is it bottlenecked by other factors, like finding the right partners or finding governments to work with?

KANIKA: The biggest bottleneck is funding. Yeah, predictable funding. If we had predictable funding, we could scale, and I think we could be doing a lot more.

SPENCER: And for listeners that might be interested in supporting you, what's the best way to do that?

KANIKA: You should go to evidenceaction.org, to our donate page. We'd be really excited by that.

SPENCER: Thank you so much for coming on. This was a fascinating conversation.

KANIKA: Thanks for having me. It was a lot of fun. I really enjoyed it.

[outro]

JOSH: A listener asks: "Which of these milestones do you think will be reached first: commercially viable nuclear fusion or unambiguous evidence of life elsewhere in the universe?"

SPENCER: Interesting question. It's a tough one. If we assume that humanity doesn't wipe itself out, it seems very likely that eventually we'll have nuclear fusion that works really well. It does seem like that's an engineering trajectory that we're on. We're not immediately there, but we're getting there. On the other hand, if you think about any kind of alien life, even a sort of single-celled alien life, it does seem fairly likely that it exists out there in the universe. There are a huge number of planets, and there are some scenarios under which there might not be other life out there, even though there are a huge number of planets. But it seems like a pretty good shot that it's out there. And then the question is, can we detect it? We have been making a lot of progress in identifying Earth-like planets that are at least at appropriate distances so that everything wouldn't be scorched from the center of their own solar systems. That being said, it might also be a huge challenge to be confident that life really is there. Perhaps it can be detected from certain atmospheric conditions. So I honestly don't know. I feel like we seem to be on a trajectory for both of them; it's tough to say which could happen faster. I think I would go with probably nuclear fusion.