Color wheel measurement problems

GiveWell aims to find and fund programs that will do the most good per dollar. To do this, we carefully evaluate potential grants before making them—assessing academic evidence, building cost-effectiveness models, and talking to people in the sector who know the program well. 

But our work doesn’t stop there. When a program we’ve supported nears the end of their funding, we also regularly evaluate its results to decide whether to continue our support. This typically involves gathering and analyzing extensive monitoring data. In most cases, the results are consistent with what we expected, and we renew the programs’ support. But sometimes we decide that, even if a program is doing a lot of good, it may not be having the impact we expected. In that case, we decide not to renew our support and instead direct those funds to where we think they’ll do much more good for people in need. 

In this episode, GiveWell CEO and co-founder Elie Hassenfeld speaks with Senior Program Officer Erin Crossett about the research that led GiveWell not to renew support for Evidence Action’s Dispensers for Safe Water—a program that installs chlorine dispensers at rural water points so that households can treat their drinking water and reduce waterborne disease—in Malawi and Uganda.

Elie and Erin discuss:

• How independent data revealed a significant gap in program reach: Early signals from a separate GiveWell-funded study and Evidence Action’s own internal review of the program in Kenya suggested chlorination rates were far lower than routine monitoring indicated. GiveWell then commissioned an independent survey in Uganda and Malawi to find out whether the same was true there. The survey found that only about a third as many people were using dispensers as previously estimated: roughly 2 million rather than 5 million. 
• Potential reasons for the data discrepancy: We believe that no single error drove the discrepancy. Instead, there were five or six contributing issues that together caused the differences in estimated usage. For example, chlorine was measured by matching a test result to a color wheel, which can be subjective and affected by lighting. This data was collected by Evidence Action’s own staff, which may have led them to interpret the color wheel results more favorably. 
• What GiveWell learned and its effect on future grantmaking: We believe that approving the initial grant in 2022 was the right decision, given what we knew at the time. We also think that we could have done better by, for example, investing earlier in independent verification. We now apply these lessons to our current grantmaking. For instance, our recent portfolio of safe water grants includes external surveys for all grants. 

As we've developed and grown our research team over the past several years, we’ve become increasingly able to support additional data collection, analyze and learn from our grants, and reallocate resources to the most cost-effective global health and development needs we find. 

Getting things right requires both honest self-assessment and grantee partners willing to open themselves to scrutiny, and we are grateful for Evidence Action’s partnership in identifying the discrepancy in usage rates. Evidence Action is now reducing its footprint in Uganda and Kenya, and winding down the program in Malawi, with a 24-month transition to help support communities and turn over operations to the governments where possible. 

Dispensers for Safe Water is still a program that helps people, and our grant provided clean water to millions. GiveWell’s decision not to renew reflects our mission to direct donor funds to where they will do the most good—not to fund everything that does good. We are continuing to support chlorine dispensers in contexts where we believe they will be highly cost-effective. For instance, we are currently considering a grant to Evidence Action to pilot variations of the program in northern Nigeria, where disease burden is much higher than the countries where we had been funding the program—and therefore the impact per dollar might pass our high bar for funding. 

Visit our All Grants Fund page to learn more about how you can support this work, and listen or subscribe to our podcast for our latest updates.

This episode was recorded on February 20, 2026 and represents our best understanding at that time.