What We’re Learning: Long-Term Outcomes of USAID’s Water and Sanitation Efforts

To better understand the long-term impact and sustainability of its interventions, the USAID Water Office is conducting a series of independent ex-post evaluations of the Agency’s water, sanitation, and hygiene (WASH) activities to inform future USAID investments in the sector.

This evaluation series will help USAID understand whether and how its activity results have been sustained years after projects close. All activities included in the series must have been closed for a minimum of three years and could not be recipients of Office of U.S. Foreign Disaster Assistance or Food for Peace funding. Preference is given to USAID missions that are at a point in their design cycle to incorporate learnings into upcoming WASH programs. This evaluation series builds upon USAID and Rotary International’s WASH Sustainability Index Tool, which is a framework to assess a WASH activity’s likelihood to be sustainable according to five factors: availability of finance for sanitation, local capacity for construction and maintenance of latrines, the influence of social norms, and governance.

In September 2016, the first evaluation in this series studied the Madagascar Rural Access to New Opportunities for Health and Prosperity (RANO-HP) activity.

What We Did

RANO-HP, implemented by a consortium led by Catholic Relief Services from 2009 to 2013, aimed to increase sustainable access to safe water supply, improve sanitation coverage, and expand hygiene practices.

This evaluation of RANO-HP focused on the sustainability of the activity’s sanitation and hygiene components, which included community-led total sanitation (CLTS), behavior change messaging, public WASH “monoblocks” (combined public water point, latrine, shower and laundry station) managed via public-private partnerships, commune-level water and sanitation business plans, village savings and loans associations (VSLAs), and microfinance products for sanitation investments. Through household survey and interviews with beneficiaries the evaluation team explored barriers and facilitators of sustainability.

Villanova University recently completed a similar evaluation of RANO-HP and sister activity RANON’ala, which focused on the sustainability of both activities’ water supply interventions and was conducted during the same time period as this evaluation. A webinar presenting results from both studies is planned for Fall 2017.

What We Learned

Unfortunately, many of RANO-HP’s results were not sustained three years after the activity closed. Despite the large gains in sanitation coverage achieved through RANO-HP’s CLTS interventions, four of the activity’s five targeted regions showed significant slippage in latrine use compared to the endline. Both shared household latrine use and private latrine use declined. The evaluation also found that open defecation was being practiced in most communities that were declared to be open defecation free at the end of RANO-HP: a partial open defecation free verification process found that in 80 percent of villages, former open defecation zones were still being used. Households with a non-literate respondent were less able to sustain WASH outcomes. However, female-headed households were slightly more likely to use a latrine compared to male-headed households, representing a reverse of the trend observed in 2013. Female-headed households also opted for higher quality latrines over time by a small margin.

The simple technologies promoted by RANO-HP (pit latrines and tippy taps) made it feasible for beneficiaries to adopt sanitation and hygiene practices. However, many households did not move up the sanitation ladder, and were not able or willing to invest in sanitation as this infrastructure decayed. Beneficiaries reported that financial constraints were the main barrier to maintenance of improved sanitation. To address this barrier, RANO-HP established VSLAs and a microfinance loan product, and a few communities have continued to utilize the VSLAs to finance WASH. One microfinance loan product established by RANO-HP was still in use, but this was only sustainable in an urban area.

These findings led to several recommendations for future USAID programs, including:

• CLTS can lead to rapid adoption of sanitation, but maintaining these gains is a challenge and may require interventions beyond CLTS, ideally implemented by local governments. Joshua Garn et al.’s recent study yields similar results.

• Financing for sanitation continues to be a challenge, and while VSLAs and microfinance have shown some mixed successes, future programs should continue to seek ways to address this barrier. A recent issue of USAID’s Water Currents newsletter focused on this topic.

• Strategies to facilitate sustained WASH behavior might benefit from addressing gendered decisionmaking dynamics, as differences between female and male headed households were observed.

Next Steps

The evaluation series is a major investment that USAID is making to better understand the factors affecting sustainability of its WASH interventions. The second evaluation in this series, of the Indonesia Environmental Service Program, will be available soon and a third on the Millennium Water Alliance-Ethiopia Program will be completed by the end of 2017. A synthesis of findings across all of the evaluations will be available at the end of the evaluation series, in addition to the evaluation reports and briefs produced for each evaluation.

Click here to view the full evaluation report on Globalwaters.org. A shorter evaluation brief provides a summary of the findings.

Join a webinar on this evaluation’s findings and those of a companion RANO-HP evaluation on water systems conducted by Villanova University’s Sustainable WASH Research Initiative on October 3. Click here to register.

Annette Fay, Water Communications and Knowledge Management Project monitoring and evaluation specialist and lead researcher for the evaluation series, and Elizabeth Jordan,  USAID water and sanitation specialist.