Washing Hands

Ex-Post Evaluation: Synthesis

Tippy tap

This synthesis summarizes key findings from a series of six independent ex-post evaluations that assessed the extent to which USAID–funded WASH activities sustained outcomes three to 10 years after closure and which factors drove those outcomes. The findings are meant to foster learning and improve evidence-based sustainable development assistance at USAID and across stakeholders in the WASH sector.

Evaluation What Does it Take to Sustain Water, Sanitation, and Hygiene Outcomes? Lessons from Six Ex-Post Evaluations
Read More
Event Recording A Synthesis of Findings from Six Ex-Post USAID WASH Activity Evaluations
Read More
Blog Unpacking the Drivers of WASH Sustainability
Read More
Brief Brief: What Does it Take to Sustain WASH Outcomes? Lessons from Six Ex-Post Evaluations
Read More
What We Did

Through its commitment to identifying sustainable approaches to water, sanitation, and hygiene (WASH), USAID commissioned a series of six ex-post evaluations of its WASH activities completed three to 10 years prior. These studies identified what outcomes had been sustained years later and why. The synthesis shares key findings from the series, which examined four rural and two urban WASH activities. All evaluations involved mixed qualitative and quantitative methods.  

  • Madagascar: Rural Access to New Opportunities for Health and Prosperity (RANO-HP), implemented 2009–2013 to improve WASH access in primarily rural areas (the ex-post only evaluated the sanitation and hygiene components). 
  • Indonesia: Environmental Services Program (ESP), implemented 2004–2010 to improve and expand safe water access in urban areas with a focus on support to utilities. 
  • Ethiopia: Millennium Water Alliance Ethiopia Program (MWA-EP), implemented 2004–2009 to improve WASH access in rural areas. 
  • India: Financial Institutions Reform and Expansion–Debt and Infrastructure (FIRE-D), implemented 1994–2011 to capacitate local stakeholders to plan, manage, and finance urban water and sanitation development, with a focus on commercial viability and social inclusion. 
  • Senegal: Programme d’Eau Potable et d’Assainissement du Millénaire (PEPAM/USAID), implemented 2009–2014 to improve WASH access in rural and peri-urban areas. 
  • Mozambique: Strengthening Communities through Integrated Programming (SCIP), implemented 2009–2015 to improve WASH access in rural and urban areas.
How We Did It

USAID commissioned its Water Communications and Knowledge Management (CKM) activity to complete the six independent evaluations to assess the extent to which USAID–funded activities achieved sustained WASH outcomes and which factors drove those outcomes. This report synthesizes findings across all six evaluations, with additional analysis of WASH literature, to foster learning and improve evidence-based sustainable development assistance across the WASH sector. Findings address topics of finance, governance, water service management, and hygiene and sanitation behavior change.

What We Learned

Finance

  1. Technical assistance to service providers for business planning, improved financial management and cost recovery, and credit enhancements were largely sustainable interventions, particularly when scaled through partnership. However, without an appropriate enabling environment, these strategies proved insufficient on their own to facilitate access to market-based finance, toward the goal of self-sufficiency.
  2. Interventions leveraging technology, such as e-governance tools and improved metering and meter-reading, facilitated long-term improvements in financial management and cost recovery, especially through the reduction of non-revenue water.
  3. Training community-based management (CBM) entities to establish bank accounts and transparent accounting practices were insufficient to achieve sustainable finance for rural water system operations and maintenance.
  4. While some limited microfinance continued to be available for WASH, it did not drive increased access to WASH services. This failure shows the need for appropriate alignment of lender incentives and processes with project goals and beneficiary needs.

Governance

  1. Unclear roles and responsibilities across actors prevented effective service delivery and discouraged sustainability. Inadequate training and resources to carry out those roles also inhibited sustainability.
  2. A committed government is an important driver of activity sustainability, particularly in terms of WASH governance reforms.
  3. Unclear tariff policies covering rural areas led to widely varying tariffs across communities, nearly all of which did not attain adequate cost recovery.

Management

  1. CBM entities did not successfully ensure long-term sustainability of water services in rural areas, regardless of capacity building and training interventions.
  2. Cost recovery for water service delivery was a key challenge to sustainability. In both rural and urban areas, setting cost-reflective tariffs and collecting these fees was a challenge, with some rural systems unable to collect fees at all. Operational inefficiencies and high losses also contributed to this challenge in urban areas.
  3. The use of public-private partnerships (PPPs) for water service delivery showed mixed success, due in part to the lack of cost recovery, government inexperience with management and oversight of PPPs, and inadequate relationship building among stakeholders.
  4. Efforts to strengthen supply chains for water system spare parts to enable maintenance by local technicians largely failed.

Hygiene and Behavior Change

  1. Handwashing social and behavior change programs focused primarily on knowledge, such as PHAST, did not work, nor did they address the multidimensional drivers of and structural barriers to behavior change.
  2. Community-led total sanitation (CLTS) did not eliminate open defecation in the long term, and communities relied upon poor quality latrines. Triggered households did not generally progress toward improved or basic sanitation, meaning households missed out on substantial health benefits over the long term.
  3. Financial barriers served as the major impediment to sustained latrine quality, maintenance, and use. Targeted subsidies combined with CLTS in Senegal offered modest improvements to sustained outcomes.
Next Steps

Overall, the ex-post evaluation series illuminates challenges and successes and provides evidence within specific contexts on whether and how particular approaches achieved sustained outcomes. While this report addresses specific activities and contexts, the findings should resonate with governments, donors, and practitioners and reinforce shifts already underway within the WASH sector. It is envisioned that the ex-post series findings, within the context of the broader literature, will provide additional insight and evidence for USAID and the WASH sector to further collective goals of improving sustainable WASH service delivery.

Read More

About The Ex-Post Evaluation Series

Sample alt text

The USAID Water Office is conducting a series of independent ex-post evaluations of the Agency’s water, sanitation, and hygiene (WASH) activities to inform future USAID investments in the sector and to better understand the long-term impact and sustainability of its interventions several years after projects close.

This evaluation series will help USAID understand whether and how its activity results have been sustained. All activities included in the series must have been closed for a minimum of three years and could not be recipients of Office of U.S. Foreign Disaster Assistance or Food for Peace funding. Preference is given to USAID missions that are at a point in their design cycle to incorporate learnings into upcoming WASH programs.

This evaluation series builds upon USAID and Rotary International’s WASH Sustainability Index Tool, a framework to assess a WASH activity’s likelihood to be sustainable according to the following factors: availability of finance for sanitation; local capacity for construction and maintenance of latrines; the influence of social norms; and governance.