A woman collects water in Zambézia Province, Mozambique. Photo credit: Forcier Consulting.

Ex-Post Evaluation: Strengthening Communities through Integrated Programming (SCIP) Activity in Mozambique

Women at tippy tap

USAID funded the Strengthening Communities through Integrated Programming (SCIP) activity in Mozambique from 2009 to 2015 in Zambézia and Nampula provinces. This evaluation focuses on SCIP’s WASH activities in Zambézia—WASH interventions to increase hygiene practices and use of clean water and sanitation facilities using community-led total sanitation and participatory hygiene and sanitation transformation.

What We Did

The evaluation team looked at the WASH component of the Strengthening Communities through Integrated Programming (SCIP) activity implemented from 2009–2015 in Zambézia Province, Mozambique. SCIP Zambézia designed WASH interventions to increase hygiene practices and use of clean water and sanitation facilities using community-led total sanitation and participatory hygiene and sanitation transformation (PHAST).

How We Did It

This evaluation assessed the sustainability of WASH outcomes in 239 communities four years after the activity ended. In August and September 2019, an evaluation team conducted 990 household surveys, including observations of 496 latrines and 836 handwashing stations; observed 46 water points (WPs), including water quality testing of 30 functional WPs; and completed 43 qualitative interviews with government officials, implementers, community health and water management committees, and community members.

What We Learned

Water point functionality and use: Four years after the activity ended, 65 percent of observed water points remained functional. Almost half of the water points tested positive for E. coli and 90 percent tested positive for fecal coliforms. This exposed communities to health risks, as only 16 percent of households reported treating their drinking water. 

Water point management: Most WPs had active water management committees in place, and community members reported a high level of satisfaction with these committees. Sixty-six percent of respondents reported paying WP fees, which often proved insufficient to cover operation and maintenance costs. Fewer households below the poverty line made these contributions. 

Sanitation quality and use: Fifty-two percent of households had access to a latrine, but only 15 percent had basic sanitation access, that is, access to an improved latrine that is not shared with other households. That level of basic sanitation was still higher than the 5 percent recorded at the activity’s (quasi-comparable) endline evaluation. Households below the national poverty line had significantly lower basic sanitation access than those above the poverty line. The evaluation team observed latrine quality as generally poor; only 23 percent of households had an observed slab. Respondents self-reported a low level of latrine use, and open defecation reportedly occurred in approximately 75 percent of the communities. Respondents reported financial and material barriers as the primary reasons they did not own a latrine. They aired frustrations about poor-quality materials leading to a continuous cycle of repair or replacement of latrines, especially after weather-related shocks.

Hygiene: Households reported high levels of handwashing with soap and water (64 percent); however, only 13 percent had observable soap and water. Of the 95 percent of households with a handwashing station, less than 2 percent had a fixed-in-place facility. The evaluation team rarely observed SCIP–promoted tippy taps, due to their reported lack of durability. Handwashing behavior did not appear to be normative.  

Resilience: While SCIP did not directly seek to enhance resilience, the evaluation team did assess impacts of cyclones Idai and Kenneth and flooding, which affected roughly half of respondents’ WASH facilities. Replacement and repair of handwashing stations took place at a high rate. However, communities struggled to repair damaged water points; less than half of respondents reported resolution of issues several months later.

Next Steps

These results suggest a need to rethink several programming approaches. Understanding the long-term sustainability of USAID–supported WASH outcomes allows USAID and other stakeholders to identify challenges and best practices that can inform future programmatic strategies. This evaluation provides further evidence of persistent challenges with community-based water point management, including the sufficiency of fee collection, high rates of nonfunctionality, and accountability of government for water quality monitoring.

In line with current sector understanding, this evaluation further codifies that PHAST approaches to handwashing promotion, low-quality handwashing stations (e.g., tippy taps), and behavior change messaging alone are ineffective in fostering normative handwashing behavior in rural communities.

About The Ex-Post Evaluation Series

The USAID Water Office is conducting a series of independent ex-post evaluations of the Agency’s water, sanitation, and hygiene (WASH) activities to inform future USAID investments in the sector and to better understand the long-term impact and sustainability of its interventions several years after projects close.

This evaluation series will help USAID understand whether and how its activity results have been sustained. All activities included in the series must have been closed for a minimum of three years and could not be recipients of Office of U.S. Foreign Disaster Assistance or Food for Peace funding. Preference is given to USAID missions that are at a point in their design cycle to incorporate learnings into upcoming WASH programs.

This evaluation series builds upon USAID and Rotary International’s WASH Sustainability Index Tool, a framework to assess a WASH activity’s likelihood to be sustainable according to the following factors: availability of finance for sanitation; local capacity for construction and maintenance of latrines; the influence of social norms; and governance.