The Challenge of Measuring Changes in Hygiene Practices

Published on: 18/09/2014

For many years the WASH sector has monitored project activities using easily measurable indicators; i.e. by counting the number of latrines constructed by a given sanitation project, measuring the quantity of water supplied by a new pump or totalling the number of public hygiene promotion sessions carried out in a village. While these are very good indicators of what activities have been carried out, they do not, however, measure a key element: what level of service do the beneficiaries receive? What new hygiene practices have been adopted thanks to the project activities?

The particular case of hygiene interventions

Focusing on the adoption of new practices or on behaviour change could mean that it is the "hygiene" component of a WASH programme that is under scrutiny, in other words, hand washing, but no. Hygiene interventions are by definition actions that have an impact on the adoption of hygiene practices. Therefore, hygiene interventions may concern the installation of a water point, a latrine or a hand-washing stand, or even a programme that combines one or more of these components. For example, the construction of a latrine and awareness-raising to encourage people to use it can have a direct impact on the practice of open defecation.

Therefore, assessing the result, effect or impact of hygiene interventions requires both the equipment necessary for behaviour change as well as behaviour itself to be taken into consideration. It is not just a matter of assessing the service offered by a latrine (for instance), but also of assessing how the beneficiaries use it (or don't) over time. This is where the difficulty of evaluating hygiene action effectiveness lies.

Of course analysis methodologies do exist. However, assessing behaviour and behaviour change over time can only be scientifically validated by longitudinal studies (observations over several years), methods that are rarely able to be envisaged in the context of development projects. Since 2008, IRC has been working on methodologies to measure water and sanitation service delivery and is trying to apply these methodologies to the world of hygiene, with the objective of finding a way to analyse behaviour change in the short term as well as the cost associated with such change (for more details on the financial dimension please refer to "Assessing hygiene-cost effectiveness: a methodology", as well as other IRC publications on the long-term cost approach on the WASHCost website – see links below).

The key hygiene practices considered by IRC to be representative of behaviour change for WASH actions are:

1. The use of improved latrines by the whole household all the time,
2. The washing of hands with soap at critical moments
3. The use and safe storage of drinking water

Other behaviours are also explored; in particular the reuse of excreta collected using EcoSan toilets (covered by another blog – see the link below).

Development of a hygiene practice scale

How did we measure changes in these practices? By breaking them down and grouping them according to categories that cover the whole spectrum from unhygienic to ideal practices. Presented in the form of a scale, this can be used to situate a beneficiary on the hygiene practice spectrum. Thus it becomes possible to compare the "before" and "after" situation for a given intervention, in other words to measure the effectiveness of hygiene interventions.

There are four levels on the hygiene practice scale. In the case of unhygienic practices, the household members lack access to the basic equipment: no latrine, no hand washing stand, no safe source of water at household level or treatment system for drinking water. At the other end of the scale, ideal practice exceeds standard recommendations and all conditions are met. Between these two extremes there is:

• Acceptable practice, which represents the minimum standards compliant with national and international standards,
• Transitional practice which, while not acceptable, highlights willingness and the beginning of behaviour change. This level is important as it makes it possible to identify the first steps towards changed behaviour even if this remains unsatisfactory.

This methodology, which has been piloted in three countries (Burkina Faso, Ghana and Mozambique) and then tested in four other countries (Sierra Leone, Bangladesh, Ethiopia and Bhutan) is constantly being improved and it helps to measure changes in practices in a realistic way that is accessible for all stakeholders. However, in a context where hygiene actions are carried out over relatively short periods, this methodology cannot be used to measure long term change. Without public health bodies that are able to monitor this, NGOs and other development stakeholders would seem to be best placed to monitor changes in hygiene practices.

We will explore the results of the application of this methodology in future blog entries, but suffice to say that the provision of equipment alone is not enough to initiate behaviour change (much less to maintain it). Furthermore, it would appear that the behaviour change and promotion methodologies used by NGOs do not always work, even in the short term (2 years).

Measuring the effectiveness of action is useful but it is only the tip of the iceberg. We need to have methodologies to better understand the social and psychological mechanisms inherent to behaviour change. The WASH sector has the advantage of being able to develop synergies with the health sector to ensure that the equipment installed is correctly used and that users get real benefits in the short and long term.

Mélanie Carrasco and Amélie Dubé, IRC