Last week, the Para Todos, Por Siempre (Everyone, Forever) initiative presented its results over 2019. It led me to reflect on the good, the bad and the ugly of our experiences with monitoring in Honduras.
Published on: 02/03/2020
Para Todos, Por Siempre (PTPS) is a partnership between 18 NGOs, government agencies and some 28 associated municipalities in Honduras. The aim of the partnership is to reach universal and sustainable access to WASH services in Honduras by 2030. It does so by working to create access in the 28 associated municipalities, and by strengthening the WASH system in those municipalities to such an extent that services can be sustainable. The approaches, methods and lessons are subsequently used to develop a model for reaching everyone, forever, and for promoting that approach amongst its members, as well as towards national government.
In order to assess progress, PTPS carries out an annual monitoring exercise, facilitated by its secretariat. The exercise consists of the following (more details available here):
This is complemented by a reflection from the government on the national level indicators, such as national levels of access to water and sanitation services (drawn from the JMP website), financial data and changes in the elements of the WASH system, as captured through specific aspects of sector development and reform.
So what transpired in 2019?
The third annual monitoring of progress of PTPS across its 26 associated municipalities and in national system strengthening
Let me start with the bad. Progress in terms of access, levels of service and performance of service provider at municipal level could not be assessed due to not having updated data. As mentioned above, PTPS uses mainly data from SIASAR. For most of the associate municipalities, data were collected through SIASAR a few years ago. Since then, data are scarcely being updated. Municipalities are expected to regularly visit communities, and fill out the data about the communities. However, the institutional mandate is weak, as this is not obligatory, and there are no incentives or penalties for (not) doing so; nor are there any fixed budgets for this. So in practice, data on access, service levels or service provider performance are only updated in SIASAR, when there are projects or programmes, funded either by NGOs or national government, which provide budget for this. There is no mandate nor budget for regular data collection and updating by municipalities. And even most of the PTPS member NGOs do not systematically plan and budget for contributing to such data collection and analysis through SIASAR in their projects.
This apparently bad situation triggered something good: a critical reflection on the need to tighten up these institutional responsibilities (and budgets). Monitoring is supposed to lead to reflection and learning. And in PTPS's case, the lack of monitoring data led to reflection. The situation with not-updating of data is not new; last year's report already mentioned that for few municipalities data were updated. The sector realised that organising monitoring in this way – i.e. promoting the benefits of SIASAR and appealing to the willingness of mayors – would not yield better results next year. It has therefore been agreed that within the Commission for Sector Reform a proposal would be elaborated to define institutional responsibilities more clearly, and make annual monitoring a strong obligation.
The same happened in a discussion around the institutional capacity of municipalities. The monitoring revealed that the institutional capacity of most municipalities associated with PTPS has not improved. In fact, in some of the municipalities that have received significant amounts of support, their capacity has gone down. But this negative result was turned into something good. A very critical reflection was held on whether the sector is not expecting too much from municipalities in terms of their institutional capacity. They are supposed to have a number of municipal platforms, have a number of dedicated staff and carry out a range of tasks related to WASH. The monitoring revealed that those platforms very quickly become inactive after initiating them. The smallest municipalities simply cannot afford to have staff exclusively dedicated to WASH. And as a result, many of the tasks are only carried out as part of externally-funded projects. Constructive proposals were presented to actually look into alternatives. One of the most promising ones is to have some of the key tasks not executed at municipal level, but at the level of mancomunidades (groups of municipalities). In that way, economies of scale can be obtained in staffing and institutional capacity for WASH. Great examples from a mancomunidad in the department of La Paz were presented at the monitoring meeting, which show high potential.
The lack of quantitative data also led to a re-appreciation for qualitative data. PTPS also organises reflection meetings with individual municipalities. Even though they may lack quantitative data on the number of people served, they do know the number of new water systems built, and the situation in most communities in their area of jurisdiction. They are able to use this qualitative data for reflection on the way forward. It may not be ideal to only work with qualitative information, but it is considered better than no information at all.
One of the ugly practices around monitoring is the setting up of parallel monitoring systems by NGOs. They often don't do that with bad intentions but driven by the absence of a functional national monitoring system. Luckily, PTPS has been able to avoid this ugly risk. One of the members needed data for its own planning purposes, and it found SIASAR data to be lacking. It developed a quick survey, which would not be filled out in the field, but as a desk-top exercise, in a meeting with the association of water committees and municipal technicians. Instead of seeing this as a risk of a parallel information system, the sector has taken this as an opportunity to discuss whether the current way of data collection of SIASAR is actually appropriate. Sure, a field visit to every single community to collect data for SIASAR would be ideal. But it is also costly. And many data would not change a lot from one year to another. Filling out data in a meeting with representatives from the water committees and municipalities' technicians can still provide good enough data. The experiences from this NGO on monitoring at a distance are now being discussed as an alternative for the more extensive field visits.
No updated data on service levels; institutional capacity of municipalities sliding back in spite of all the effort PTPS members are putting into that; and potentially parallel data collection exercises. At first sight, the annual monitoring of PTPS over 2019 revealed a bad, and even ugly situation. But – as monitoring is supposed to do – it triggered the good: a reflection on whether we are taking the right approaches – to institutional responsibilities for monitoring, to municipal capacity building, to data collection – and examples of members and associate municipalities trying out alternatives that have potential. Above all, it leads to a reflection that the perfect should not be the enemy of the good when it comes to monitoring. It is better to reflect on impartial qualitative information, collected through a desk exercise with technicians and water committees than not monitor at all - even though it falls short of the ideal of monitoring that we had put forward as PTPS.
At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.
