Syria Emergency Mulitsectorial Humanitarian Response Final Evaluation 2020

The programme aimed to achieve its objectives by:

1. Rehabilitating water stations and networks, undertaking vector control activities, chlorination mapping and emergency response, and sustainable solutions for water and sanitation facilities for camps;
2. Providing three months food security assistance for the most vulnerable internally displaced persons (IDP) and host community households through food vouchers and agricultural start-up kits;
3. Providing multi-sector support to IDPs and returnees living in camps, informal settlements and damaged/unfinished buildings through non-food items (NFI), Shelter and winterization activities;
4. Establishing Protection centres and child friendly spaces in the most conflict-affected and inaccessible locations while also mainstreaming protection across all activities.

Final Evaluation

A final evaluation was undertaken to assess the degree to which the ECHO-funded programme met its stated scope and objectives against the Development Assistance Committee (DAC) criteria of Relevance, Appropriateness, Coverage, Effectiveness, Efficiency, Impact, Connectedness and Coherence. Due to COVID-19 travel restrictions, the evaluation was led remotely and involved a desk-review of key programme documents, reports and monitoring data as well as a qualitative primary data collection exercise, including in-person (through a local consultant) focus group discussions (FGDs) with programme participants, in-person key informant interviews (KIIs) with local authorities and other stakeholders, and remote KIIs with Concern staff. The programme performance under each DAC criteria was then scored on a 0-4 scale, with 0 representing a totally unacceptable performance and 4 representing an outstanding performance above expectations.

Evaluation results

The programme was assessed as satisfactory (3) in terms of its relevance, in that it has addressed the immediate priority needs identified at programme design stage, succeeding to reach some of the most vulnerable communities, while also being able to adapt its approaches, implementation strategies and targets as the situation dramatically changed during the course of the programme. Going forward, to ensure the interventions remain relevant to the constantly changing situation, it is recommended that Concern Syria considers the possibility of conducting a Context Analysis or at least a Multi Sectorial Needs Assessment including a Gender Analysis or a Rapid Gender Analysis to ensure the needs of women, men, boys and girls are understood and addressed.

On appropriateness, the intervention was scored as outstanding (4). The time period covered by this programme was extremely challenging with an escalation in conflict as well as the COVID-19 crisis that hit starting from April 2020. These events caused massive people displacement and a serious economic crisis, exacerbated by the depreciation of the Syrian Pound, and increased the number of people in need and their overall needs. Not only did Concern find itself in the position of having to adapt all its interventions to the COVID-19 pandemic, it also had to respond to an unprecedented number of support requests from poor and vulnerable people (to a point, 1,000 per month). Concern’s agile structures and great flexibility have allowed them to effectively and timely respond to the new emerging needs and adapting the existing interventions to the pandemic, demonstrating its ability to go above and beyond the originally established scope and targets when needs arose.

Coverage was also found to be outstanding (4) during this evaluation. During the course of the programme, Concern was one of the very few INGOs whose coverage had overall not contracted with an escalation in conflict 2019 and the COVID-19 pandemic. As of today, Concern works across Northern Syria, and is often the only or one of the few INGOs present or the only INGO to provide a certain type of support; even though it is not always possible for Concern to reach some of the most remote locations due to access and security reasons. Concern‘s high quality targeting mechanisms ensure that the poorest and most vulnerable households and individuals are targeted. Going forward, as needs continue to increase due to the economic crisis and context continues to change, Concern could consider reviewing its geographic and household level targeting strategies to ensure it is present in the most vulnerable locations and continues to target the most vulnerable households and individuals

The effectiveness of the intervention was assessed as satisfactory (3). Despite the extremely challenging circumstances encountered during the programme timeframe, Concern managed to implement all the activities proposed under this action, often exceeding its targets in terms of number of individuals and households reached, as well as in terms of outcome indicators performance. In total, the intervention reached 837,473 individuals through this action, significantly exceeding the target of 299,720. With the exception of the kerosene stoves distributions and the WASH services in camps, interventions under this programme were met with great satisfaction among both male and female beneficiaries in rural and urban areas, as well as in camps and informal settlements. Where satisfaction is lower, Concern Syria is aware of the issues and will have to work into advocacy options for these to be addressed at the appropriate levels. Lastly, greater attention should be put on monitoring unintended consequences and the extent to which women benefit from the programme compared to men.

The efficiency criteria was assessed as satisfactory (3). The COVID-19 pandemic and the related severe restrictions (including lockdowns, movement restrictions and curfews) have forced Concern to re-think, re-prioritise and adapt all its existing and planned activities while posing all sorts of logistical challenges in the day-to-day operations. This could all be done thanks to the sound management of day-to-day operations and strong pre-existing collaborations across teams internally, and with the Clusters and the Local Authorities. The light and flexible monitoring system in place has allowed Concern to collect and analyse information throughout these difficult months, while programme participants do communicate with Concern through a well-established Complaints Response Mechanism (CRM) system of which they have good knowledge. Programme participants are not consistently included in programme design – this is an area Concern needs to strengthen going forward.

The impact of the intervention was assessed as satisfactory (3). The intervention was mostly of an emergency, life-saving type; despite this, the water rehabilitation works on the one hand and the Protection component on the other will have more longer-term benefits on the lives of individuals, households and communities. The medium-to-longer term effects of the food security and shelter/winterisation components were mostly hindered by the worsening of the economic situation, but also arise due to the many compelling needs and priorities that extremely poor and vulnerable people who experienced conflict have in Northern Syria. Going forward, the possibility of complementing the food security activities with some livelihood assistance and/or to develop a holistic package of interventions should be considered.

The programme was scored as between satisfactory and acceptable in terms of its connectedness. The approach chosen for the water rehabilitation component was mostly by direct implementation rather than WASH system strengthening, which may mean that the improvements achieved thanks to the programme may not be sustainable in the long run. In terms of the support provided under food security and emergency assistance, this was greatly appreciated and has had important immediate impacts on the lives of beneficiaries, but this type of support is not able to create a sustainable improvement in these households’ and individuals’ living conditions. The clear limitations of this programme approach, chosen to respond to save lives and respond to the immediate needs of a population living a triple crisis, are well-understood by Concern Syria, who is aware of the need of looking at more long-term, sustainable solutions going forward (should the conditions allow).

Last but not least, the programme was scored as satisfactory (3) in terms of its coherence, in that it is aligned to Concern’s internal policies as well as to the main international standards for both delivery strategies and standards for quality of assistance.

 

Overall, it is impressive to see how much Concern Syria has been able to achieve in the 12 months under this programme despite the massive challenges that occurred at both the local and the global level, that impacted every aspect of the day-to-day operations and that, despite this, did not affect Concern Syria’s capacity to deliver high quality life-saving assistance.

• 4 - Outstanding: outstanding performance (above expectations)

• 3 - Satisfactory: performance in line with what would be expected of a well-functioning organisation (in line with expectations)

• 2 - Acceptable: generally acceptable performance but with some clear, and documented, shortcomings

• 1 - Minimal: barely acceptable performance with some major shortcomings and reservations

• 0 - Unacceptable: totally unacceptable performance or insufficient data to make an assessment

This page covers humanitarian aid activities implemented with the financial assistance of the European Union, the European Civil Protection and Humanitarian Aid Operations Department (ECHO). The views expressed herein should not be taken, in any way, to reflect the official opinion of the European Union, and the European Commission is not responsible for any use that may be made of the information it contains.