Emergency Response Meta Evaluation 2017 - 2019

This report is the continuation of a process of organisational learning from Concern’s emergency responses that has seen five meta-evaluations carried out:

1. April 2001, covering the period 1990 to 2000
2. April 2005, covering the period 2000 to 2004
3. July 2009, covering the period 2005 to 2008
4. August 2013, covering the period 2009 to 2012
5. March 2018, covering the period 2013 to 2016

Thirty-four evaluations were reviewed and analysed as part of this meta evaluation. Evaluations were reviewed from programmes delivered in the following countries: Afghanistan, Bangladesh, CAR, DRC, Ethiopia, Iraq, Kenya, Lebanon, Malawi, Mozambique, Nepal, Pakistan, Sierra Leone, Somalia, South Sudan, Syria, Turkey, Yemen, and Zambia. Overall, this represents a greater number and better balance of evaluations than was evident in previous meta evaluations.

Relevance: Was the most commonly used criterion and all of the interventions evaluated were considered appropriate and relevant. They addressed key humanitarian needs and, where necessary, showed flexibility and a willingness to adapt. This is particularly important when responding to programmes in both sudden-onset and protracted crises, where accessibility can be constrained and programmes can be required to expand into new areas as more communities become accessible.

Coherence: Despite not being consistently used in the evaluations reviewed, overall there was satisfactory coherence shown in Concern’s programmes. The majority of the programmes reviewed against this criterion adhered to Concern’s Approach to Emergencies paper and reported that staff have been trained on protection, the Core Humanitarian Standard (CHS), and the humanitarian principles.

Coverage: Overall, Concern’s emergency responses are considered satisfactory in terms of coverage. The majority of the projects evaluated were implemented in areas with levels of vulnerability and a relatively low presence of humanitarian actors. Insecurity is a significant factor that needs to be taken into account but, as highlighted in the meta-evaluation of the HPP programmes, Concern has been courageous in selecting areas of operations and has measures in place to help to ensure the security of its staff. In line with its mandate and strategy, Concern has demonstrated a commitment to reaching those most in need in the hardest to reach areas, with particular attention given to the most vulnerable, including children, people with disabilities, and women.

Efficiency: Overall, Concern’s emergency responses are considered satisfactory in terms of efficiency. The majority of the projects were considered to be cost-effective. Logistics and procurement systems were found to be strong, and while some delays were reported, these were mainly due to external factors linked to the difficult operational environments in which Concern works.

Effectiveness: Overall, the majority of the programmes evaluated were timely and effective in reaching their targets. In numerous cases, information from assessments and M&E activities were used to make adjustments. This is particularly important considering the volatile nature of emergencies, particularly in contexts of protracted crisis, but also in the aftermath of a natural disaster where information from rapid assessments is constantly reviewed and updated.

Impact: The majority of the evaluations reported the responses to have had a positive impact on beneficiaries. The increase in the level of employment among beneficiaries as a direct result of the projects implemented has been highlighted. A number of projects implementing cash programming have been considered to have boosted economic empowerment in the communities in which the projects were implemented.

Connectedness and Sustainability: Consistent with the extended DAC criteria, the Approach to Emergency paper and the Evaluating Concern’s Emergency Responses guidelines recommend the use of connectedness rather than sustainability, but the evaluations reviewed presented a mix of the two. In terms of connectedness, the overall analysis changes, and the assessment leans towards the majority of the evaluations being considered Satisfactory. A number of examples of the introduction of medium-term thinking in the emergency response implemented were reported in the evaluations reviewed.

The Executive Summary and Full Meta Evaluation Reports can be downloaded below: