9.8 Operational Peer Reviews and Inter-Agency Humanitarian Evaluations
  • 06 Dec 2023
  • 12 Minutes to read
  • Dark
    Light

9.8 Operational Peer Reviews and Inter-Agency Humanitarian Evaluations

  • Dark
    Light

Article Summary

Operational Peer Review     

An Operational Peer Review (OPR) is an internal inter-agency management tool which identifies areas for improvement in a crisis no later than 5 months after a Humanitarian Scale-Up Activation (this is a formal mechanism for the mobilisation of system-wide capacities and resources beyond standard levels - see more in 1.2.5). For other types of humanitarian responses, an OPR may be triggered by a request from the RC/HC and HCT, the IASC Emergency Directors Group or the IASC Principals.

It is designed to be a light, brief and collaborative process, undertaken by a team of senior humanitarian practitioners from the UN, NGOs and other IASC organisations. The review aims to help determine whether adjustments (or “course correctors”) need to be made to the collective humanitarian response, and OPR recommendations provide the RC/HC and HCT with an opportunity to (in real time) reflect on the direction and performance of a humanitarian response. They also support key strategic decisions for the response, such as whether to extend the Scale-Up at the end of the activation period. OPRs can also help to identify good practice or learning to share with other operations (on leadership arrangements; key operational or coordination mechanism obstacles and implementation of the HPC). OPR Reports are internal to the humanitarian system.  

Inter-Agency Humanitarian Evaluation

An inter-agency humanitarian evaluation (IAHE) is an independent assessment of results of the collective humanitarian response by IASC partners. IAHEs evaluate the extent to which planned collective results have been achieved i.e. if the HRP objectives and needs of the affected population have been met (and how humanitarian reform efforts have contributed to that achievement). They aim to contribute to accountability and strategic learning for the humanitarian system. They also seek to promote AAP through the provision of feedback on the results of the response to affected communities.

In case of a crisis that required a Scale-Up Activation, IAHEs needs to be conducted within 9-12 months – and will take the findings of the OPR into consideration. For all other cases, there is no fixed time frame for when an IAHE should be conducted, as a particular emergency situation may require flexibility for an evaluation process. Usually, IAHEs are formally launched by the ERC. Using a systematic and defined set of selection and prioritization criteria, a list of priority countries for crisis-specific evaluations and priority themes for thematic evaluations are established on an annual basis. 

IAHEs are conducted by teams of independent evaluation experts who focus on the quality of aid delivered; the degree to which affected people have been protected; and progress towards both the response’s objectives and targets set by the RC/HC and HCT. IAHE results are available to the public (unlike OPRs).

What is the role of the FSC Coordinator? When required, the Coordinator will participate in the OPR and IAHE processes as established at country level. This usually involves ICCG meetings or workshops with the OPR/evaluation teams as well as provision of written cluster specific inputs where relevant. Beyond this, it is usually the CLAs and senior NGO representatives that actively participates. 

The OPR will include site visits, key informant interviews, an HCT workshop and self-assessment exercises in-country whereas for IAHEs the Coordinator may contribute to the IAHE preparation and evaluation field mission phases. 

For IAHEs recommendations, an in-country advisory group is appointed by the RC/HC to develop a management response plan responding to these. The advisory group is not at coordinator level however, FSC Coordinators may, as relevant, support the implementation of this response plan. OCHA will inform the ICCG on this. 

Similarly, for OPR recommendations the Coordinator may be required to support the implementation of recommended actions where relevant.

Guidance and Resources:

ANNEX I: FSC Standard Operating Procedures – Deployment of FSC Coordinator to the Country FSC 

The FSC Standard Operating Procedures (SOPs) (see the FSC website) were developed to provide greater clarity and guidance to those assigned to lead and manage an FSC in countries with humanitarian crises. The SOPs include quick guidance on what to do and achieve once the cluster is activated and should be seen as complementary to other existing tools (including this handbook). These SOPs describe concisely the main deliverables, key actions and tasks that CLAs, FSC Coordinators and IMOs should develop during the first days and weeks of any new humanitarian crisis in a country. 

Note: These SOPs are currently under review – please check with GST for updates.

ANNEX II: HPC Steps and Timeline  

Note: For protracted crises, detailed step-by-step guidance on the HPC is produced yearly and Coordinators are encouraged to check for the most recent version for further details.

However, a typical HRP process will commonly include the following steps – all with the active participation of the FSC Coordinator:

Step 1: [HNO Planning Phase] Agree on scope of the analysis – JUNE 

  • Prior to initiating the HNO, the parameters of the analysis will be agreed at ICCG inputs and HCT level, including deciding on a costing methodology. A detailed country specific timeline outlining the different milestones for implementation of the HPC should be provided at the country level). This is commonly done through a HPC kick-off workshop.  

Note: There should be a clear justification for any changes in the costing methods. This should be discussed with FSC partners. All clusters should then report and discuss this with OCHA and the ICCG. Once the calendar is set, the FSC coordinator elaborates the FSC timeline and informs the partners about the milestones.

Step 2: [Data Consolidation Phase] Undertake secondary data review: analyse trends, identify opportunities for joint analysis with development/peace actors, and identify data gaps – JULY / AUGUST

  • Clusters undertake secondary data review, including the analysing trends and identifying critical data gaps. Next, to better serve the information requirements, the ICCG (or the Assessment and Analysis Working Group), with HCT guidance, will decide whether a joint needs assessment (such as a MSNA) is required, or harmonized in-depth sectoral assessments.

Note: See 6.5.3 for details on when and how the FSC should support MSNAs.

Step 3: Plan and collect primary data (as appropriate) – JULY / AUGUST

  • Clusters may undertake a joint needs assessment (MSNA or FSA, as required by the context). According to the HPC ten steps, this will be undertaken during July and August, however, the timeline may have to be adjusted (initiated earlier if possible) to accommodate the time requirement for this type of exercise in the specific context. For MSNAs, this is discussed at ICCG level and for FSAs, this is discussed at FSC level with partners. 

Note: See 6.5.3 on MSNAs and 6.4 on FSAs for details on the role of the Coordinator. 

Step 4: Conduct joint intersectoral needs analysis – JULY/AUGUST/SEPTEMBER

  • A joint intersectoral analysis is undertaken, and the intersectoral narrative, needs and severity analysis is drafted (see text box on JIAF in 9.4.1). 

Note: During this period, the FSC HNO chapter should be drafted, and sectoral PIN calculated. A detailed country specific timeline should be provided at the country level. The Coordinator should advocate for the timeline to be aligned with FSC processes such as the IPC.  

Step 5: [HRP Planning Phase] Define the scope of the HRP and formulate initial objectives – AUGUST/SEPTEMBER

  • This step focusses on inter-sectoral process.

Note: Ensure that food security is as visible as possible in the overall strategic objectives. 

Step 6: [Response Analysis Phase] Conduct response analysis – SEPTEMBER/OCTOBER

  • Review appropriateness, relevance, and feasibility of different responses.
  • Articulate intersectoral and multi-sectoral response approaches based on the results from the response analysis (based on severity, time-criticality, and complementarities/ synergies).
  • Estimate target population number.

Step 7: [Response Planning Phase] Finalize strategic and specific objectives and indicators - OCTOBER

  • Finalize formulation of strategic and specific objectives and identify indicators to monitor specific cluster objectives.
  • Clusters develop response plans and define cluster objectives (activities, indicators, targets).
  • Sub-national and/or government consultation/review draft HRP response parameters.
  • Present and seek endorsement by the HCT of the strategic objective and approach, number of people targeted, and response monitoring framework.

Step 8: Formulate projects/activities and estimate cost of the response plan - NOVEMBER

  • Initiate drafting of HRP (if Project based: project development, vetting and upload).
  • Estimate the cost of the response.
  • Secure HC/HCT endorsement.
  • Finalize and draft response plan.

Note: Formulating activities and estimating cost takes time (usually around 3-5 months). This is crucial to factor in if the unit costs for food security activities are not yet available.

Step 9: Conduct After Action Review – DECEMBER / JANUARY 

  • The ICCG (with OCHA facilitating) discusses strengths and weaknesses from the finalised process to streamline for coming year.

Note: The participation of the FSC team is crucial to ensure that any concerns are raised in appropriate fora. See here for more on the After Action Reviews of the Humanitarian Programme Cycle. 

Step 10: Finalize and implement monitoring plan – JANUARY/FEBRUARY 

  • Prepare the monitoring plan.
  • Conduct monitoring activities throughout the year.
  • Share information gathered by the monitoring work.

For more details see the HPC 2023 Facilitation Package. Coordinators should check for updated versions

Annex III: Examples of FSC Targeting 

The GST is currently developing a summary of HRP targeting examples (gFSC, Placeholder, 2023 – please check with the GST for updates).   

ANNEX IV HRP Costing Options – Pros and Cons

Options
Pros
Cons
Project based costing
  • Provides overview of response capacity, and early identification of possible gaps and duplications.
  • Allows financial tracking of response (projects requirements vs funding received in FTS). 
  • Visibility to partners (appealing agencies listed in HRP and FTS).
  • Match between partners’ financial asks and HRP costing.
  • Possibility to prioritize projects during an emergency.
  • Possibility to link directly to humanitarian pooled funds (facilitating submission process for partners).
  • Provides higher transparency of process through the project Review Committee that would include various actors (including NNGOs).
  • Allows the FSC teams to better understand partners' capacity/strategies.
  • Overall cluster/sector financial requirements reflect capacities of appealing agencies, which can be much lower than needs in some cases (though this could be considered appropriate to avoid unrealistic plans).
  • Disconnect between targets and financial requirements if former is not defined based on submitted projects.
  • Partners have to submit one (or more) projects to be part of the HRP , and the process takes longer time because it includes projects' submission/revision/clearance. Some partners may not be willing to engage / be excluded if timing not convenient / they do not see benefit of it 3102395-200this can lead to an underestimated plan
  • Information (“transparency”) on cost breakdown could be less visible 3102395-200 this can lead to a risk of budget overinflating or under estimation (impacting quality standards). The FSC needs to check information / ensure agency capacity.
  • Lengthy process to review all projects – vetting done with FSC committee when possible.
Unit based costing
  • Financial requirement is directly linked to needs.
  • Allows partners to focus on strategic decisions of the planning process.
  • Once unit costs are calculated, easy to put in place, e.g. also for revisions / rapid onset emergencies
  • FSC partners do not have to submit project sheets, nor be involved in lengthy vetting process
  • Lengthy process to get full endorsement from partners on set of activities and average costs which might require regular review and updating (as a result of fluctuating needs and/or prices).
  • Partners may not be willing to share their unit costs.
  • Support costs can vary considerably between geographic areas but also due to agency capacity.
  • Costing of complementary activities may be overlooked 3102395-200this can have an impact on programme quality.
  • High difference in costing among agencies (UN vs INGOs vs NNGOs) 3102395-200i.e. partners’ financial asks may not be fully reflected in HRP costing.
  • Certain level of funding may not translate into the expected levels of achievements if actual costs of operations differ from HRP costing.
  • Inability to conduct detailed financial tracking at project / partner level in FTS (as no projects are submitted on HPC).
  • Reduced partners’ visibility (no list of partners in HRP and FTS).
  • Less information for coordination team on planned interventions and potential gaps / duplications.
  • Less direct engagement of partners with clusters (more with donors).
Mixed/Hybrid approach

Same as unit based:

  • Financial requirement is directly linked to needs.
  • Allows partners to focus on strategic decisions of the planning process. 
  • Once unit costs are calculated, easy to put in place, e.g. also for revisions / rapid onset emergencies.

Same as project based:

  • Provides overview of response capacity, and early identification of possible gaps and duplications.
Visibility to partners.

Same as unit based:

  • Lengthy process to get full endorsement from partners on set of activities and average costs.
  • Difference in costing among agencies (UN vs INGOs vs NNGOs) 3102395-200i.e. partners’ financial asks may not be fully reflected in HRP costing.
  • Inability to conduct detailed financial tracking at project / partner level in FTS.



Was this article helpful?