Evaluator – Міжнародна організація з міграції (МОМ), Представництво в Україні

Evaluation Terms of Reference

Evaluator

“Enhancing Successful Reintegration of Veterans from the Conflict in Eastern Ukraine and their Family Members through Socio Economic Support” (FC.0198)

Commissioned by: International Organization for Migration (IOM), Ukraine

Managed by: Sarah Harris, Regional M&E Officer (IOM Regional Office in Vienna)

1. Evaluation context

The International Organization for Migration (IOM) is the lead UN agency in advancing migration policy and practice, addressing humanitarian and development challenges of migration, and using peacebuilding to prevent future displacement. IOM does this together with governmental, intergovernmental, civil society, and private sector partners through well-informed, coordinated and protection-centred actions to avoid duplicating efforts meanwhile overlooking critical gaps. IOM’s Mission in Ukraine was established in 1996, and IOM has supported several areas of work in Ukraine including emergency, transition and recovery; migrant protection and assistance; migration and border management; migration and development; and migration health services and travel assistance. IOM protects and assists those displaced or stranded by crisis and supports populations and their communities to recover. IOM works to mitigate adverse drivers that force people from their homes, help build resilience and reduce disaster risk so that movement and migration can be a choice. Currently, the Organization’s number one priority is the safety and protection of all those fleeing the country – including third country nationals – and their ability to access assistance.

The project to be evaluated ran from January 2021 to May 2022, thus impacted in its final months by the full-scale invasion of Ukraine by the Russian Federation that started on 24 February 2022. This is a Phase II project building on a previous Phase I project that ran from December 2018 until end of August 2020.

As of January 2020, according to the Ministry of Veterans Affairs (MinVet), a total of 378,115 registered veterans of the conflict in Eastern Ukraine (over 16,000 of whom were women) had returned to their homes, with many struggling to reintegrate into civilian life. These veterans found themselves facing significant challenges on account of their experiences on the battlefield and were often left without the services or support they need due to already overburdened government institutions and communities. Through support from the European Union (EU), more specifically through the Instrument contributing to Stability and Peace, the International Organization for Migration (IOM) implemented a community-based support project for veterans of the conflict and their families in three pilot regions (Kyiv, Dnipropetrovsk, and Lviv) that ran from December 2018 until end of August 2020 (Phase I project). As of 1 June 2020, 8,435 veterans and community members had benefited from livelihood, social cohesion, and mental health and psychosocial support (MHPSS) activities under the project.

In January 2020, as part of the Phase I project, IOM published a household survey on sociodemographic and socioeconomic characteristics of veterans and their families, and the difficulties veterans face in readjusting to civilian life. The survey found that veterans felt excluded from society and strongly identified themselves with their reference group. Almost half of respondents (49%) indicated that they had faced bias or unfair treatment (discrimination) directed at veterans in different spheres. In addition, nearly half (46%) had not returned to their previous workplace after military service for several reasons, including a change in their mindset and difficulties to perform tasks related to communication with others because of a perceived deterioration in relationships with colleagues. Further, over one-third (35%) articulated their interest to start their own business and requested the support of IOM to do so.

During Phase I, most veterans who participated in workshops on civilian employment and business development trainings emphasized their usefulness. Through pre- and post-training questionnaires administered during the workshops, veterans indicated that the skills gained, and tools developed during the sessions would definitely be helpful during daily communication with colleagues, as well as in business development and interaction with clients, thus reemphasizing the need to continue pursuing this type of assistance. On average, the level of awareness of civilian employment training participants increased by 63% and of business development training participants by 41%.

Veteran businesses have not been spared by the impact of the COVID-19 crisis, increasing veterans’ fragile situation. IOM sought to support its livelihood beneficiaries and consolidate gains made during the project by providing supplementary grants to at least 98 during a No-Cost Extension of Phase I. However, there was an identified need to further support such types of businesses beyond the Phase I.

Mental health and psychosocial support (MHPSS) is another area identified by IOM as key for ensuring veterans’ reintegration. While veterans are more reluctant to consider themselves in need of psychosocial support, over half (51%) indicated that counselling or individual consultations with a psychologist would be desirable for other veterans in general. However, 56% of surveyed respondents reported that veterans might neglect psychological support because they do not want to show their problems. Moreover, there is a lack of clinical psychologists and psychiatrists able to handle this more complex caseload, and a confusion over the referral networks and coordination between relevant service providers.

Participants of focus group discussions conducted within the IOM household level survey mentioned a need to provide individual consultations and trainings for veterans’ family members. Such measures are important to help veterans’ family members understand the mindset of a person who engaged in combat operations and build relations with them. Besides, during Phase I, there were requests from veterans and their family members to arrange family counselling. IOM was not able to fulfil such a request due to a low number of MHPSS workers able to provide such type of support, thus within Phase II, IOM ran a training on how to conduct family counselling and will organize four family retreats.

Finally, ongoing work of IOM with the Ministry of Veterans has revealed a need for further supporting the Ministry’s efforts to put in place and implement a strategic policy framework on protection of veterans’ rights protection. Such a need was articulated in recommendations of the legislative review undertaken under the ongoing project, as well as following feedback received from the Minister’s cabinet to IOM.

Within this context, the Phase II objective was to contribute to enhancing the Government of Ukraine’s efforts to ensure sustainable reintegration of veterans from the conflict in Eastern Ukraine through three outcomes: (1) Veterans and their families actively contribute to social and economic life in communities of return in Ukraine; (2) Improved psychosocial well-being of male and female veterans and their families during reintegration to peaceful civilian life; and (3) Ukrainian authorities plan strategically for assistance programming based on information on the situation of veterans and increased capacities.

The project aimed to achieve this through the following activities: provision of livelihood support, MHPSS services for veterans, community stabilization activities, and capacity development activities for service providers. The activities aimed to support an additional 7,910 (totalling 41,034) male and female veterans, support workers (non-combatants supporting veterans on the front), dependents, and widows and widowers of combatants in their transition to civilian life, as well as other conflict-affected groups.

2. Evaluation purpose

Reflections on evaluations conducted on earlier projects have informed the design and implementation of IOM Ukraine’s durable solutions programming. The implementation of the project was informed to an extent by the lessons learned from and recommendations provided by the evaluations of IOM Ukraine’s actions addressing veterans’ return to civilian life. Similarly, the purpose of this evaluation is to assess the relevance, efficiency, effectiveness, impact, sustainability, and coherence of the operations carried out under the project to feed into future project development and support related fundraising efforts for future phases of the community level durable solutions programme. It is also being conducted for use by IOM with the aim of informing design and implementation of future programmes and activities. Further, results may be used by stakeholders (government authorities, NGOs, etc.) to assess the relevance and accountability to intended beneficiaries, as well as by the donor to assess value for money.¹

3. Evaluation scope

The final external evaluation of the project Enhancing Successful Reintegration of Veterans from the

Conflict in Eastern Ukraine and their Family Members through Socio-Economic Support should cover:

• Full project period (1/5/2021-4/5/2022);

• Full geographic scope to the extent feasible in the current wartime realities (remotely with some limited field visits e.g., to Uzhhorod, Ivano-Frankivsk and/or Lviv areas);

• All groups of project participants, including beneficiaries and stakeholders (NGOs, international experts, local and regional government officials, government ministries, etc.)

Further, the evaluation should include explicit recommendations, including on approach towards needs assessment and project design, and good practices and lessons learned from the evaluated period.

The evaluation shall explicitly incorporate the following cross-cutting themes:

1. Protection mainstreaming: Inclusion of humanitarian protection principles into crisis response by ensuring that any response is provided in a way that avoids unintended negative effects (do no harm), is delivered according to needs, prioritizes safety and dignity, is grounded on participation and empowerment of local capacities, and ultimately holds humanitarian actors accountable vis-à-vis affected individuals and communities. IOM is committed to mainstreaming protection across all its humanitarian programming, as this aims to ensure safe programming. IOM incorporates the following four protection mainstreaming principles, which are fundamental to crisis and post-crisis response:

1. Prioritize safety and dignity and avoid causing harm
2. Secure meaningful access

3. Ensure accountability

4. Ensure participation and empowerment

The evaluation should assess the extent to which protection has been mainstreamed in the overall design and in the achievement of outputs and outcomes (effectiveness). Integrating protection mainstreaming considerations in evaluations means also ensuring that evaluations are conducted using a participatory and inclusive approach (e.g., sex and age diversity during implementation). For emergency and protracted crises response projects, evaluations might also assess impact, sustainability, and enhanced capacities of local partners, affected individuals and communities.

1. With full respect to the principle of intentionality in evaluations, which means that evaluations should only be undertaken if there is a clear intent to use the evaluation findings. See UNEG, Norms and Standards for Evaluation (2016).

2. Gender mainstreaming: The process of assessing the implications of any planned action, including legislation, policies, and programmes, for people of different gender groups, in all areas and at all levels. It is an approach for making everyone’s concerns and experiences an integral dimension of the design, implementation, monitoring, and evaluation of interventions in all political, economic, and societal spheres so that all gender groups benefit equally, and inequality is not perpetuated.

IOM aims to promote gender equality and ensure that all beneficiaries and populations assisted are receiving needed services and support, taking into consideration gender-specific experiences, so that interventions do not perpetuate gender inequalities. The following are a few points to ensure gender mainstreaming and to monitor it within an intervention as a cross-cutting theme:

1. Ensure that interventions address different needs (and capacities) of a diverse beneficiary population, with an aim to eliminate gender disparities and contribute to gender equality.

2. Assess how well an intervention captures gender perspectives, including by using gender-sensitive indicators, which are disaggregated by sex, as well as indicators that measure gender-specific changes, such as prevalence of gender-based violence or perceptions of gender norms, roles and relations.

3. Ensure that progress on gender-sensitive indicators is monitored regularly and adapted, as needed, to ensure that all intended beneficiaries are covered.

The evaluation should assess the extent to which gender has been mainstreamed throughout the action, in line with IOM’s Guidance for Addressing Gender in Evaluations. Consequently, during data collection, the evaluator shall ensure that persons being interviewed or surveyed are diverse and gender-representative of all concerned project partners and beneficiaries. Surveys, interview questions and other data collection instruments should include gender issues. Evaluation reports should firmly incorporate a gender perspective, such as analysis of sex-disaggregated data and attention to project effects and impacts related to gender equality.

3. Disability inclusion: IOM interventions must ensure that activities address barriers that prevent persons with disabilities in all their diversity from participating in, or having access to, services and/or protection, in line with the Convention of the Rights of Persons with Disabilities (CRPD). The CRPD defines persons with disabilities as those who have long-term sensory, physical, psychosocial, intellectual, or other impairments that, in interaction with various barriers, may hinder their full and effective participation in society on an equal basis with others. It affirms that States Parties must protect and promote the rights of persons with disabilities in their laws, policies, and practices; and must also comply with the treaty’s standards when they engage in international cooperation. The following are a few simple principles of disability inclusion which may be evaluated as a cross-cutting theme:

1. Promoting meaningful participation of persons with disabilities.

2. Addressing the barriers faced by persons with disabilities.

3. Empowering persons with disabilities and supports them to develop their capacities.

The project evaluation should cover different aspects of disability inclusion. Evaluation questions mainstreamed across the different evaluation criteria or under a specific criterion shows the extent and the quality of disability inclusion. Evaluation stakeholder mapping and data collection methods should involve persons with disabilities and their representative organizations. Persons with disabilities and organizations of persons with disabilities can enrich evaluation by providing first-hand information on their situation and experience. Evaluation should also examine if barriers have been removed to allow full participation of persons with disabilities. It can also include long-term impact analysis on the lives of persons with disabilities and the recognition of their rights according to international standards.

The above themes are to be intentionally incorporated within the evaluation methodology, including in assessing individual evaluation criteria, selection of respondents, and data analysis.

4. Evaluation criteria

The project will be assessed according to all six of the OECD-DAC (2019)² evaluation criteria: relevance, coherence, efficiency, effectiveness, impact, and sustainability.

5. Evaluation questions

The table below specifies questions for each of the criteria, which shall be answered by the evaluation.

Revisions may be proposed by the evaluator during the inception phase.

To the extent possible, for each criterion, the evaluator should assess conditions before and after the full-scale invasion of Ukraine by the Russian Federation on 24 February 2022, considering the impacts of wartime realities during the final months of the project. This includes any shifts in needs and alignment to other projects (relevance, coherence), performance and results (efficiency, effectiveness), and impact and sustainability prospects.

2. The OECD/DAC adapted definitions and principles for use, along with an explanatory brochure, can be found at OECD, n.d. as well as summarized in the IOM M&E Guidelines (pp. 220-22

19. Do the partners benefiting from the intervention have adequate capacities (technical, financial, and managerial) for ensuring that the benefits are retained in the long run, and are they committed to do so?

20. To what extent have target groups, and possibly other relevant interest groups and stakeholders, been involved in discussions about sustainability?

6. Evaluation methodology

The final evaluation will be conducted in line with IOM policies and guidance on M&E, as outlined in the IOM M&E Guidelines. Final methodology will be proposed by the evaluator and confirmed following discussions with the project team and evaluation manager.

IOM suggests a mixed methods approach utilizing secondary and primary sources, and both qualitative and quantitative approaches to ensure triangulation. Participative techniques are also envisioned, as feasible. Suggested appropriate methods include:

• Desk review of project documentation, including narrative donor updates, mid-point project review report, and monitoring data and reports.

• Interviews, focus group discussions, and/or surveys with project team, implementation partners and stakeholders including beneficiaries, community members, MHPSS workers, members of Community Support Groups (CSG), CSO representatives, local authorities, government officials.

The evaluation will for the most part be done remotely, with travel to up to three communities depending on the security situation (e.g., Uzhhorod, Ivano-Frankivsk and/or Lviv areas).

7. Ethics, norms, and standards for evaluation

IOM abides by the Norms and Standards of UNEG and expects all evaluation stakeholders and the consultant(s) to be familiar and compliant with the UNEG Ethical Guidelines for Evaluation, as well as the UNEG Codes of Conduct for Evaluation in the UN System.

The evaluation must be conducted in full respect of IOM Data Protection Principles.

8. Evaluation deliverables

The following deliverables are to be provided by the evaluator throughout the evaluation process, upon a pre-agreed schedule confirmed in the inception phase (see below section on time schedule):

• Inception report (including evaluation matrix and final methodology)³
• Draft evaluation report submitted for comments to evaluation manager

• Debrief on initial field findings (PowerPoint)

3. See for reference the IOM inception report template and IOM sample example evaluation matrices.

• Final evaluation report consisting of at least 20 pages (ideally 20-30, max 50) without annexes⁴
• Evaluation brief (two-page summary)⁵
• Presentation of the evaluation findings to key stakeholders
• Management response matrix (draft, partially filled out)⁶
• Brief progress report(s) during the evaluation depending on needs

9. Specifications of roles

The evaluator’s primary focal point for the evaluation will be Sarah HARRIS, Regional M&E Officer in IOM’s Regional Office in Vienna, who shall act as lead evaluation manager in coordination with the project manager (Alissa LALIME) and IM/MEL manager for E&S at IOM Ukraine (Karolina KRELINOVA).

Further, following project team members will be available to support the final evaluation (TBC during the inception phase considering the workload and burden on staff in the context of wartime realities):

A quality assurance process shall be integrated into the evaluation, as follows:

1. Final terms of reference (ToR) shall be agreed between evaluation manager and evaluator.

2. The inception report shall be reviewed by the evaluation manager and project team, revised by the evaluator, and finalized only upon evaluation manager’s acceptance.

3. The evaluator will timely raise any emergent challenges or barriers with the evaluation manager, who will support with finding appropriate solutions to facilitate the evaluation process.

4. The final report and brief shall be reviewed by the evaluation manager and project team, revised by the evaluator, and finalized only upon evaluation manager’s acceptance.

4. Though IOM does not oblige evaluators to use the same reporting format, evaluators are expected to address all components outlined in the IOM components template and template for evaluation final report as per the IOM M&E Guidelines (see p. 237).

5. An IOM template for the brief will be provided by IOM developed on Microsoft Publisher. The brief should provide a short (two pages) overview of the evaluation including key project information, findings, conclusions, and recommendations.

6. Using the IOM template for Management Response and Follow-up.

10. Time schedule

An indicative schedule for the evaluation is provided below, to be finalized upon agreement with the evaluator and approval of the Evaluation Manager (EM) during the inception phase. The timeline assumes start date of 1 August 2022 and anticipates the evaluator’s commitment for 25 full working days.

11. Evaluation requirements

The external evaluator should meet the following requirements:

• Advanced degree in a relevant field such as social sciences or statistics is required.

• Minimum of five years of experience conducting evaluations is required.

• Experience in the country or region, in community stabilization and livelihoods is preferred.

• Skills in evaluation design, qualitative data collection and analysis, drafting and editing in English, communication, time management and cultural sensitivity are required.

12. Submission of application

Interested evaluators are invited to submit the following materials to [email protected], referring to “Final External Evaluation – FC.0198” in the subject line of your message:

• CV of the evaluator

• References – names and contacts of three referees

• Technical proposal – outlining proposed evaluation methodology including data collection plans and analysis techniques, quality control measures, and timelines.

• Financial proposal – all-inclusive budget for the evaluation, to include expert fees and all other expenses to be incurred.

Any questions should be addressed to the e-mail mentioned above, with the Evaluation Manager in copy (Sarah Harris, [email protected]).

• Kindly note that private entrepreneurs (PE) cannot be considered as applicable for providing services under this consultancy. A consultancy contract (as an individual, not as a business entity) will be established between IOM and the evaluation consultant.

Deadline for applications: 11 July 2022