Evaluator – Представництво МОМ в Україні

Background of the Consultancy

1. Evaluation context

The IOM Mission in Ukraine was established in 1996, when Ukraine became an observer state of IOM. In 2001, Ukraine requested membership in IOM, which was formalised with the Ukrainian Parliament’s ratification of the IOM Constitution in 2002. In line with IOM’s global strategy, IOM Ukraine aims to advance the understanding of the opportunities and challenges of migration in the Ukrainian context. Maximising those opportunities and minimising the challenges presented by migratory movements are the guiding principles of all activities and programmes the Mission is engaged in.

The IOM Mission in Ukraine provides assistance to internally displaced persons (IDPs) and conflict-affected people, fights trafficking in human beings, assists the Government in dealing with irregular migration, improving its migration management system, and creating migrant-inclusive health practices and policies. At the same time, IOM Ukraine engages in harnessing the development potential of migration, disseminating migration information, supporting integration of migrants, promoting the benefits of cultural diversity, and counteracting xenophobia and intolerance. During the years of presence in Ukraine, IOM has assisted over half a million migrants (Ukrainians and other nationalities), IDPs, potential migrants, victims of trafficking and other vulnerable groups, directly or through its project partners.

Project background

In 2019, approximately 280,000 youth left Ukraine to seek better opportunities for themselves and their families. This consistent outward migration has left the civic and social lives of communities of origin drained and with a lack of youth in public engagement. This weakens the social fabric as well as the communities’ capacities to withstand current and future socioeconomic disruptions. In stark contrast to this, the COVID-19 pandemic drastically decreased the outflow of Ukrainian youth in 2020 and also triggered the return of many young migrants to their places of origin in Ukraine due to the real or perceived threat of travel restrictions and the economic downturn which decreased labour demand in many countries of destination. While these contradictory migration trends represent a unique opportunity for the country to invest in the returned youth and actively involve them in society, it simultaneously has the potential to further increase social dissatisfaction and local tensions as eager youth are reintroduced into communities and labour markets that are already struggling with the economic recession and strained social services. In response, IOM implemented a pilot and now a second project, both funded by UK Foreign, Commonwealth and Development Office (FCDO).

During the pilot phase of “Engage, Connect, Empower”^[1], IOM observed first-hand the challenges that these circumstances have placed not only on youth and their communities, but also on the Government of Ukraine at central, regional, and community levels. As learned through direct programming, the challenges do not stop at youth engagement at the community level. Youth must also have complementary and collaborative exchanges with authorities, youth-focused organisations, and the private sector to enhance the sustainability of their community-based initiatives. Furthermore, it was clear that inter-community and inter-regional interactions between youth are a key mechanism for reinforcing their motivation and expanding their expertise in strengthening their communities. The best practices that were identified during the final internal evaluation of the previous phase, were already incorporated into the project documents of the ongoing project proposal and so far was integrated into the project activities.

In line with the approach adopted for the pilot phase, IOM intends to continue building upon the assessment provided by the Ministry of Youth and Sports of Ukraine that identified primary factors contributing to the general disengagement of youth from public life: 1) lack of engagement and social skills; 2) lack of access to decision-making; and 3) lack of access and resources to promote youth agendas at the community level. Through the pilot phase, IOM found that seeking to positively address these issues led to increased willingness by youth to actively engage in community life. In the second phase, IOM seeks to gain a stronger evidence base for this observation through strengthened research and learning components. Moreover, the approach is in line with the newly issued National Youth Strategy till 2030, adopted by Presidential Order in March 2021, particularly with its core principle of participation–where any youth-related decisions are made with youth involvement. In addition, the IOM approach is linked with the Strategy’s key priority of youth capacity building by involvement of young people in public life, increasing their independence, competitiveness, and developing their civic competencies.

Based on these lessons learned from the first phase of the ECE project, within the current project IOM aim to: a) increase the emphasis on engagement of youth-focused organisations and authorities to strengthen the whole-of-society approach; b) increase the youth-to-youth and the youth to-government interactions; and c) to expand the geographic coverage to an additional three oblasts (Odesa, Mykolaiv, Kherson) which allows for increased interregional learning and cooperation.

Project description

The overall objective is to strengthen Ukrainian youth as forces of cohesion and resiliency by supporting them to bring about positive changes in their communities. To contribute towards that objective, IOM proposed a three-pronged intervention as detailed below:

• Outcome 1: Youth, youth-focused organisations, and authorities are engaged for community-based initiatives. Activities leading to Outcome 1 focus on building the capacities of key stakeholders so that they improve the understanding of each other and feel more engaged for community-based initiatives. Under this component IOM will provide tailored training to members of youth initiative groups identified through a competitive call for expression of interest. Initiative groups from each region will be given the opportunity to put what they have learnt into practice by coming up with a region-level advocacy campaign design, to be implemented through a small grant provided by the project. In parallel, in consultation with the Ministry of Youth and Sports and youth-focused organisations, IOM will run a youth research on the social, psychological, economic and cultural perspectives of youth, their views and attitudes towards engagement, cohesion and resilience, as well as the youth infrastructure and its profiling, which will inform policies, IOM’s interventions, and boost the youth engagement agenda in Ukraine.
• Outcome 2: Youth, youth-focused organisations, and authorities are constructively connected across Ukraine. Activities leading to Outcome 2 focus on bringing together and facilitating exchanges between key stakeholders to promote collaboration. Under this component IOM will select, among the members of each initiative groups, a small number of youth who have demonstrated particular motivation and leadership skills and will involve them in exchange tours around all the regions targeted by the project, to share and experience best practices on youth-led community initiatives.
• Outcome 3: Youth are empowered to be drivers of change in their communities. Activities leading to Outcome 3 focus on supporting youth’s influence on community policy and fostering youth-led partnerships with various social stakeholders. Under this component IOM will continue supporting youth empowerment by organizing community-level youth consultations on the development of environmental protection agendas. The results will feed further consultation workshops between the youth and local authorities to identify potential areas of collaboration in policy development, as well as actions proposed by the youth that could possibly be incorporated in the local development plans.

2. Evaluation purpose and objective

The purpose of the evaluation is to look at the project’s performance, to identify gaps and aspects of project design and implementation that could be improved till the project end date, and to find ways to ensure sustainability of the project activities as well as find evidence that demonstrates change from the delivery of the projects interventions. IOM senior management and IOM project, procurement and finance teams in the Mission will use the evaluation to better understand the effectiveness, efficiency and relevance of the project activities and to amend the project activities in line with the identified findings.

3. Evaluation scope

The midterm evaluation will cover the entire period of project implementation, from 1 April 2021 to 31 September 2021. The interviews will be carried out with the project implementer (IOM), project partners (Ministry of Youth and Sports, youth centres in each of the six targeted oblasts, namely, Volyn, Zakarpattya, Ivano-Frankivsk, Odesa, Mykolaiv and Kherson regions; youth organizations, etc.), beneficiaries (Ministry of Youth and Sports, members of the youth initiative groups, workers of the youth centres and youth-based NGOs and community members), donor and other project stakeholders (UN agencies, INGOs and NGOs). Face-to-face interviews and focus group discussions will take place in the targeted regions if the epidemiological situation is favourable and/or via remote communication tools, including but not limited to phone, email, video and web conferencing et al.

4. Evaluation criteria

The midterm evaluation will use the following main OECD-DAC criteria: relevance, coherence, effectiveness, efficiency, impact, and sustainability. In addition, this evaluation will consider cross-cutting issues of gender, disability inclusion and environmental protection.

5. Evaluation questions

Relevance:

• Does the project still respond to the needs of the target groups/stakeholders, including beneficiary needs and strategic priorities of the government? And is it delivering what needs to be delivered?
• Do the project activities and outputs take into account relevant policies and guidelines?
• Were recommendations and lessons learned presented during the final evaluation of the previous project taken into account during the implementation of the current project?

Coherence:

• Do synergies exist with other interventions carried out by IOM as well as intervention partners?
• To what extent does the intervention add value/avoid duplication in the current context?

Effectiveness:

• Are the target beneficiaries satisfied with the activities implemented within the project so far?
• Is the intervention achieving its intended results?

Efficiency:

• Were the project activities undertaken and were the outputs delivered on time?
• How well are resources (funds, expertise, and time) being used?

Impact:

• What difference or change does the intervention contribute to, or is expected to contribute to, whether positive or negative, intended or unintended?
• What other related factors can be identified that may support or undermine those changes?
• Should other activities, if any, be implemented to maximize the impact of the ongoing project?

Sustainability:

• Has an exit strategy been developed to promote sustained project benefits, is it being implemented and monitored, and is there regular stakeholder consultation and communication? If not, what specifically should be considered during the rest of the project implementation period?
• Is the project supported by local institutions and well-integrated into local social and cultural structures?
• Are there already any indications about the likelihood of sustained project benefits, such as plans by the target groups to continue making use of the services produced?

Cross-cutting issues:

• Did girls, boys, women and men equally benefit from the project? What are/were some of the barriers to meaningful participation and what has been or will be done to address these barriers?
• Does the intervention identify enablers that facilitate the participation of persons with disabilities? What appropriate measures were taken to remove barriers and promote enablers, to ensure that persons with disabilities can participate meaningfully? What more should be done, if anything, to ensure the participation of persons with disabilities?
• Did the intervention bring relevant benefits and innovation for environmental sensitivity and sustainability? If not, what more should be done?

6. Evaluation methodology

The midterm evaluation will be conducted by an IOM staff on IOM’s global roster of trained evaluators. The evaluation should be conducted in line with IOM M&E policies, guidelines, and procedures  as outlined in the IOM M&E Guidelines and the IOM Project Handbook, consisting of three main stages:

• Inception Phase, including desk review – project documents, including monthly, quarterly narrative and financial reports with relevant annexes, and other updates to the donor, monitoring plan and tools, including surveys and field reports, and other relevant documents such as evaluation report for the previous phase. Drafting inception report, including detailed evaluation methodology, timeline, evaluation matrix, and data collection tools, and developing data collection tools (i.e. KII checklists and short questionnaires);
• Data collection among project stakeholders, project partners, local authorities and community members, beneficiaries and project staff through combination of field visits and/or virtual/e-communication tools[2];
• Synthesis Phase in which the results are compiled into the evaluation report, along with an accompanying learning brief to summarize the main findings and recommendations.

7. Ethics, norms and standards for evaluation

IOM abides by the norms and standards of the UN Evaluation Group (UNEG) and expects all evaluation stakeholders to be familiar with the ethical conduct guidelines of UNEG, and the evaluator with the UNEG code of conduct as well. The evaluation must also be conducted according to IOM Data Protection Principles.

8. Evaluation deliverables

• Inception report with an evaluation matrix and a draft interview guide, along with any other needed data collection tools, to describe the evaluator’s understanding of the TOR and how he/she will conduct the evaluation used including any revisions to the methodology (at minimum it should include the evaluation matrix and tools; evaluator may choose to develop other aspects of inception report per the IOM template as needed). This should be submitted to the evaluation manager, for comments and discussion, to finalize plans prior to data collection activities;
• Based on findings received and discussions with the project team during the presentation of preliminary findings, a draft evaluation report should be shared for the Project Manager’s feedback;
• Once comments received are addressed, a final evaluation report that includes an executive summary, list of acronyms, introduction, evaluation context and purpose, evaluation framework and methodology, findings, conclusions and recommendations should be submitted to IOM. The length of the final evaluation report should be approximately 20 pages excluding annexes;
• Together with the final evaluation report a two-page Evaluation Brief with key findings, conclusions and recommendations should be provided. IOM will provide a template as guidance, which can be adapted by the evaluator, but which should be no longer than two pages. Page one should include: Identification of audience at start of the brief; Project information (project title, countries covered, project type and code, project duration, project period, donor(s), and budget); Evaluation background (purpose, team, timeframe, type of evaluation, and methodology); Brief description of the project. Page two should summarize the evaluation results: Key findings and/or conclusions, best practices and lessons learned (optional), and key recommendation;
• In line with the project proposal the key findings of the evaluation to be presented to the project stakeholders during the mid-term review. The evaluator should prepare a PowerPoint presentation with a particular focus on the findings, conclusions and recommendations. If possible, the evaluator will be asked to present findings to the audience either be among the participants offline event (if the COVID-19 restrictions will allow) or join the event via Zoom/Teams; and
• Finally, the Evaluator should prepare a draft management response matrix by inserting recommendations as well as an indicative timeframe or deadline for implementation, using the IOM template for management response. The Evaluation Manager will then be responsible for finalizing the matrix after the conclusion of the evaluation.

All deliverables are to be written in English and meet good language standards. The final report should meet the standards laid out in the UNEG Quality Checklist for Evaluation Reports.

9. Specification of Roles

The Evaluator will be responsible for preparing for and carrying out data collection and analysis, and providing all deliverables specified in the paragraph above as according to the timeline below, as well as for adhering to evaluation norms, standards, and ethical conduct at all stages of the evaluation process. The evaluator should also provide periodic feedback to the Evaluation Manager on progress and any challenges.

The Evaluation Manager will timely provide the evaluator with all necessary background materials, project-related documents and monitoring data/tools, and will also provide logistical support and facilitating data collection (i.e. helping arrange any planned travel, meetings, interpretation and translation).  The EM is also responsible for managing the evaluation process, which includes helping address any challenges identified by the evaluator and by giving feedback on all deliverables (coordinating and consolidating feedback from other relevant IOM staff, such as project team, senior management, and regional office colleagues as needed).

1. Time schedule

This evaluation is scheduled to start ideally in October 2021, and the final evaluation report should be submitted no later than November 15, 2021.

Tentative timeline (activities listed can be done simultaneously) that could be amended upon discussions with evaluator is presented below: 

1. Evaluation budget

A total budget of GBP 6,000 has been allocated for the midterm evaluation of the project that should include the evaluators’ travel fare, per diem, local transportation, professional interpretation, and any other associated costs as required.

Required Qualifications:

Education and experience

• Postgraduate degree, preferably in Social Sciences (e.g. Economics, Sociology, Political Science, Migration Studies, Development Studies) or Law.
• At least 5 years of professional experience of conducting research.
• Demonstrated experience of conducting policy analysis and/or supporting policy development.
• Demonstrated experience in conducting national research projects, of publication in peer-reviewed journals or of being commissioned by the government to prepare reports.
• Excellent proven analytical and communication skills.
• Sound understanding of the migration, environment and climate change nexus and related challenges and opportunities will be an asset.
• Experience in liaising with the sectoral ministries, diplomatic missions and international organizations.
• Previous professional experience on the relevant topics (e.g. climate and environmental policies, clean energy, circular economy, sustainable agriculture, green designs, etc.) will be an asset.

Languages:

• Fluency in Ukrainian and/or Russian languages is mandatory.
• Fluency in English is a distinct advantage. All reporting should be undertaken in English.

How to apply:

Interested candidates are invited to submit their CV and cover letter to:

IOM Kyiv: [email protected]

8 Mykhailivska St., Kyiv, 01001

Please, indicate the position you are applying for in the subject line of your message.

Closing date for applications is 21 September 2021; however, interested candidates are strongly encouraged to apply sooner.

Only shortlisted candidates will be contacted.

Please note that Private entrepreneurs (PE) cannot be considered as applicable for providing services under this consultancy.

^[1] Engage, Connect, Empower: Catalysing Youth Participation to Promote Cohesive and Resilient Communities, project implementation October 2020 – March 2021; project code IS.0048, https://primaforall.iom.int/sites/Europe/UA10P0518/Pages/Home.aspx

[2] In case of travel restrictions, when the site visits cannot be conducted or operationally not feasible.