18.07.2022

Evaluation Consultant on Local Governance for lessons learned report development (PMCI Memo) for USAID DOBRE Program/ Консультант/ка з оцінки з питань місцевого самоврядування для написання звіту про отриманні уроки (PMCI мемо) в програму USAID DOBRE

ЗАВЕРШЕНО

Global Communities is seeking qualified short-term consultant for USAID-funded Decentralization Offering Better Results and Efficiency Program (DOBRE):

POSITION: Evaluation Consultant on Local Governance for lessons learned report development (PMCI memo)
PERIOD OF PERFORMANCE:                        3 months
PLACE OF PERFORMANCE:                          Kyiv, Communities located in Kharkiv, Dnipropetrovsk, Kherson, Kirovohrad, Mykolaiv, Ivano-Frankivsk, Ternopil Chernihiv, Chernivtsi, and Zaporizhia Oblasts
LOE (Level of Effort) DAYS:                           40 days

OBJECTIVE:

The USAID Program, “Decentralization Offering Better Results and Efficiency” (DOBRE), is a six-year program, implemented by Global Communities and funded by the United States Agency for International Development (USAID). DOBRE is working to help consolidated communities (CCs) in Ukraine to meet the challenges and seize the opportunities that decentralization brings, by improving their local governance and increasing the engagement of their citizens in policy- and decision-making. Areas of support include strategic planning; financial management; public service delivery; local economic development, and gender- and youth-responsive policies. DOBRE is also working to foster linkages and cooperation across CCs. Partners with Global Communities in the DOBRE Program Consortium include the Ukrainian Crisis Media Center (UCMC); SocialBoost; the Foundation in Support of Local Democracy (FSLD/FRDL), the Malopolska School of Public Administration at the Krakow University of Economics (MSAP/UEK), Poland; and the National Democratic Institute (NDI).

The USAID DOBRE Program is working closely with 100 CCs in 10 Oblasts of Ukraine: Dnipropetrovsk, Ivano-Frankivsk, Kharkiv, Kherson, Kirovohrad, Mykolayiv, Ternopil, Chernivtsi, Chernihiv and Zaporizhzhia.

The DOBRE Program used the Participatory Municipal Capacity Index (PMCI) as a tool for measuring the progress of participating communities in local government capacity. It is a tool provided to 100 CCs, to strengthen their ability for self-assessment of needs and priorities, such that the process of continuous learning and growth will provide for sustainable improvements in government capacity.

PRIMARY RESPONSIBILITIES / SPECIFIC TASKS:

Based on the results of its work, DOBRE plans to publish a series of studies on the results, lessons learned, and recommendations on the implementation of key instruments and practices. DOBRE communities` recent progress in capacity measured with PMCI throughout 2017-2022 in 100 communities will be one of the main areas of research.

Purpose of the assignment:

To capture findings and insights from the DOBRE Program experience related to both (1) the implementation of Participatory Municipal Capacity Index (PMCI) as an assessment and capacity building tool in DOBRE communities and the effectiveness of this tool to achieve Program results; and (2) potential for further use of the PMCI tool outside the DOBRE Program.

 Goals: 

Capture main results for local government capacity strengthening after using the PMCI tool (Matrix and Road Map);
Provide recommendations to stakeholders beyond DOBRE to inform the implementation of community assessment capacity building tool;
Promote the positive results of DOBRE and GC for use in capacity building development; 
Develop PMCI result infographic (as a separate annex).

The PMCI memo (up to 30 pages) will be developed using the following data sources: Participatory Municipal Capacity Index (PMCI) assessment data and its results; program performance indicators data where applicable; any other publicly available data.

Objectives and scope of the study

The objectives of the study are:

To understand to what extent the PMCI tool (including PMCI Matrix and Road Map) was successful in strengthening local government competencies such as strategic and financial management, service delivery, citizen engagement, including youth engagement, gender equality, tools of e-democracy etc.;
To identify the factors that influenced the dynamics of community capacity building;
To define the challenges faced by the DOBRE Program in implementing the PMCI in communities and what limitations the tool has;
To develop feasible recommendations regarding the implementation of the PMCI (Matrix and Road Map) or similar tools.

Key questions:

To what extent did participating local governments’ (LG) self-assessments (using the PMCI tool and process) indicate a measurable increase in capacity?
In which capacity areas do scores indicate the highest/lowest increase? What were the enabling factors of that?
What correlations can we see between changes in capacity and characteristics related to the community or its local government, for example: population size, date of amalgamation, geographic location (oblast), economic/income level etc.
To what extent do LGs that have applied the PMCI consider it a useful in enabling them to identify their own strengths and weaknesses, and make decisions about how to invest in further capacity strengthening?
What do LG officials consider most/ least useful about the PMCI tool and process?
How likely do LG officials consider it that the LG will continue to apply the PMCI in some way from now on, without facilitation from DOBRE?
How can we promote and expand the use of the PMCI tool beyond the DOBRE Program?

Methodology

The proposed methods of data collection and analysis should include:

Desk review and analysis of the PMCI tool and its results, DOBRE project reports, monitoring data, success stories, research results, learning product about different areas of DOBRE intervention (if available). The consultant will use quantitative data collected through application of the PMCI tool in each of the participating communities from the program’s inception. For Cohorts 1-3 this will be 4-5 (including baseline) sets of data collected through annual assessment processes. For Cohort 4, this will be 2 sets of data (including baseline).
Key informant interviews by telephone (skype, zoom, etc.) with program staff and representatives of local authorities. The consultant will use qualitative data collected from key informants or small focus groups of local authorities and DOBRE program staff who have participated in the PMCI process. The qualitative data will supplement the quantitative data to provide participants’ perspective on the value of the PMCI and the likelihood their LG will continue to apply it post-DOBRE.

We will use data analyses to illustrate changes and successful results, including detailed descriptions of specific ways for achieving desirable outcomes and important context details, and the potential and actual impact at the local level. This ‘how we learned to do it’ focus in presenting program results will be helpful for future scaling up of the project experience in Ukraine. The assignment will entail qualitative and quantitative data analysis to uncover the themes relevant to the research questions.

Key audiences for the report will be:

GC headquarters 
USAID 
Other donors working in Ukraine on decentralization
Implementers of local government programs
The Ministry of Communities and Territories Development of Ukraine
Associations of local governments and local government representatives.

Scope of Work, deliverables & timeline

The following steps are foreseen with required deliverables and proposed timelines

Develop an Inception Report in English with a detailed work plan should include desk research results, methodology detailing relevant data sources, strategies to collect and analyze data.

For the inception report development, the expert should get familiar with the Program DOBRE, its objectives, beneficiaries, partners, and respective program documents on instruments and mechanisms implemented. All Program documents, in particular PMCI methodology, PMCI community results and other necessary documents will be provided by the DOBRE Monitoring, Evaluation and Learning (MEL) Team.

The inception report should include data collection tools (questionnaires, guides for focus group discussion if applicable etc.). Questionnaires should be designed for two categories of interviewees: local governance representatives and DOBRE staff (program officers and technical experts).

Estimated LOE for Task 1 – 8 working days

Key informant interviews by telephone (by skype, zoom, etc.) or focus groups with local authorities and DOBRE program staff who have participated in the PMCI process.

In-depth interviews and consultations with local authorities to understand the PMCI assessment process in CCs, as well as identify best practices, barriers, potential and actual impact, and challenges in promoting the PMCI tool. At least 1 interview with local officials in 15 selected communities.Selected communities are expected to be from different oblasts, different in size, and different in progress.
In-depth interviews and consultations with the DOBRE Program officers and technical experts to understand the program implementation process and quality, as well as identify best practices, barriers and challenges in promoting the PMCI tool. At least 6 interviews with program officers who were responsible for the PMCI assessment in selected communities, and 3-4 interviews with DOBRE technical experts.

Estimated LOE for Task 2 – 12 working days

Preparation of the draft report (PMCI memo) and draft infographic of results of the PMCI assessment in English and Ukrainian.

The draft report should include the following parts:

Description of the PMCI assessment methodology – brief description of the history of communities’ capacity assessment, what methodology was used to and how it changed according to the different circumstances.
Findings and Conclusions – this part should be presented based on a careful analysis of the data identified, outcomes defined, and specific achievements of the DOBRE Program. Conclusions should be based on evidence and logically related to evaluation objectives and evaluation questions.
Recommendations – this section should seek to provide very practical, feasible recommendations. The recommendations will be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the report. This recommendation can be used to have better achieved the project’s goals in the future or in future programming, or in similar projects.

The draft infographic of the results of PMCI assessment should visualize the results obtained and give an understanding of how the capacity of communities in different areas has changed over time.

Estimated LOE for Task 3 – 15 working days

Preparation of the final report and results infographic annex. DOBRE MEL team and Program management will review the first draft. The consultant will finalize all recommendations and make the necessary changes to the document and the final report will be refined to reflect the recommendations made.

Estimated LOE for Task 4 – 5 working days

COORDINATION AND SUPERVISION:

To ensure the fulfillment of the above tasks, the Consultant will:

Liaise and ensure constant communication/coordination with the DOBRE Program team and consultants/experts regarding all aspects of the assignment to ensure coordinated inputs into the PMCI memo development;
Submit all the deliverables for review, comments and approval to the DOBRE Program team as requested;
Hold consultations and briefings with DOBRE project team as requested.

THE SELECTION PROCESS:

Global Communities will select consultants based on meeting the following criteria:

Qualifications and Experience of Local Evaluation Consultant:

Master’s or Specialist degree in Sociology, Statistics, Economics, Policy Analyses, Public Administration and other related areas;
At least three years of experience in the field of monitoring, evaluation, or expertise in the area of social research, including data collection, analysis, and report writing;
Strong analytical skills and ability to apply statistical methods to test hypotheses;
Experience as evaluation consultant/expert/researcher in international technical assistance projects evaluation will be an added advantage;
Experience and knowledge of local government management and decentralization reform;
Previous experience (or ability to demonstrate the capacity) of working with the CCs;
Familiarity with data visualization tools;
Working level of English (Intermediate).

To apply please send your CV, technical offer with methodology and summary of your experience and day-rate as described above to [email protected] indicating “Local Evaluation Consultant” in the subject line by 10 AM, July 29, 2022. Only applicants selected for interviews will be contacted. No telephone inquiries will be accepted.