External Program Evaluation – Request for Proposals, IREX (тендер)

1. Background 

Learn to Discern in Education (L2D-Ed) programme is aimed to equip Ukrainian educators with the awareness, knowledge, and skills that lead to critical information consumption, and the pedagogical skills to effectively transmit CIC to students across a variety of disciplines throughout the Ukrainian secondary school system. This will help to bring up Ukraine’s next generation of well-informed, discerning voters, leaders, and activists.  

Within the project more than 3603 teachers from 1558 schools across the country took part in the program. They received an extended media literacy training, learned how to integrate exercises on the development of critical information consumption skills and received materials for school lessons.  

IREX is looking for a vendor to conduct an external evaluation among teachers L2D-ed program participants.  

Goal of an external evaluation: To acquire an in-depth understanding of the L2D-Ed instructional strengths, as well as areas for improvement, assess the impact of the program on resilience to malign information operations. 

Objectives: 

• Find out what changes have taken place in the consumption of information and media content by teachers after participating in the program 
• Find out benefits of the trainings beyond misinformation. find out benefits of the trainings beyond misinformation. Explore connection of mental health and information consumption; and how the programme benefits former. 
• Determine the resistance to misinformation of teachers participating in the program  
• Investigate the experience of integrating media literacy and CIC skills in teacher education. What worked, what were the difficulties.   
• Find out the changes that teachers noticed in students after integrating media literacy and CIC skills into subjects.   
• Find out recommendations from teachers on what can be done to facilitate the integration of media literacy and CIC skills into subjects. 
• To test project participants teachers their resilience to disinformation on practical examples and compare their results with the teachers from the control group. 

Target audience (TA): 

TA 1: secondary schools’ teachers, project participants (database will be provided by IREX, n=3603). The teachers joined the project in different period of time. There were 5 program expansions. 

• Pilot-March 2018​ 
• Phase 1-January 2019​ 
• Phase 2-August 2019 
• Phase 3-June 2020 
• Phase 4-February 2021 
• Phase 5-November 2021 

TA 2: secondary schools’ teachers who teach Ukrainian Language and Literature or Arts & Culture or Ukrainian History and World History, that have not been covered by the project activities. 

Regions of an external evaluation: all oblast of Ukraine. 

1. Scope of work 

IREX expects from selected vendor administering two parts of research.  

1. Qualitative part of an external evaluation 

Target audience 1: secondary schools’ teachers, project participants (database will be provided by IREX). 

Vendor will: 

• Conduct 2-4 in-depth interviews before the FGD to test the guide. 
• Conduct 15 focus groups discussions (FGDs) with secondary schools’ teachers, project participants. At least 3 for each of phases. 
• Guide for FGD is provided by IREX.  
• Conduct fieldwork.  
• Prepare narrative report. 

1. Quantitative part of an external evaluation 

1. Target audience (TA): 

1. TA 1: secondary schools’ teachers, project participants (database will be provided by IREX). Total sample size not less than 400. 

1. TA 2: secondary schools’ teachers, that have not been covered by the project activities. Total sample size not less than 400. 

1. Survey questionnaire is provided by IREX. Interview duration: 20-25 min.   

1. Method of interview: f2f/online survey with CATI. 

1. Vendor will: 

• Develop design of the survey, including sampling plan (sample size for teachers program participants and for teachers from the control group, quotas, method of recruitment, distribution by geographic region: 6 regions (Donbas, east, central, south, north, west), type of settlement (oblast center, city other than oblast center, urban village, village), subject (Ukrainian Language, Ukrainian Literature, Arts & Culture, Ukrainian History, World History), and for TA 1 phase of expansions (Pilot-March 2018, Phase 1-January 2019, Phase 2-August 2019, Phase 3-June 2020, Phase 4-February 2021, Phase 5-November 2021). 

1. Conduct pre-test of the survey 

1. Piloting report 

1. Conduct fieldwork.  

1. Prepare analytical report. 

Deliverables 

• Methodology and sociological tools that can assess the goal of evaluation.  
• Translation guide for FGD and survey questionnaire from English to Ukrainian. 
• Piloting report (up to 1 page) 
• Technical report for survey (including sampling procedures, fieldwork description, response rate, etc.). 
• Database with coded open question and codebook for the survey in Excel (xls/csv) or SPPS (sav) in English and Ukrainian. 
• Analytical report in PowerPoint in English and Ukrainian.  
• Transcripts in language of respondents. 
• A short summary report of the evaluation with key findings and recommendations based on quantitative and qualitative parts of an external evaluation. 

Timing/duration. Work should be completed by May 2, 2022. 

1. Proposal requirements 

Proposal should be sent to [email protected] no later than 18:00 February 27th, 2022.  

Any clarification questions regarding to the RfP should be submitted to [email protected] not later than 16:59 February 20th.  

The vendor will be selected by March 4, 2022. All companies that apply will be informed about the selection results. 

 Elements of the proposal:  

• Narrative proposal (maximum size of the narrative proposal in MS Word is 6 pages (font size 12, 1.5 line spacing) which includes   
• Methodology with description of survey design and how the company plan to reach teachers from TA 2., design the sample for assessing goal of the evaluation. 
• Capacity of the organization (staff have significant experience in conducting all types of research at the highest level, the presence of interviewers in all regions of the country, the availability of the necessary technical equipment, the ability to conduct research using qualitative and quantitative methodologies and capacity to use of different research methods, ets); 
• Activity report with relevant examples of completed similar assignments performed during the last 2 years, and specifically a description of one assignment performed during pandemic context. Add contacts (mobile phone and e-mail) of representatives of three previous clients who can provide recommendations to your company on the provision of conducting surveys. 
• Timeline by different activities preliminary proposed in the goal.   
• Budget calculations by budget lines. Budget calculations to be sent in a separate MSExcel file. Vendors may calculate more than one budget option depending on methods/sample they propose. Budget template 
• Organization documents 
• Copy of the VAT payer’s Certificate (subject to registration as a VAT payer) or the Single Tax Payment Certificate or extracts from the registers of VAT payers and single taxpayers. 
• Copy of the Certificate of state registration of a legal entity. 
• Copy of the Charter and extract from the protocol on the appointment of the head (for legal entities) 
• A copy or original of the document confirming the authority to sign the contract and the documents of the counterparty’s proposal. 

Budget ceiling is 17,500 USD without VAT.  
The payment is made in UAH according to the official currency exchange rate, determined by the National Bank of Ukraine as of the date of invoice issue. 

PLEASE NOTE: Applicants must state in their proposal that they understand that IREX is exempt from VAT and that this dismissal will apply to all invoices. Failure to declare the applicants’ intention to recognize the VAT exemption may serve as grounds for exclusion from consideration in accordance with this RFP.  

IREX is the executor of international technical assistance programs and projects in Ukraine in accordance with the Agreement between the Government of Ukraine and the US Government on Humanitarian and Technical-Economic Cooperation of May 7, 1992 (hereinafter – the “Agreement”) and procures goods, works and services for the project. international technical assistance, in accordance with the above Agreement and the Procedure for attracting, using and monitoring international technical assistance, approved by the Cabinet of Ministers of Ukraine dated February 15, 2002 N 153 (153-2002-n) “On establishing a single system of involvement, use and monitoring international technical assistance “; the value of such goods (works, services) is exempt from value added tax (VAT). Procurement of goods, works and services is carried out at the expense of the international technical assistance project and corresponds to the category (type) of goods, works and services specified in the procurement plan. 

The Project shall provide the winner with a copy of the Project Registration Card, which procures goods, works and services issued by the Ministry of Economic Development and Trade of Ukraine (MEDTU) and certified by the Project seal, as well as a copy of the procurement plan or an extract from the procurement plan. 

The bidder draws up a tax invoice for the provision of goods (works, services) marked “Excluding VAT” in the following manner. The tax invoice indicates the grounds for VAT exemption (name of the Project, number and date of the relevant contract). The subcontractor submits to the state tax authority at its location a declaration taking into account the above operations and indicates the value added tax benefit code № 14060049 in accordance with the Directory of tax benefits. (See Information and VAT Exemption Procedure). 

Language of submission is English.  

1. Selection criteria