ActionAid UK and Yappika-ActionAid, members of the ActionAid International federation, are seeking the services of an Indonesian national consultant or consultancy team to conduct an external final evaluation into ActionAid’s DEC-funded humanitarian response (September 2018 – August 2020) to the 2018 Central Sulawesi earthquake/tsunami.
Yappika-ActionAid was established in 1991 as a non-profit organization focusing on fighting poverty and injustice and building the capacity of local organizations. On September 28th 2018, Central Sulawesi, Indonesia, was struck by an earthquake causing liquefaction and a destructive tsunami; the combined effects of these events led to the deaths of an estimated 4,430 people and the injury and displacement of many more. Yappika-ActionAid (YAA), a member of the ActionAid International federation, has been implementing a multi-sector humanitarian response to this ‘triple crisis’ in three affected areas: Palu, Sigi Regency and Donggala Regency. The Central Sulawesi earthquake and tsunami was YAA’s second humanitarian crisis response, after its response to the tsunami in Aceh at the end of 2004. YAA’s response has been part-funded by funds raised through a Disasters Emergency Committee (DEC) appeal in the UK, including match funds from the UK government. The response was divided into two phases.
• Phase 1: 28th September 2018 – 31st March 2019 (17,447 planned beneficiaries total)
o Outcome A: Increase access to urgent and basic assistance to the most tsunami-affected people in 3 affected districts of Central Sulawesi, Indonesia.
o Outcome B: Enhanced security, dignity and empowerment of affected-community members through establishment of women-led community-based protection mechanism in 3 affected districts of Central Sulawesi, Indonesia.
o Outcome C: Kick-start recovery of livelihoods of the most disaster affected vulnerable groups in 3 affected districts of Central Sulawesi, Indonesia
o Outcome D: Establishment of accountability system and information sharing
o Outcome E: Partner staff understand and implement the Core Humanitarian Standard
• Phase 2: 1st April 2019 – 30th September 2020 (3,775 planned beneficiaries total)
o Outcome A: Vulnerable people have enhanced access to safe, gender-sensitive, inclusive and disaster resilient houses and DRR initiatives in Central Sulawesi, Indonesia
o Outcome B: Affected women have access to safe spaces and protection services in Central Sulawesi, Indonesia
o Outcome C: Vulnerable groups, especially women, have restarted their livelihood in Central Sulawesi, Indonesia
o Outcome D: Community members take positive steps to prevent COVID-19 transmission and mitigate secondary impacts of COVID-19 (outcome added in May 2020 in response to COVID-19 pandemic)
The response has been implemented in collaboration with local civil society partners: Solidaritas Perempuan (Phase 1 only), FATAYAT-NU (Phase 2 only), Sikola Mombine and WALHI Sulawesi Tengah. Technical and grant management support and donor relationship management with the DEC has been provided by ActionAid UK. The response was designed and implemented in line with ActionAid’s humanitarian signature, emphasising the importance of women’s leadership in humanitarian crises, shifting the power and accountability, all underpinned by a commitment to building resilience.
This Call for Expressions of Interest in conducting the final evaluation will be the basis of the final Terms of Reference for the consultancy.
Objectives and purpose of the evaluation
This final evaluation will generate rigorous data, analysis and learning related to the design, implementation and results of both phases of the response, for two purposes:
• The primary purpose of the evaluation is to support continual improvement of humanitarian responses by ActionAid and other humanitarian organisations, including Yappika-ActionAid’s ongoing humanitarian response work in Indonesia. A thorough management response process will be conducted following the submission of the final report to digest key findings and recommendations and develop an action plan for their implementation. Spaces will be made for internal and external reflection on the evaluation findings, to identify any implications for broader programming approaches.
• The evaluation is also a key element of ActionAid’s accountability framework. Accountability to affected and at-risk communities is a key element of ActionAid’s humanitarian signature and the evaluation will offer these communities the opportunity to provide critical feedback on the response. COVID-19-related restrictions permitting, the preliminary findings of the evaluation will be shared, discussed and validated with project beneficiaries and other affected communities. The final evaluation report will be uploaded in full to the ALNAP website in fulfilment of ActionAid’s contractual obligations with the DEC.
The Core Humanitarian Standard on Quality and Accountability (CHS) will be the primary framework for this evaluation, and all key evaluation questions (see below) are derived from the CHS Nine Commitments. However, it is expected that other relevant ActionAid and sector standards and frameworks may be utilised in the course of the evaluation, including
• ActionAid Human Rights-Based Approach (HRBA)
• ActionAid Accountability, Learning and Planning System (ALPS)
• OECD Development Assistance Committee Criteria
• Humanitarian Charter and Minimum Standards in Humanitarian Response
• The Sphere Handbook
Any ActionAid internal frameworks will be made available to the contracted consultant during the inception period.
Key evaluation questions
Scope of the evaluation
The evaluator(s) will address the evaluation questions listed above across all three project intervention areas (Palu, Sigi Regency, Donggala Regency) and every listed outcome over the two phases of the response. The evaluation will focus on those activities funded by DEC funds; however, activities funded from other sources are also within the scope of the evaluation.
ActionAid will work with the evaluator(s) to identify suitable participants and stakeholders at the local and national level for interview. It is expected that they will be drawn from, at a minimum: affected and at-risk communities benefiting from the response, including women from Women Friendly Spaces and livelihood groups; staff from YAA and AAUK; local civil society partners Sikola Mombine, WALHI Sulawesi Tengah and FATAYAT-NU; representatives of local government agencies; and other international non-governmental organisations involved in the response.
Expected evaluation approach and methodology
The evaluation will adopt a mixed-methods approach, with a focus on gathering qualitative data to complement and enrich existing quantitative datasets (to be made available to the evaluation team) gathered through ongoing project monitoring against the response logframe and DEC output table. In line with good practice in qualitative research, sampling will be conducted purposively and sample size and composition will target saturation.
In light of the ongoing COVID-19 pandemic and government restrictions on movement and assembly, data collection for this evaluation will be conducted remotely to ensure the safety and security of everyone involved. Due to internal travel restrictions, it is expected that the lead evaluator, if they are not already based in Sulawesi, will remain in their current base. The lead evaluator will work with assistants based in Sulawesi to make contact with evaluation participants and make logistical arrangements for their remote engagement in interviews or group discussions. ActionAid will remain open to the possibility of adjusting the evaluation methodology in light of developments in the COVID-19 situation.
ActionAid expects all evaluations to be carried out in line with our Evaluation Standards. The full ActionAid standards will be shared on request, but in practical terms require the evaluation and evaluator to consider the points laid out in Box 1 below. It is expected that after initial discussions, the evaluator(s) will develop a comprehensive inception report, including draft data collection tools, for addressing the key evaluation questions above, which will be finalised in collaboration with ActionAid.
Coordination, roles and responsibilities
Representatives from YAA, AAUK and implementing partners will form an evaluation committee, responsible for coordination, decision-making and follow-up (including the management response) relevant to the evaluation; a staff member from YAA will lead this committee. The selected consultant or consultancy team will be contracted by and report directly to Yappika-ActionAid but will also collaborate with AAUK and implementing partners as necessary. YAA will recruit local research assistants in Central Sulawesi; once recruited, the research assistants will report to and be managed by the evaluator(s). The evaluator(s) will coordinate with civil society implementing partners when planning and implementing data collection activities; partners will provide operational and logistical support to the evaluator(s) in the carrying out of these activities. The evaluator(s) will be expected to provide regular progress updates to the evaluation committee, at a frequency and in a format to be agreed during the inception period.
The intended outputs of the consultancy are as follows:
• Evaluation report of no more than 25 pages, (excluding annexes) as a Word document, written in clear and concise English with minimal jargon. The report will include:
o Cover page (title of the evaluation report, date, name of consultants, photo)
o Contents table
o Executive summary of no more than 2 pages outlining the purpose of the evaluation, main points of analysis, key findings, conclusions and recommendations
o Introduction outlining the background to the intervention and the evaluation
o Purpose and objectives of the evaluation
o Methodology/approach, indicators used, and limitations of the evaluation
o Major findings (data analysis, including gender analysis)
o Lessons learned and recommendations
o Annexes: details of data collection tools, schedule of field visits and meetings; list of people interviewed; bibliography of key documents consulted; ToR for the evaluation
• A presentation of preliminary findings and recommendations to the AA Evaluation Committee, immediately prior to the submission of the draft report to AA.
• A PowerPoint presentation summarising key finalised findings and recommendations that can be used to disseminate findings within ActionAid and with external stakeholders.
• An ActionAid Evaluation Summary, 2-3 page abstract of the evaluation in standard format.
• The raw data (all transcripts, quantitative data, data collection tools) must be handed over to ActionAid together with the evaluation report. All data should be anonymised. The evaluators may withhold data from ActionAid if it is felt that anonymising the data is effectively impossible.
• Where photos are included, these should be provided to ActionAid in high resolution JPEG format, and catalogued by ActionAid on Storieshub according to our photo guidelines. The evaluation team must ensure photos are obtained in line with ActionAid guidelines on consent.
• Diagrams/flowcharts/infographics developed by the evaluation team can be included but the original artwork should also be submitted as separate files along with the evaluation. Note ActionAid reserves the right to modify artwork as appropriate
The evaluation Terms of Reference will form part of their contractual agreement with ActionAid and will detail issues of data ownership and intellectual property rights.
The total value of this contract is GBP 18,000, inclusive of consultant day rates and all costs associated with the evaluators’ involvement with the evaluation, including data collection costs and personal expenses. Local research assistants will be recruited and contracted by YAA and YAA will bear directly any costs associated with their involvement, including fees and expenses – please do not include local research assistants in your financial proposal. Any financial proposals over this value will not be considered. ActionAid is looking for a financial proposal that offers good value for money and maximises potential efficiencies to deliver the outputs within budget.
The following key documents will be made available to the evaluation team. Please note that some documentation will only be available in either English or Bahasa.
• Project documentation including project proposal, logframe, output table, management reports, needs assessments, case studies, and previous evaluations;
• AA International and country strategic frameworks
• AA International frameworks: AA Human-Rights Based Approach (HRBA), Accountability, Learning and Planning System (ALPS), AA MEL Approach, and AA Feminist Research Guidelines.
• ActionAid guidelines on gathering content (photos, case studies etc.)
• Internal guidelines and policies (AA Strategy, staff security policy, child protection policy etc)
• Contact details for key staff, communities of practice and stakeholders and relevant websites/ pages.
• ActionAid’s child protection policy, research ethics standards, global SHEA policy and other organisational policies will be included as an annex to the Terms of Reference and therefore will form part of the contract with the selected evaluator(s), as they will be expected to abide by these policies for the duration of their contract with AA.
Successful applications for this consultancy will be able to demonstrate their competency and experience against the following criteria:
1. Expertise in responses to humanitarian crises, women’s rights and gender equality
2. Thematic expertise in humanitarian sectors relevant to the response, including food, shelter, protection, livelihoods, WASH and DRR, and key humanitarian principles and standards (e.g. CHS)
3. Geographic expertise in Indonesia in general and Central Sulawesi in particular
4. Excellent written and spoken English and Bahasa Indonesia
5. Experience in designing and managing rigorous evaluation and/or research processes
6. Experience in innovative qualitative and quantitative methods and analysis
7. Experience in designing and conducting participatory and empowerment-focused methods and strong downward accountability mechanisms to actively share results and learning
8. Experience in engaging in feminist, gender-transformative and/or gender-sensitive evaluation
9. Evidence of use of ethical considerations and methodological measures for that respect the rights, safety and security of all stakeholders
10. Evidence of successful collaboration with NGOs, and particularly collaborations which included capacity building and ‘on-the-job’ training with field office staff.
11. Evidence of client responsiveness, creativity and flexibility of approaches towards clients’ needs and/or challenges in research implementation.
12. Evidence of producing clear, concise reports and presentations in English and Bahasa
13. The lead evaluator must be based in Indonesia
14. The lead evaluator or a senior member of the evaluation team must be a woman
All applications for this evaluation consultancy must include the following documents:
• Expression of interest demonstrating how the applicants meet the Evaluator Specification as above
• Technical proposal for the evaluation study including proposed activities schedule/work plan with time frame, methodologies including tools and participatory and gender sensitive approaches that will be used and why, in line with the above-mentioned project priorities and evaluation standards
• Financial proposal (separating core consultancy costs and daily rates of team members and in-country data collection costs)
• CV(s) of all members of the evaluation team, including an articulation of the role each member will play
• Examples of relevant written outputs (e.g. research or policy work) conducted on behalf of a client (the lead evaluator should be the primary author)
• Signed reference or certificate of satisfactory completion of previous evaluation work conducted on behalf of a client
Please submit your application, including all relevant supporting documents, via email to Fransisca Fitri, Executive Director of Yappika-ActionAid, at email@example.com and cc office@ yappika-actionaid.or.id. The deadline for submission of applications is Monday 3th August at 5pm Western Indonesia Time / 6pm Central Indonesia Time. Any applications submitted after this time will not be considered. ActionAid will be in touch with all applicants within 7 days to let them know whether or not they are to be invited for interview.