Cambodia Nutrition Project –Impact Evaluation Study Survey Firm for Midline Data Collection

Health Sector Support Project 1 (HSSP1): End of Project Assessment

Location

Cambodia. Location within country: 12 provinces

Project Description

The main objective of HSSP1 was to contribute to the improvement of the health status of the population by increasing the accessibility and quality of health services and assisting the Royal Government of Cambodia to implement its Health Sector Strategic Plan and strengthen the sector’s capacity to manage resources efficiently. The objectives of the end of project assessment were: (i) Determine the achievements of the project over the 6 year period from 2003 to the end of 2008, and derive lessons that can be applied to the second Health Sector Support Program; (ii) Establish baseline values for the HSSP2 indicators.

A team of Angkor Research staff carried out 3 surveys:

Household survey of over 1,800 individuals within 44 clusters. The response rate was 98%, and interviewers conducted 10.7 interviews per day.
Qualitative survey on accessibility, quality, affordability and coverage of health services, through interviews and focus group discussions in 6 villages with a total of 208 respondents and participants.
Health facility assessment of 20 randomly selected health facilities. This survey included interviewing health facility directors (n=12) and health workers (n=93), clinical observations (n=186), and patient exit interviews (n=313).

Angkor Research staff were responsible for all survey activities including research design, sampling, staff recruitment and training, field work and supervision, database design, data entry, data cleaning, validation and analysis, and reporting.

Baseline Survey: Impact Evaluation of the Voice and Action: Social Accountability for Improved Service Delivery (I-SAF) Project

Location

Cambodia. Location within country: Battambang, Kampong Cham, Kampong Chhnang, Kampong Speu, Kampong Thom, Kampot, Koh Kong, Kratie, Mondulkiri, Preah Vihear, Prey Veng, Pursat, Ratanakiri, Stung Treng, Tboung Khmum (15 provinces)

Project Description

In order to enhance the accountability of service providers and local officials and, as a result, improve the quality of service delivery, the Cambodian government, non-governmental organizations (NGOs), and bilateral donors have – with assistance from the World Bank through the Voice and Action Program (VAP) – developed the Social Accountability Framework (SAF). Whereas evaluations of social accountability interventions in other contexts indicate that such interventions have been unsuccessful due to a failure to provide service providers and local officials with incentives to change their behavior, the design of the SAF builds on lessons from the body existing research by providing for coordinated interventions by both state and non-state actors.
The Voice and Action Program (VAP) Impact Evaluation (VAP-IE) will use a randomized controlled trial conducted across 42 districts in 15 provinces of Cambodia to determine whether the coordinated interventions mandated by SAF improve the quality of educational, health, and commune services provided to villagers in rural Cambodia and increase villagers’ engagement with local government. Evidence from the VAP-IE will be used by national authorities, NGOs, bilateral donors, and the World Bank to inform future funding and design decisions related to the National Decentralization Reform.

At the baseline, Angkor Research’s involvement in this large-scale, randomized control trial included instrument review/translation (with back-translation), primary CAPI data collection (tablet-based), data management, analysis and reporting to evaluate the effects of I-SAF on a range of household and service delivery indicators at the commune level. The sample is representative of the I-SAF program implementation areas, and covered 336 villages in 22 provinces of Cambodia. On the supply side, Angkor Research conducted interviews with 140 health centers, 168 commune councils, 168 primary schools, and 336 village chiefs (812 IDIs). 3,455 households were targeted for quantitative interviews, out of which 3,363 were completed (97.3% response rate). Data collection also included photos of local government offices, and GPS locations.

Evaluation of a World Food Programme Pilot of Food vs. Cash Assistance: Baseline and Tracking Surveys

Location

Cambodia. Location within country: 7 provinces (Banteay Meanchey, Kampong Cham, Kampong Speu, Kampong Thom, Siem Reap, Prey Veng, Pursat)

Project Description

The World Bank is partnering with the WFP in Cambodia to evaluate a pilot design to test food vs. cash support in the context of take home rations at primary school level. The objective of the study is to evaluate the impact of cash vs. food assistance in school feeding programs. A baseline survey of 4,300 households was started in May 2011 in 7 provinces, followed by a smaller tracking survey of 700 households in March 2012.

Angkor Research staff conducted all data collection and data entry activities, including anthropometric data. Specific activities of this consultancy included translating and finalising the research instrument, recruitment and training of teams of enumerators, preparing a logistical plan for carrying out data collection, piloting the survey instruments, implementing the household survey and data entry and processing.

The baseline survey sampled 4,300 households and 421 schools within the seven target provinces. Anthropometric data was collected from approximately 15,000 respondents. The response rate for the baseline survey was 94.4%. The tracking survey in 2012 collected data from 700 of the baseline households and 100 schools.

Evaluation of a World Food Programme Pilot of Food vs. Cash Assistance: Follow-up Survey

Location

Cambodia. Location within country: 7 provinces (Banteay Meanchey, Kampong Cham, Kampong Speu, Kampong Thom, Siem Reap, Prey Veng, Pursat)

Project Description

The World Bank partnered with WFP in Cambodia to evaluate a pilot design providing food and cash support to impoverished households, in the context of take-home rations for students at the primary school level. The objective of the study is to evaluate the impact of cash vs. food assistance in WFP school feeding programs on a number of household and individual indicators for health, nutrition, educational ability and socioeconomic status. The longitudinal impact evaluation (tracking the same households across all survey rounds) followed 4,300 households with primary school children enrolled in the WFP school feeding program over the course of one year. The baseline survey of 4,300 households began in March 2011 in the 7 target provinces, followed by a smaller tracking survey of 700 households selected from the baseline sample in March 2012. The follow-up survey was conducted from June to November 2012.

Angkor Research staff conducted all data collection and data entry activities for all three rounds (baseline, tracking and follow-up) of this impact evaluation. Specific activities conducted for the follow-up survey included: reviewing, translating and finalizing the research instrument, recruitment and training of enumerator teams of, preparing a logistical plan for carrying out data collection, piloting the survey instruments, implementing the household survey, and data entry and processing.

The follow-up survey sampled the same 4,300 households and 421 schools from the baseline survey. In addition to the modified baseline instrument, anthropometric data was collected from approximately 15,000 respondents. The follow-up survey was completed within the timeframe agreed upon with the World Bank, with fieldwork conducted over 5 weeks. The response rate for the follow-up survey was 94.5%, which is consistent with the baseline survey and within the parameters of the contract and evaluation methodology. Because of the length of the instrument (over 80 pages of data per household, plus anthropometric data), enumerators averaged 4.7 households each per day.

Survey Supervision and Quality Assurance: Health Markets and Utilization Survey and Cambodia Medical Worker’s Professional Development Survey

Location

Cambodia. Location within country: Phnom Penh and 9 provinces (Banteay Meanchey, Kampong Cham, Kampong Speu, Kampot, Preah Vihear, Prey Veng, Siem Reap, Takeo and Kampong Thom)

Project Description

The main purpose of this assignment was to provide independent supervision and quality assurance for two World Bank surveys being fielded in Cambodia by third-party consultants as part of the 2nd Health Sector Support Program (HSSP2). The 2011 Cambodia Medical Worker’s Professional Development Survey was a labour market survey for health medical professionals in public, private, and non-profit sectors to inform decision making regarding appropriate compensation policies and incentives for health medical workers in the public sector. The study improved understanding of health worker motivation and how different financing schemes impact compensation, motivation, work hours and private practice.

The survey was conducted in 19 operational districts (ODs) with a sample of 1,000 health professionals. The Health Markets and Utilization Survey was conducted to measure the prices, distances, and market shares associated with health care providers, and to measure health and health seeking behavior in the households that they tend, in order to describe the structure of health care markets and inform government and donor programs that seek to improve the quality of care in Cambodia. The survey was conducted in 8 ODs with a sample of approximately 2,000 households and 1,000 health providers.

Staff of Angkor Research conducted all aspects of the quality control procedures for both surveys, including:

Collaboration with implementing agencies in the field to supervise and implement quality control procedures for data collection. This involved oversight of sampling applications, interview observations, and
Re-interviews with 5% of total samples for each survey and respondent type (households and health facilities);
Verification of proper implementation of field supervision and quality control protocols by the implementing agencies;
Verification of data entry accuracy and database maintenance procedures via a random check of 5% of entered data;
Real-time collaboration and feedback to implementing firms to improve implementation & data quality as required.

Outputs for this project included bi-weekly progress reports throughout the course of the project (oversight of data collection and data entry activities), re-interviews with approximately 100 medical professionals and 100 household respondents, and entry of approximately 200 questionnaires for data entry verification. Re-interviews were conducted face-to-face and by telephone when necessary.

Impact Evaluation of Service Delivery Grants (SDGs) in the Health Sector in Cambodia: Baseline Survey

Location

Cambodia. Location within country: 23 provinces/municipalities (Banteay Meanchey, Battambang, Pailin, Kampong Cham, Kampong Chhnang, Kampong Speu, Kampong Thom, Kampot, Kep, Kandal, Kratie, Mondulkiri, Oddar Meanchey, Phnom Penh, Prey Veng, Pursat, Ratanakiri, Siem Reap, Sihanoukville, Stung Treng, Svay Rieng, Takeo, Tboung Khmum)

Project Description

The World Bank, in collaboration with the Ministry of Health (MOH), is supporting the redesign of Service Delivery Grants (SDGs) that will be implemented as part of the larger Health Equity and Quality Improvement Project (H-EQIP). H-EQIP aims to increase the sustainability of these innovations by improving their resourcing and management as envisaged in the government’s Health Strategic Plan 2016-2020 (HSP3). It will further strengthen the results-based focus of SDGs with a specific goal of improving quality of health service delivery and utilization of services by the poor and will use a multi-pronged approach to strengthening health systems, especially to support improvements in quality of care.

As part of its support, WB designed a rigorous impact evaluation to measure improvements in quality (and quantity) of health service delivery attributed to the SDGs. The results from this quantitative evaluation are expected to inform the Royal Government of Cambodia (RGC) and Development Partners on the efficacy of the proposed intervention. Impact evaluation will take place at the same 3 levels where SDGs are inputted: health centers, hospitals and operational districts (ODs).

The objectives of the baseline data collection are to provide a reference point against which the interventions’ effects can be measured, help inform the intervention design, and contribute to the evidence base on quality of care in Cambodia.

At the baseline, Angkor Research’s involvement in this large-scale, randomized control trial included sample selection, instrument review/translation (from the WB RBF IE Toolkit, with back-translation), primary CAPI data collection (tablet-based) and data management for surveys of both the demand-side and supply-side aspects of health care utilization and access, to evaluate the effects of SDGs on a range of household, service delivery and health care indicators.

The sample is nationally representative, and covered 70 ODs in 23 provinces of Cambodia. On the supply-side, Angkor Research conducted interviews with 70 OD directors, health facility evaluations at 140 public health centers (including medical record audits and drug/medical equipment inventories), 545 interviews with medical staff (including situational vignettes), and 1,000 exit interviews with patients. In the catchment areas of the target health centers, 2,500 households were interviewed on a range of indicators, including health equity fund (HEF) status anthropometrics, health-seeking behavior, out-of-pocket health expenditures and maternal/child health. Household data is longitudinal, and can be linked between survey rounds. Data collection also includes interviews with village authorities and GPS locations in all villages. The household survey was completed with a 98.7% response rate.

Options for Integrating HIV & TB in Domestic Health Financing Mechanisms in Cambodia

Health Equity Fund Utilisation Survey

Location

Cambodia. Location within country: 12 provinces (Banteay Meanchey, Battambang, Kampong Cham, Kampong Chhnang, Kampong Thom, Koh Kong, Kratie, Oddar Meanchey, Preah Sihanouk, Pursat, Takeo, Tbong Khmum)

Project Description

Health Equity Funds (HEF) are grants managed by third party operators (NGOs or CBOs) that aim to increase access of the poor to health services in hospitals and generate additional revenue for referral hospitals. HEFs are now a key building block in the overall strategy being developed by the Government and its development partners for tackling inequities in the health sector. HEFs are being implemented in all of Cambodia’s 23 provinces and Phnom Penh municipality and are fully rolled out (by URC scheme) in 51 (out of 81) ODs (63%). The MoH aims to reach full national geographical coverage in 2014. However, a number of surveys including CDHS 2010 and CSES 2011 have suggested that HEFs may not always be used by their members to receive medical care free of charge. Overall, available data indicates that HEFs may be suffering from underutilization, although these estimates need refining with studies devised specifically to address this question.

The main purpose of this assignment is to measure – using a combination of quantitative and qualitative methods – the level of utilization of Health Equity Funds (HEFs) among the poor and the various factors that explain the utilization pattern observed. This study is expected to help generate evidence that will improve access to health services by the poor in Cambodia as well as inform the policy dialogue with the Royal Government of Cambodia (RGC) with the aim of strengthening RGC capacity to move towards universal health coverage (UHC).

After a public procurement and bidding process, Angkor Research was awarded the contract to conduct HEF Utilisation Survey from the World Bank (Cambodia office). The study combines both quantitative and qualitative components, and includes treatment (HEF members) and control (non-HEF members) groups to understand utilisation patterns of HEF members relative to the general population. The 2,000 household quantitative survey (1,000 treatment and 1,000 control) is also segregated by distance to the nearest public health facilities; and qualitative in-depth interviews (IDIs) were conducted with nearly 300 key informants including HEF members, HEF operators, public hospital/health center directors and doctors, and village health support group volunteers. Angkor Research was responsible for all components of this mixed methods study, including: initial research design and literature review; sample selection methodology, framework and selection; instrument design, translation and pre-testing; national ethical approval (NECHR); field data collection; data entry and data management; transcription, translation and coding of all qualitative data; quantitative and qualitative data analysis; and report writing. The quantitative household survey was conducted among 2,003 households. The response rate was 98.5%, and individual interviewers completed an average of 5.8 interviews per day.

Impact Evaluation of Service Delivery Grants in the Health Sector in Cambodia: Baseline Survey