Igna Bonfrer & Ellen Van de Poel
The end of 2014 has been marked by several scientific publications on the experience of Performance Based Financing (PBF) in Burundi. Igna Bonfrer and Ellen Van de Poel (Erasmus University Rotterdam) summarize some of the key findings of their own research. PBF brings benefits to the population, but there are still areas for improvement.
With the start of 2015, we can gear up for what promises to be an important year for international health care financing. We will see the closing of the Millennium Development Goals and the start of the Sustainable Development Goals. The latter are expected to include the aim to achieve Universal Health Coverage (UHC): providing good quality care to everyone who needs it, without causing financial hardship. PBF could be an important tool to fasten progress towards UHC by incentivizing the delivery of quality health care, but attention needs to be paid that services remain accessible for the poorest households.
Enthusiasm for performance based financing
Over the last decade, PBF has attracted interest from policy makers across low- and middle income countries. In Africa alone, more than 35 countries are in the process of introducing payment methods that reward performance. While there is great enthusiasm among policy makers and practitioners about the achievements of PBF, robust evidence on its effects in LMIC remains limited. We are therefore happy to share the findings of two of the recently published studies evaluating the effects of PBF in Burundi.
Evidence on its effects in Burundi
We evaluated the effects of PBF on maternal and child care utilization and quality. In 2008, PBF was introduced for a range of health care services in different pilot provinces over time and subsequently a roll-out to nationwide level took place in April 2010. We first estimated the effects of PBF in nine pilot provinces out of the 17 provinces in total, using data collected by the Dutch aid organization Cordaid in these nine provinces. We then used the nationally representative Burundi Demographic and Health Survey (BDHS containing data over the period 2005-2010) to estimate effects at the nationwide level from all pilot provinces and the first four to nine months of nationwide roll-out. Our data allowed us to study the initial 2.5 years of PBF in Burundi.
Promising findings from pilots but limited effects at nationwide level
The findings from the nine pilot provinces are promising but the subsequent study at nationwide level showed considerably smaller effects. Let’s take a look at some details. While in 2006, before PBF was implemented, 62 out of 100 live births took place in a health care facility, this increased with 36 percent in the nine pilot provinces as a result of the introduction of PBF. In these pilot provinces, PBF was also found to double the proportion of women reporting to use modern family planning methods, to 18 percent, which means there likely still is considerable unmet need. The overall quality score based on external audits of the structure and process quality of facilities increased with 45 percent as a result of these PBF pilots. In the nationwide study we also investigated quality of care by looking at what happened during antenatal care visits, based on mothers’ reports, and found that the behaviour of health care providers changed to become more in line with international antenatal care guidelines (blood pressure measurement and anti-tetanus vaccinations). While the improvement in the overall quality score was large and significant, this was not confirmed by self-reported patient satisfaction. We also found no sustained effects on antenatal care use, vaccinations and the use of bed nets in the pilot provinces.
Our nationwide study using data from all provinces, confirmed some of the above findings, though effects were smaller in size. It was especially striking that the large effects on births taking place in a health care facility found in the nine pilot provinces was not replicated when making a nationwide comparison, using the BDHS data. When reducing these data to only include the nine pilot provinces, we did find a positive effect but in size this effect is only one fourth of the effect found using the data collected by Cordaid. This suggests that some of the differences in the results between both studies are driven by the selection of the nine pilot provinces: apparently improvements were easier to make in these specific pilot provinces and maybe those involved there were more motivated. However, this still does not explain the largest share of the differences in the effects in both studies. At this stage we can only guess which other factors drive the smaller effect sizes in the nationwide study.
Initiation of care may require more effort
Overall our findings suggest that PBF has had some strong positive impacts when using the data from the nine pilot provinces but not so much when making a nationwide comparison, over the initial 2.5 years of implementation. Especially for types of care which require a behavioural change of health care workers when the patient is already in the facility we see improvements. Improvements are smaller for services which require effort from the provider to change patients’ utilization choices. Gertler & Vermeersch (2012) also found that the Rwandan PBF programme was more successful to increase utilization of services that are under the provider’s control, rather than those that depend on patients’ choices. Initiation of care may require more effort than continuation; an initial visit could therefore in the future be rewarded with a higher payment to facilities than the follow-up visits.
Benefits to the poor
With a supply side intervention like PBF, one of the main worries is that the poor do not benefit sufficiently. Indeed, when studying only the poorest forty percent of the population, we find in most cases that effects are similar or smaller compared to those in the general population. So far the PBF policy in Burundi did not include elements to target the poor, investments in specific strategies for this group are therefore likely to be needed to improve equity in health and health care.
Conclusion
So, does the evidence from these two studies endorse the enthusiasm for PBF in the case of Burundi? Based on the data from the nine pilot provinces where Cordaid was involved, the scientific evidence is supportive. When estimating the effects based on data from all 17 provinces, the effects of PBF on the outcomes that we have studied for the pilot period and the subsequent initial months of nationwide roll-out, are also positive in most cases but less impressive. Our studies provide some supportive evidence for the many countries engaged in piloting PBF to sustain their efforts, but also indicate that the assumption that PBF works with any indicator requires more scrutiny and that the concern for equity should be better integrated into PBF schemes.
The end of 2014 has been marked by several scientific publications on the experience of Performance Based Financing (PBF) in Burundi. Igna Bonfrer and Ellen Van de Poel (Erasmus University Rotterdam) summarize some of the key findings of their own research. PBF brings benefits to the population, but there are still areas for improvement.
With the start of 2015, we can gear up for what promises to be an important year for international health care financing. We will see the closing of the Millennium Development Goals and the start of the Sustainable Development Goals. The latter are expected to include the aim to achieve Universal Health Coverage (UHC): providing good quality care to everyone who needs it, without causing financial hardship. PBF could be an important tool to fasten progress towards UHC by incentivizing the delivery of quality health care, but attention needs to be paid that services remain accessible for the poorest households.
Enthusiasm for performance based financing
Over the last decade, PBF has attracted interest from policy makers across low- and middle income countries. In Africa alone, more than 35 countries are in the process of introducing payment methods that reward performance. While there is great enthusiasm among policy makers and practitioners about the achievements of PBF, robust evidence on its effects in LMIC remains limited. We are therefore happy to share the findings of two of the recently published studies evaluating the effects of PBF in Burundi.
Evidence on its effects in Burundi
We evaluated the effects of PBF on maternal and child care utilization and quality. In 2008, PBF was introduced for a range of health care services in different pilot provinces over time and subsequently a roll-out to nationwide level took place in April 2010. We first estimated the effects of PBF in nine pilot provinces out of the 17 provinces in total, using data collected by the Dutch aid organization Cordaid in these nine provinces. We then used the nationally representative Burundi Demographic and Health Survey (BDHS containing data over the period 2005-2010) to estimate effects at the nationwide level from all pilot provinces and the first four to nine months of nationwide roll-out. Our data allowed us to study the initial 2.5 years of PBF in Burundi.
Promising findings from pilots but limited effects at nationwide level
The findings from the nine pilot provinces are promising but the subsequent study at nationwide level showed considerably smaller effects. Let’s take a look at some details. While in 2006, before PBF was implemented, 62 out of 100 live births took place in a health care facility, this increased with 36 percent in the nine pilot provinces as a result of the introduction of PBF. In these pilot provinces, PBF was also found to double the proportion of women reporting to use modern family planning methods, to 18 percent, which means there likely still is considerable unmet need. The overall quality score based on external audits of the structure and process quality of facilities increased with 45 percent as a result of these PBF pilots. In the nationwide study we also investigated quality of care by looking at what happened during antenatal care visits, based on mothers’ reports, and found that the behaviour of health care providers changed to become more in line with international antenatal care guidelines (blood pressure measurement and anti-tetanus vaccinations). While the improvement in the overall quality score was large and significant, this was not confirmed by self-reported patient satisfaction. We also found no sustained effects on antenatal care use, vaccinations and the use of bed nets in the pilot provinces.
Our nationwide study using data from all provinces, confirmed some of the above findings, though effects were smaller in size. It was especially striking that the large effects on births taking place in a health care facility found in the nine pilot provinces was not replicated when making a nationwide comparison, using the BDHS data. When reducing these data to only include the nine pilot provinces, we did find a positive effect but in size this effect is only one fourth of the effect found using the data collected by Cordaid. This suggests that some of the differences in the results between both studies are driven by the selection of the nine pilot provinces: apparently improvements were easier to make in these specific pilot provinces and maybe those involved there were more motivated. However, this still does not explain the largest share of the differences in the effects in both studies. At this stage we can only guess which other factors drive the smaller effect sizes in the nationwide study.
Initiation of care may require more effort
Overall our findings suggest that PBF has had some strong positive impacts when using the data from the nine pilot provinces but not so much when making a nationwide comparison, over the initial 2.5 years of implementation. Especially for types of care which require a behavioural change of health care workers when the patient is already in the facility we see improvements. Improvements are smaller for services which require effort from the provider to change patients’ utilization choices. Gertler & Vermeersch (2012) also found that the Rwandan PBF programme was more successful to increase utilization of services that are under the provider’s control, rather than those that depend on patients’ choices. Initiation of care may require more effort than continuation; an initial visit could therefore in the future be rewarded with a higher payment to facilities than the follow-up visits.
Benefits to the poor
With a supply side intervention like PBF, one of the main worries is that the poor do not benefit sufficiently. Indeed, when studying only the poorest forty percent of the population, we find in most cases that effects are similar or smaller compared to those in the general population. So far the PBF policy in Burundi did not include elements to target the poor, investments in specific strategies for this group are therefore likely to be needed to improve equity in health and health care.
Conclusion
So, does the evidence from these two studies endorse the enthusiasm for PBF in the case of Burundi? Based on the data from the nine pilot provinces where Cordaid was involved, the scientific evidence is supportive. When estimating the effects based on data from all 17 provinces, the effects of PBF on the outcomes that we have studied for the pilot period and the subsequent initial months of nationwide roll-out, are also positive in most cases but less impressive. Our studies provide some supportive evidence for the many countries engaged in piloting PBF to sustain their efforts, but also indicate that the assumption that PBF works with any indicator requires more scrutiny and that the concern for equity should be better integrated into PBF schemes.
