Joël Arthur Kiendrébéogo
In this blogpost, Joël Arthur Kiendrébéogo (Burkina Faso, Centre Muraz) shares the most burning gaps in the research on Results-Based Financing (RBF), identified through group work by some participants at a satellite session of the Third Global Symposium on Health System Research (Cape Town, September 30 – October 3, 2014). The session was organized by the World Bank in partnership with the PBF Community of Practice.
The morning plenary session was an excellent introduction to our working group. Indeed, it was devoted to presenting the results of rigorous impact evaluations completed in Zimbabwe, Argentina and India, as well as that of research in progress (especially in Nigeria) or forthcoming (the implementation research program of the AHPSR). In the afternoon, we had breakout sessions. I joined the group dedicated to identifying the burning gaps that must be filled in the research agenda on RBF in order to generate more evidence and knowledge about the approach.
Four key gaps were identified
The Symposium being on health system research, not surprisingly there were a lot of researchers in our working group. Together, we identified four main priorities in terms of research agenda on RBF.
1. What are the systemic effects of RBF?
Some researchers have previously argued for the need to open the "black box" and this concern was also raised in our discussion. In my opinion it is crucial for the long-term sustainability of the RBF approach. Indeed this "openness" is important, not only to refine the design and implementation processes, but also to understand how the strategy interacts with the core building blocks of health systems at all levels. That understanding will allow RBF to become better and structurally integrated in these systems (and not to be seen as an isolated funding mechanism). Current research, however, is mostly focused on measuring impact on quantity and quality (by using proxy indicators) of healthcare services provided, without really trying to understand how and why these results are obtained. There is a definite need for practitioners to have thorough knowledge of the real impact (intended or unintended, short and long term) of RBF, beyond simple theories of change, on:
· Organizational changes and institutional arrangements in health systems: governance and leadership; contracting processes; resources use and reallocation; accountability; transparency; autonomy and entrepreneurship; redistribution of power; possible articulations, synergies and tensions with other health funding mechanisms or with other types of reforms such as decentralization and implementation challenges.
· Healthcare providers: intrinsic and extrinsic motivation, behavioral change (to fill the three gaps: the know gap, the know-can gap, the can-do gap; or news skills developed to adapt to the new situation).
· Services delivery: the actual "added value" of RBF in improving quality of care (including health system responsiveness and user satisfaction) and community involvement.
· Important dimensions of health systems: such as equity, protection against financial risk and efficiency (allocative and technical), globally RBF relations with universal health coverage.
2. What are the economic and financial implications of RBF?
The actual costs of RBF implementation are not well known. We need to accumulate a much better understanding of them, especially with the prospect of scaling up: better distinguish investment and running costs, identify how to minimize monitoring costs and to seize economies of scale… The cost-effectiveness and cost-benefit analysis constitute a missing link in the RBF research agenda.
3. What is the role of private sector in RBF?
The involvement of the private sector (besides faith-based health facilities contracted by the State to deliver health services) in RBF programs is still faltering in many countries. The private sector is however a key player in the provision of health services and more research may shed light on the sector’s comparative advantages and how these can be used in RBF strategy.
4. What are the methodological challenges for research teams?
RBF raises specific scientific challenges. Several of them have already been identified in the recent past : the first Cochrane review and the following debate on the PBF Community of Practice online forum in 2012) and the scientific workshop the following year in Bergen (Norway). Maybe the most important is to remember that RBF, a complex health policy, gets into a complex adaptive system (the health system) embedded in a specific and dynamic context. It is crucial to conduct research iteratively to reflect these continuous changes and to use mixed methods (quantitative and qualitative). Researchers must also work to refine existing methods and techniques or seek to develop new ones.
In Cape Town, we also discussed the question of the independence of research teams: it is an important one because it addresses the credibility of scientific evidence produced and the ownership of research results by countries
An apprehension beyond all these questions
Any RBF impact evaluation rests on a comparison between a group of health producers (e.g. health facilities, households…) benefiting from the RBF intervention and a control group. Our working group in Cape Town raised the recurring and still unanswered question about what would be the most appropriate status for the control group: no intervention at all, equal money with an input based contract, equal money with another strategy resting on another concurrent theory of change?
This is not an academic question: our responsibility is also to find the right mix of policy instruments, including provider payment mechanisms.
Personally, I believe that our countries are heading towards mixed provider payment systems. In general, the concern about sustainability (at the cessation of external funding for RBF and/or when funds will be directed to other strategies) should lead governments to make budget reallocations or work to expand fiscal space for health, with innovative financing for instance. But are policymakers prepared for that and are health systems ready?
Panel discussion
We had the opportunity to report our recommendations in the plenary session. As a young researcher looking for data for my PhD, I heard something from the World Bank I was not aware of: RBF databases of countries for which it funds impact evaluations are a public good and will be made available to all (after a limited period of exclusivity for the team in charge of the study). We can hope that this decision (that we hope actual very soon) will boost the research agenda on RBF. As it was clear from our working group, there are indeed many questions which still need to be answered.
In this blogpost, Joël Arthur Kiendrébéogo (Burkina Faso, Centre Muraz) shares the most burning gaps in the research on Results-Based Financing (RBF), identified through group work by some participants at a satellite session of the Third Global Symposium on Health System Research (Cape Town, September 30 – October 3, 2014). The session was organized by the World Bank in partnership with the PBF Community of Practice.
The morning plenary session was an excellent introduction to our working group. Indeed, it was devoted to presenting the results of rigorous impact evaluations completed in Zimbabwe, Argentina and India, as well as that of research in progress (especially in Nigeria) or forthcoming (the implementation research program of the AHPSR). In the afternoon, we had breakout sessions. I joined the group dedicated to identifying the burning gaps that must be filled in the research agenda on RBF in order to generate more evidence and knowledge about the approach.
Four key gaps were identified
The Symposium being on health system research, not surprisingly there were a lot of researchers in our working group. Together, we identified four main priorities in terms of research agenda on RBF.
1. What are the systemic effects of RBF?
Some researchers have previously argued for the need to open the "black box" and this concern was also raised in our discussion. In my opinion it is crucial for the long-term sustainability of the RBF approach. Indeed this "openness" is important, not only to refine the design and implementation processes, but also to understand how the strategy interacts with the core building blocks of health systems at all levels. That understanding will allow RBF to become better and structurally integrated in these systems (and not to be seen as an isolated funding mechanism). Current research, however, is mostly focused on measuring impact on quantity and quality (by using proxy indicators) of healthcare services provided, without really trying to understand how and why these results are obtained. There is a definite need for practitioners to have thorough knowledge of the real impact (intended or unintended, short and long term) of RBF, beyond simple theories of change, on:
· Organizational changes and institutional arrangements in health systems: governance and leadership; contracting processes; resources use and reallocation; accountability; transparency; autonomy and entrepreneurship; redistribution of power; possible articulations, synergies and tensions with other health funding mechanisms or with other types of reforms such as decentralization and implementation challenges.
· Healthcare providers: intrinsic and extrinsic motivation, behavioral change (to fill the three gaps: the know gap, the know-can gap, the can-do gap; or news skills developed to adapt to the new situation).
· Services delivery: the actual "added value" of RBF in improving quality of care (including health system responsiveness and user satisfaction) and community involvement.
· Important dimensions of health systems: such as equity, protection against financial risk and efficiency (allocative and technical), globally RBF relations with universal health coverage.
2. What are the economic and financial implications of RBF?
The actual costs of RBF implementation are not well known. We need to accumulate a much better understanding of them, especially with the prospect of scaling up: better distinguish investment and running costs, identify how to minimize monitoring costs and to seize economies of scale… The cost-effectiveness and cost-benefit analysis constitute a missing link in the RBF research agenda.
3. What is the role of private sector in RBF?
The involvement of the private sector (besides faith-based health facilities contracted by the State to deliver health services) in RBF programs is still faltering in many countries. The private sector is however a key player in the provision of health services and more research may shed light on the sector’s comparative advantages and how these can be used in RBF strategy.
4. What are the methodological challenges for research teams?
RBF raises specific scientific challenges. Several of them have already been identified in the recent past : the first Cochrane review and the following debate on the PBF Community of Practice online forum in 2012) and the scientific workshop the following year in Bergen (Norway). Maybe the most important is to remember that RBF, a complex health policy, gets into a complex adaptive system (the health system) embedded in a specific and dynamic context. It is crucial to conduct research iteratively to reflect these continuous changes and to use mixed methods (quantitative and qualitative). Researchers must also work to refine existing methods and techniques or seek to develop new ones.
In Cape Town, we also discussed the question of the independence of research teams: it is an important one because it addresses the credibility of scientific evidence produced and the ownership of research results by countries
An apprehension beyond all these questions
Any RBF impact evaluation rests on a comparison between a group of health producers (e.g. health facilities, households…) benefiting from the RBF intervention and a control group. Our working group in Cape Town raised the recurring and still unanswered question about what would be the most appropriate status for the control group: no intervention at all, equal money with an input based contract, equal money with another strategy resting on another concurrent theory of change?
This is not an academic question: our responsibility is also to find the right mix of policy instruments, including provider payment mechanisms.
Personally, I believe that our countries are heading towards mixed provider payment systems. In general, the concern about sustainability (at the cessation of external funding for RBF and/or when funds will be directed to other strategies) should lead governments to make budget reallocations or work to expand fiscal space for health, with innovative financing for instance. But are policymakers prepared for that and are health systems ready?
Panel discussion
We had the opportunity to report our recommendations in the plenary session. As a young researcher looking for data for my PhD, I heard something from the World Bank I was not aware of: RBF databases of countries for which it funds impact evaluations are a public good and will be made available to all (after a limited period of exclusivity for the team in charge of the study). We can hope that this decision (that we hope actual very soon) will boost the research agenda on RBF. As it was clear from our working group, there are indeed many questions which still need to be answered.
