In this blogpost, Bruno Meessen (ITM, Antwerp) shares some of the key lessons he learned from the Argentinian Plan NACER. He compares this experience with the Performance Based Financing schemes that are being developed in Africa. Some interesting lessons...
My first real encounter with the Plan NACER (now renamed “Programa SUMAR”, since the expansion of the target group and the benefit packages) (1) was in December 2013, at a donor meeting on RBF in Oslo. Martin Sabignoso, the Lionel Messi of Performance Based Financing (see picture), presented the Argentinian experience to us in a very convincing way. I was therefore happy to accept an invitation by the World Bank to attend its annual “Results and implementation workshop for RBF” (25 March – 1 April) in Buenos Aires. It was time to discover the experience in the country itself. Here are my own take home lessons (2).
Commonalities between SUMAR and PBF …
The Programa SUMAR and what we call Performance-Based Financing (PBF) in Africa, have many things in common. For a start, they both seem to work. In Buenos Aires, we had the chance to listen to Paul Gertler (Berkeley University). Paul is a big name in the research community: among many other things, he proved the impact of the Oportunidades scheme in Mexico and of PBF in Rwanda. He shared his findings on the impact of the Plan NACER on several key health outcomes. The figures are impressive: the Plan reduced the number of babies with a low birth weight, by increasing prenatal care use and quality; combined with better care for low birth weight babies, this led to a reduction in neonatal mortality among the scheme beneficiaries. One of the most interesting results is the reduction of the inequity in terms of health status across provinces: the poorest ones have caught up with the average in terms of infant mortality rates. The strategy is also very cost-effective.
Another similarity between NACER and PBF is the transformative power of the strategies – a feature which is growingly recognized as a key strength of such schemes. In Argentina, this could be seen at many levels. For instance, the program accelerated the adoption of ICT in the health system. Another nice example is the fact that NACER transformed the public health subsystem from one to which people were implicitly belonging (free health care) and implicitly covered (no list of health care services) into a system with formalization of the entitlement (people have to enroll in the program) and an explicit package of health services (Plan NACER Health Care Package). This has forced the public subsystem to collect names of people, which is seen as an important step in the development of a close relationship between the system and each eligible household. (3)
More fundamentally, it has demonstrated to the rest of the health administration the importance of focusing on results. During the field visit, I asked Claudio Scalamogna, general coordinator of the Programa SUMAR in the Province of Chaco, whether the new style of management brought by the Plan could one day penetrate other social sectors. He answered “This is my dream”.
A last similarity I want to highlight is the narrative used to present the scheme. When one listens to Martin, you get the impression you’re listening to a PBF champion. This is of course interesting, as both strategies developed in parallel and independently. I asked Martin how he explains this; his answer was straightforward: “Results Based Financing is about human beings, how human beings can improve their work, and how we, as policy makers, can empower the health workers towards a prioritized goal. Our assessment in Argentina was that in order to improve health of the under-covered population, we had to redesign incentives to health facilities, improve work conditions, give autonomy, provide opportunities for training and for greater participation in the health system. We think that is the secret and I am not surprised it applies to other countries as well”.
But the differences are even more interesting…
A first difference: Argentina is a federal state and the Programa SUMAR reflects this reality. The program rests on a two-tier contractual system: there’s one contract between the national level and each province and another one between the province and health facilities. The contract between the central government and the province has two components: 60% is paid on a capitation basis (in 2013, around 2.6 US$ per person who has registered for the SUMAR– but existing enrollees count only if they used at least one health service during the last 12 months), 40% is performance-based using tracer indicators (see below). Initially, the whole funding was coming from the national level (with money from a loan granted by the World Bank). But since 2009, the provinces have also been co-financing the program, up to 15% nowadays. This share is expected to grow further. Martin told us that an efficient trick to keep control over such a two-level system is to use the national funding to remunerate the provincial unit teams. This allows the central level to define the profiles of members of the provincial units and to have a say on recruitment by provincial units.
The national government maintains some key decision rights: it defines the list of health services that are covered under the program (the benefit package) and the categories of costs which can be covered at facility level with the revenue collected from the program. But the rest is under the control of the province. For instance, each province defines the fee to be paid to the health facility for each service. While the program´s rules stipulates that facilities are allowed to allocate to staff incentives up to 50% of the program revenue, it is up to each province to decide whether that percentage will be 0 or 50. Of course, below the province, the health facilities are also granted key decision rights: thus the facility has the final decision on what to purchase (but with a variable degree of autonomy across provinces, as we could observe during our visit). We were told that in general, health facilities are prioritizing training opportunities, equipment and infrastructure improvements.
A second difference: more complex payment formulae
The National government puts the emphasis on 14 key interventions scientifically proven to have an impact on priority health problems. As quality is key for the effectiveness of these interventions, a lot of stress is put on fulfilling quality conditions. According to this principle, a province receives more resources as long as its performance is above defined targets. For example, if the province manages to give a first prenatal check-up before the 13th week to at least 29% of the eligible pregnant women, it will receive additional resources. The higher the percentage of women with early check-ups, the higher the retribution (It is a continuous payment function). Each tracer has a mandatory set of information regarding the quality of the health service to be reported for every case. For our example, the province must report – along with the name of the woman and her ID number – the date of the first check-up, the week of gestation, the date of the last menstrual period, the probable date of birth and, of course, the name of the health facility that provided the health care.
The contract between the Province and the facilities is simpler. They are paid on a fee-for-service basis (a price per quantity, as in PBF systems). For a selected set of health services they have also an incentive to match some standards for quality. All of this is subject of ex post verification (see below). Additionally, some provinces are using higher prices for health services that fulfill the quality matrix. In this case, they are extending the sanction scheme post verification in order to penalize the cases that do not match these criteria.
I discussed this approach with Gyuri Fritsche (World Bank). He reminded me that this approach – to pay for an output only if it has some key attributes of quality – was the initial strategy of the Belgian Technical Cooperation project in Rwanda (except that the BTC verified the fulfillment of quality attributes ex ante). This model had not been adopted for the scale up in Rwanda and elsewhere in Africa, as it was considered too difficult to bring to scale: collecting quantity and quality indicators separately was seen as much easier to implement – the former being collected monthly, the latter quarterly. The Argentinian system is indeed much more demanding in terms of routine data collection, but as we know, there is an ICT revolution going on. This is something we may have to reconsider in the future.
A third difference: allocation of the RBF revenue to individual bonuses is not the dominant model. Only seven out of twenty four provinces allow their facilities to pay incentives to the personnel. Martin told us that several reasons explain this heterogeneity. Some provinces do not want it for legal reasons, others because they don’t think direct incentives to personnel will bring better results; some also doubt they would be able to sustain a bonus scheme in the future. As a matter of fact, in the health centre we visited, the transfer received from SUMAR represented just 4% of the health centre wage bill. Not sure this would be enough to induce a dramatic change.
We were of course curious to know whether there was a difference in performance between the provinces which have allowed facilities to pay a bonus with the funding and those that haven’t. I asked the question to Paul and Martin. Paul explained that the outcome variables he used to prove the effectiveness of the NACER plan were very rare events; the sample sizes unfortunately did not allow to ascertain a possible positive contribution of individual bonuses. He is about to check this possible determinant with another data set. Martin’s assessment was that the main determinant of the surge in performance has been the expansion of autonomy at facility level: to have resources and be free to allocate them. As he put it nicely: “Health facilities have set up collective management approaches. RBF represents a real change in the culture of the health system, including at health facility level. For the first time health staff are participating in allocation decisions. This is also incentivizing!” The staff of the health centre we visited gave us another (complementary) explanation: the economic crisis of 2001 was really terrible and the health system experienced a collapse. The reinjection of funds brought by the NACER plan and the fact that the health facility staff had decision rights was a relief and created a new dynamic.
A fourth difference: a more important role for external verification. In PBF schemes, external verification is mainly there as a mechanism to reassure sponsors about the reliability of the verification system; it is henceforth called ‘counter-verification’. The External Concurrent Audit in Argentina is more integrated in the general functioning of the RBF scheme. For instance, the information gathered by the external audit firm feeds into the supervision program of the internal auditors. The permanence of the external audit allows the internal audit to focus on supervision, coaching and problem solving. This is something very valuable as this allows the program supervisor not to be perceived as controller. In Argentina, the external audit’s decisions have biding authority: if a discrepancy from the rules is found, no interference by authorities is possible: the penalty will have to be paid. This external audit is costly, but interestingly enough, the fines it charges to health facilities are nearly covering the full cost of doing the external audit.
A fifth difference: the management of Programa SUMAR involves a lot of people. Olivier Basenya (Ministry of Health, Burundi) and I were a bit shocked by the numbers mentioned when we heard them for the first time in Oslo: 150 staff at the national office of the program and 700 for the whole country! Compare: Burundi’s central level PBF team counts only 7 ministry of health staff and 3 staff affiliated to partners. Are too many people involved in SUMAR management?
As explained by Martin: “Our job is to develop the strategic planning, to provide support and to supervise the 7,000 health facilities involved in the program. We are also administrating the 24 legal performance contracts. Our team is therefore multidisciplinary: we have economists, physicians, lawyers, social psychologists, experts in communication, experts in information technology, accountants… Health progress is multi-causal, so you need to build multiple solutions, multi perspective solutions. We are always fine-tuning the strategy, trying to learn what the reality is saying to us. We are also very committed to training the provincial teams. SUMAR is also a program that integrates all other substantial programs. We are working with more than 30 health programs in our Ministry of Health”.
We also noticed many medical doctors in the health center we visited (well, most were absent, but they were reported as members of the team). So, it is possible that the relatively large team of the SUMAR program partly reflects a context-specific reality, but we agree that some PBF national units in Africa could be strengthened.
A last difference: the system is steered more dynamically
A big danger with a PBF system is self-satisfaction: you set up the system and then you run it more or less in a routine way. At the workshop, I checked with Olivier Basenya about his main take home message, and he duly agreed: “the Argentinian national unit steers its RBF system in a more dynamic way than we do”.
To conclude
Argentina was a refreshing experience for many PBF experts. Let’s keep in mind that RBF is not about implementing a single standard model. Although there are principles and good practices, there might be even better practices out there that we still have to discover. We learned a lot from Argentina during our stay in the country. Conversely, the Argentinian team is also eager to learn from other country experiences, so don’t hesitate to invite Martin.
(1) In this text, I refer to “NACER” when I refer to the initial scheme. I use “SUMAR” when I refer to what is in place today.
(2) I express my sincere gratitude to the SUMAR team, and particularly Mrs Jesica Azar who kindly answered my numerous questions on the way to the health center.
(3) I find this idea interesting and could be considered in some African countries. Let’s illustrate the concept with the case of Burundi. Each child under 5 would still receive treatment for free, but the health centre would be reimbursed by the national PBF scheme only if the health centre can provide for each utilization by a child his ‘free health care’ number. The great benefit of such a model is that it incentivizes health facilities to enroll families in the program, a way to develop a stronger bond with them. I also imagine that our ICT/PBF colleagues could develop solutions to provide information on utilization by enrollees to health centres (lost in follow-up, etc) to improve continuity of care.
Commonalities between SUMAR and PBF …
The Programa SUMAR and what we call Performance-Based Financing (PBF) in Africa, have many things in common. For a start, they both seem to work. In Buenos Aires, we had the chance to listen to Paul Gertler (Berkeley University). Paul is a big name in the research community: among many other things, he proved the impact of the Oportunidades scheme in Mexico and of PBF in Rwanda. He shared his findings on the impact of the Plan NACER on several key health outcomes. The figures are impressive: the Plan reduced the number of babies with a low birth weight, by increasing prenatal care use and quality; combined with better care for low birth weight babies, this led to a reduction in neonatal mortality among the scheme beneficiaries. One of the most interesting results is the reduction of the inequity in terms of health status across provinces: the poorest ones have caught up with the average in terms of infant mortality rates. The strategy is also very cost-effective.
Another similarity between NACER and PBF is the transformative power of the strategies – a feature which is growingly recognized as a key strength of such schemes. In Argentina, this could be seen at many levels. For instance, the program accelerated the adoption of ICT in the health system. Another nice example is the fact that NACER transformed the public health subsystem from one to which people were implicitly belonging (free health care) and implicitly covered (no list of health care services) into a system with formalization of the entitlement (people have to enroll in the program) and an explicit package of health services (Plan NACER Health Care Package). This has forced the public subsystem to collect names of people, which is seen as an important step in the development of a close relationship between the system and each eligible household. (3)
More fundamentally, it has demonstrated to the rest of the health administration the importance of focusing on results. During the field visit, I asked Claudio Scalamogna, general coordinator of the Programa SUMAR in the Province of Chaco, whether the new style of management brought by the Plan could one day penetrate other social sectors. He answered “This is my dream”.
A last similarity I want to highlight is the narrative used to present the scheme. When one listens to Martin, you get the impression you’re listening to a PBF champion. This is of course interesting, as both strategies developed in parallel and independently. I asked Martin how he explains this; his answer was straightforward: “Results Based Financing is about human beings, how human beings can improve their work, and how we, as policy makers, can empower the health workers towards a prioritized goal. Our assessment in Argentina was that in order to improve health of the under-covered population, we had to redesign incentives to health facilities, improve work conditions, give autonomy, provide opportunities for training and for greater participation in the health system. We think that is the secret and I am not surprised it applies to other countries as well”.
But the differences are even more interesting…
A first difference: Argentina is a federal state and the Programa SUMAR reflects this reality. The program rests on a two-tier contractual system: there’s one contract between the national level and each province and another one between the province and health facilities. The contract between the central government and the province has two components: 60% is paid on a capitation basis (in 2013, around 2.6 US$ per person who has registered for the SUMAR– but existing enrollees count only if they used at least one health service during the last 12 months), 40% is performance-based using tracer indicators (see below). Initially, the whole funding was coming from the national level (with money from a loan granted by the World Bank). But since 2009, the provinces have also been co-financing the program, up to 15% nowadays. This share is expected to grow further. Martin told us that an efficient trick to keep control over such a two-level system is to use the national funding to remunerate the provincial unit teams. This allows the central level to define the profiles of members of the provincial units and to have a say on recruitment by provincial units.
The national government maintains some key decision rights: it defines the list of health services that are covered under the program (the benefit package) and the categories of costs which can be covered at facility level with the revenue collected from the program. But the rest is under the control of the province. For instance, each province defines the fee to be paid to the health facility for each service. While the program´s rules stipulates that facilities are allowed to allocate to staff incentives up to 50% of the program revenue, it is up to each province to decide whether that percentage will be 0 or 50. Of course, below the province, the health facilities are also granted key decision rights: thus the facility has the final decision on what to purchase (but with a variable degree of autonomy across provinces, as we could observe during our visit). We were told that in general, health facilities are prioritizing training opportunities, equipment and infrastructure improvements.
A second difference: more complex payment formulae
The National government puts the emphasis on 14 key interventions scientifically proven to have an impact on priority health problems. As quality is key for the effectiveness of these interventions, a lot of stress is put on fulfilling quality conditions. According to this principle, a province receives more resources as long as its performance is above defined targets. For example, if the province manages to give a first prenatal check-up before the 13th week to at least 29% of the eligible pregnant women, it will receive additional resources. The higher the percentage of women with early check-ups, the higher the retribution (It is a continuous payment function). Each tracer has a mandatory set of information regarding the quality of the health service to be reported for every case. For our example, the province must report – along with the name of the woman and her ID number – the date of the first check-up, the week of gestation, the date of the last menstrual period, the probable date of birth and, of course, the name of the health facility that provided the health care.
The contract between the Province and the facilities is simpler. They are paid on a fee-for-service basis (a price per quantity, as in PBF systems). For a selected set of health services they have also an incentive to match some standards for quality. All of this is subject of ex post verification (see below). Additionally, some provinces are using higher prices for health services that fulfill the quality matrix. In this case, they are extending the sanction scheme post verification in order to penalize the cases that do not match these criteria.
I discussed this approach with Gyuri Fritsche (World Bank). He reminded me that this approach – to pay for an output only if it has some key attributes of quality – was the initial strategy of the Belgian Technical Cooperation project in Rwanda (except that the BTC verified the fulfillment of quality attributes ex ante). This model had not been adopted for the scale up in Rwanda and elsewhere in Africa, as it was considered too difficult to bring to scale: collecting quantity and quality indicators separately was seen as much easier to implement – the former being collected monthly, the latter quarterly. The Argentinian system is indeed much more demanding in terms of routine data collection, but as we know, there is an ICT revolution going on. This is something we may have to reconsider in the future.
A third difference: allocation of the RBF revenue to individual bonuses is not the dominant model. Only seven out of twenty four provinces allow their facilities to pay incentives to the personnel. Martin told us that several reasons explain this heterogeneity. Some provinces do not want it for legal reasons, others because they don’t think direct incentives to personnel will bring better results; some also doubt they would be able to sustain a bonus scheme in the future. As a matter of fact, in the health centre we visited, the transfer received from SUMAR represented just 4% of the health centre wage bill. Not sure this would be enough to induce a dramatic change.
We were of course curious to know whether there was a difference in performance between the provinces which have allowed facilities to pay a bonus with the funding and those that haven’t. I asked the question to Paul and Martin. Paul explained that the outcome variables he used to prove the effectiveness of the NACER plan were very rare events; the sample sizes unfortunately did not allow to ascertain a possible positive contribution of individual bonuses. He is about to check this possible determinant with another data set. Martin’s assessment was that the main determinant of the surge in performance has been the expansion of autonomy at facility level: to have resources and be free to allocate them. As he put it nicely: “Health facilities have set up collective management approaches. RBF represents a real change in the culture of the health system, including at health facility level. For the first time health staff are participating in allocation decisions. This is also incentivizing!” The staff of the health centre we visited gave us another (complementary) explanation: the economic crisis of 2001 was really terrible and the health system experienced a collapse. The reinjection of funds brought by the NACER plan and the fact that the health facility staff had decision rights was a relief and created a new dynamic.
A fourth difference: a more important role for external verification. In PBF schemes, external verification is mainly there as a mechanism to reassure sponsors about the reliability of the verification system; it is henceforth called ‘counter-verification’. The External Concurrent Audit in Argentina is more integrated in the general functioning of the RBF scheme. For instance, the information gathered by the external audit firm feeds into the supervision program of the internal auditors. The permanence of the external audit allows the internal audit to focus on supervision, coaching and problem solving. This is something very valuable as this allows the program supervisor not to be perceived as controller. In Argentina, the external audit’s decisions have biding authority: if a discrepancy from the rules is found, no interference by authorities is possible: the penalty will have to be paid. This external audit is costly, but interestingly enough, the fines it charges to health facilities are nearly covering the full cost of doing the external audit.
A fifth difference: the management of Programa SUMAR involves a lot of people. Olivier Basenya (Ministry of Health, Burundi) and I were a bit shocked by the numbers mentioned when we heard them for the first time in Oslo: 150 staff at the national office of the program and 700 for the whole country! Compare: Burundi’s central level PBF team counts only 7 ministry of health staff and 3 staff affiliated to partners. Are too many people involved in SUMAR management?
As explained by Martin: “Our job is to develop the strategic planning, to provide support and to supervise the 7,000 health facilities involved in the program. We are also administrating the 24 legal performance contracts. Our team is therefore multidisciplinary: we have economists, physicians, lawyers, social psychologists, experts in communication, experts in information technology, accountants… Health progress is multi-causal, so you need to build multiple solutions, multi perspective solutions. We are always fine-tuning the strategy, trying to learn what the reality is saying to us. We are also very committed to training the provincial teams. SUMAR is also a program that integrates all other substantial programs. We are working with more than 30 health programs in our Ministry of Health”.
We also noticed many medical doctors in the health center we visited (well, most were absent, but they were reported as members of the team). So, it is possible that the relatively large team of the SUMAR program partly reflects a context-specific reality, but we agree that some PBF national units in Africa could be strengthened.
A last difference: the system is steered more dynamically
A big danger with a PBF system is self-satisfaction: you set up the system and then you run it more or less in a routine way. At the workshop, I checked with Olivier Basenya about his main take home message, and he duly agreed: “the Argentinian national unit steers its RBF system in a more dynamic way than we do”.
To conclude
Argentina was a refreshing experience for many PBF experts. Let’s keep in mind that RBF is not about implementing a single standard model. Although there are principles and good practices, there might be even better practices out there that we still have to discover. We learned a lot from Argentina during our stay in the country. Conversely, the Argentinian team is also eager to learn from other country experiences, so don’t hesitate to invite Martin.
(1) In this text, I refer to “NACER” when I refer to the initial scheme. I use “SUMAR” when I refer to what is in place today.
(2) I express my sincere gratitude to the SUMAR team, and particularly Mrs Jesica Azar who kindly answered my numerous questions on the way to the health center.
(3) I find this idea interesting and could be considered in some African countries. Let’s illustrate the concept with the case of Burundi. Each child under 5 would still receive treatment for free, but the health centre would be reimbursed by the national PBF scheme only if the health centre can provide for each utilization by a child his ‘free health care’ number. The great benefit of such a model is that it incentivizes health facilities to enroll families in the program, a way to develop a stronger bond with them. I also imagine that our ICT/PBF colleagues could develop solutions to provide information on utilization by enrollees to health centres (lost in follow-up, etc) to improve continuity of care.
