Financing Health in Africa - Le blog
  • Home
  • Bloggers
  • Collaborative projects
  • Join our COPs
  • Resources
  • About Us
  • Contact Us

Verification of results: the experience of Zimbabwe

1/26/2017

3 Commentaires

 
Erik Josephson
Verifying results before payment is a core principle of Results-Based Financing (RBF). But verification is also costly. In a recent blog post, Matthieu Antony called for less intensive approaches. Some countries are moving in this direction. Erik Josephon reports on the approach adopted in Zimbabwe. 
Picture
Results-based financing (RBF) mechanisms have been growing very quickly since Cambodia first initiated one in 1999. They are now commonplace across Africa, in several Latin American countries and several OECD countries also have versions of RBF mechanisms. One of the core principles of RBF is to verify results prior to payment. In many settings the methodology put in place to achieve this has been to verify the results of all quantity indicators, and to conduct an assessment of quality, at every health facility at least once per quarter. However with increasing scrutiny of the non-incentive operating cost of RBF mechanisms, the intensity of verification is becoming the topic of discussion across the RBF community, and in health policy-making circles.

Towards a less costly verification

Specifically questions are being asked as to how to construct a verification methodology which is less onerous, and less costly, but which still provides close to the same level of trust that the services being paid for were actually provided.

In certain settings, such as Afghanistan, sampling has been used to reduce the intensity of verification each quarter. This is certainly an avenue worth pursuing, albeit that non-risk-based sampling does come with disadvantages, such as the possibility that some facilities won’t be visited as often as they need to be. Where the sampling is random, the methodology isn’t designed to focus where the problems may be. After all low data management capacity is not universal – some facilities will have stronger staff than others – and the desire to commit fraud will only be present in a select few facilities. Random sampling does not take these factors into consideration.

A different way of looking at verification is to design the methodology such that it specifically takes into account that error in declaration, resulting from whatever motivation, will not be uniformly distributed across health facilities. That is to say, errors will be larger and more persistent over time in some facilities. With this in mind some examples of what is being called risk-based verification are springing up. This has been happening in Argentina for some years, and is based on the volume of patients at each facility. Another such example is in Zimbabwe.

The evolving verification in Zimbabwe

During a visit to Harare in September 2016, I sat with Dr. Tafadzwa Goverwa who is a Provincial Medical Director, Chenjerai Sisimayi of the World Bank and Arjanne Rietsema of Cordaid. Dr Goverwa and Arjanne Riestema helpfully tag-teamed to set the context of verification in the Zimbabwean RBF.

Dr Goverwa started by explaining how verification was originally conducted in Zimbabwe. Verification was on a monthly basis, with all indicators verified. It was a heavy verification – every facility, every indicator, every month. We applied sanctions to errors in declaration since the start of the program. Arjanne Rietsema clarified that if there were differences between declared and verified of more than 5%, that specific indicator would not be paid at all. This verification was carried out by Cordaid. Dr Goverwa then explained that the reason for changing from the old methodology was largely financial. The sustainability of the program was on their minds. Going to all facilities every month entails a significant cost.

Arjanne pointed out that during the monthly, all facility verification, they weren’t finding that many errors, and started to wonder at the utility of this intensity of verification. The needs changed. At the beginning the verifiers would also build capacity on operational plans, and the workings of the health center committee. Over a period of time, this capacity building became less necessary as facility staff took on their responsibilities.

This last point is important when thinking about the evolution of verification, and particularly its frequency. Verifiers have from the start of RBF also in some cases been supervisors. In order to encourage district or county supervisors to carry out their role, and often to have the resources to do so, the first order verification role has been given to district or county supervisors. With time this dual role can sometimes be seen as conflicted.

So initially in Zimbabwe they conducted monthly verification both because of the capacity building needs, and also because they suspected the level of error between declared and verified results would warrant it. About two years into the program, they noted the level of error wasn’t as large any more, and so they started thinking about a lighter verification methodology. However rather than simply lessen the frequency, they had other ideas.

The adoption of risk-based verification

As part of the preparation for this interview, I also spoke with Petra Vergeer of the World Bank. She explained that around the time that Zimbabwe was considering changing verification approaches, she and three colleagues (including the author of this blog) were in the midst of writing a cross-case analysis of six country case studies on verification in RBF. Petra shared the findings and recommendations from the cross-case analysis with the team in Zimbabwe, and one of the recommendations was to introduce risk-based verification. Petra helped the team think through what triggers to use, based on lessons learned from, for example, Argentina and the United Kingdom.

Dr Goverwa explained that in the risk-based methodology the Zimbabwe team settled on, the review of differences between declared and verified data occurs every six months (in June and December according to the Zimbabwe verification manual). If the average of errors has been under 5% between declared and verified for a given health facility, they are classified as green. Error means the verified result is different positively or negatively from the declared result. Green faciliites are verified once per quarter, and during that visit results are checked in full for one of the months in the foregoing quarter, and a lighter verification is carried out for the other two months in the quarter. The lighter verification involves spot checks of high-volume indicators, or indicators where the volume has moved by more than 20% from the previous month. Where a facility has had an average error rate of between 5% and 10%, they are classified as amber. Amber facilities are visited twice per quarter. Facilities with an average error rate above 10% are classified as red, as are new health facilities, and hospitals due to their high volumes. Red facilities are visited every month.

I remarked that the analytical aspect of this, e.g. identifying which facilities had had errors in each category, looking for high-volume indicators and checking trends between months would be much simplified with a dedicated IT solution. Chenjerai Sisimayi explained that for now, the data is downloaded from the DHIS2 software which is used as a health information system in Zimbabwe, and the analysis is done in Excel. There is a desire, but currently no plans, to create software which would do much of the analytical heavy lifting.

RBF currently covers 394 health facilities in Zimbabwe. As of September 2016, 319 facilities were in the green category, 51 were in the amber, and 34 were in the red, meaning that only 10% of facilities are in the red category. Arjanne Rietsema noted that only a few rural health centers are in that category, and that most of the red facilities are hospitals, which stay red by design, given the financial risk resulting from them receiving large RBF payments. She also noted that they have wanted to include in the verification process an activity to pick one non-incentivized indicator at random and include that in the assessment of error, however this has not yet been implemented. Quality verification is carried out quarterly in all cases, with no risk categorization.

Real benefits

I asked how the revised verification methodology had impacted on cost. Arjanne Rietsema said that a limited assessment had been done, and it had shown an 80% reduction in vehicle costs. And as Chenjerai Sisimayi remarked, the improvement in the quality of data entry has continued in the context of the risk-based verification.

Some within the RBF community would posit that over time, the intensity of verification can reduce because the mindsets of health facility managers changes and they realize that correctly declared data has multiple advantages. Dr Goverwa noted that health facility managers in Zimbabwe now know that incorrectly entered data will result in having to correct it down the line. As a result registers have become more complete, where entire columns used to be empty, they are now filled in. Arjanne Rietsema added that there is in some districts an intentional competition created between health facilities, as their color status is made known publicly. So moving from red to amber, and amber to green can be the source of pride. And the reverse is also true.

Another important element is the sanction policy. Any indicator with a difference between declared and verified of more than 5% is not paid. I mentioned that this is very strict compared to most other RBF implementations where there may be sanctions which are rarely implemented, and others where there are no sanctions at all, in policy or in practice. I asked whether there had been resistance to the sanction policy at the start. Arjanne’s response was that since the sanction policy came in at the start of RBF, no-one questioned it. From the point of view of facility managers, this was part and parcel of the RBF approach. And they very quickly understood its implications on their RBF payments.

It’s worth noting that although Zimbabwe has moved to a risk-based methodology, they have effectively moved from a very heavy initial verification methodology, to one which would still be considered heavy in some settings. Even green facilities are visited once per quarter, which in other places is the starting point from which regulators and purchasers are trying to lighten the burden.

Zimbabwe is blazing a trail for verification in RBF in LICs and LMICs. There will very likely be a number of countries looking to follow suit, with Benin already known to be among them. Verification is a key part of RBF and particularly the theory of change of RBF. Thinking through how to make verification, and with it RBF in general, more cost-effective is an important challenge, and is particularly important for quality of care, which is far more open to gaming than is quantity of services delivered.
3 Commentaires
Manga Danlami Haruna
2/4/2017 10:17:35 am

What a great idea toward achieving RBF intervention in the world. For a country that is managing over 1000 health facilities contract in only 3 state such risk-based verficatin will difficult to handle. However, we hope to be there.

Répondre
Vera Mussah
2/6/2017 01:30:49 pm

Thanks Eric for briliiantly writing this blog. I have a clear understanding of what risk based verification is all about from this blog. I would like to ask a few questions:
1. Who conducts the quantify verification? Is it the MOH or Cordaid? Or is it a separate entity that conducts this exercise?
2. Is the quality verification conducted separately from the quantity? What is the cost implication?
3. Who are those that conduct the verification? Is it a mixed team of clinicians and data officers?
4. Are there meetings held to review the outcome of the results from the verification exercise?

Thanks,

Vera-Liberia

Répondre
Arjanne Rietsema link
2/16/2017 12:01:11 pm

Dear Vera. Greetings from Cordaid in Zimbabwe. During the first years Cordaid conducted the quantity verification.By the end of last year the MOH started doing what we call a "first level verification". And Cordaid as the external verifier, verifies 50% of the sample that has been verified by the MOH. Quality verification is conducted by the District Health Teams. They receive USD 60 per health facility per quarter. The team comprises of different disciplines, including a clinician. The MOH is currently reflecting on how to make the quality verification more objective. Does anyone have suggestions? The District team does take time for a debriefing after the visit, but follow up on recommendations can be improved. The MOH is currently experimenting a CQI intervention, whereby health facilities receive monthly coaching visits to follow up on proposed quality improvements. I hope this clarifies.

Répondre



Laisser un réponse.


    Our websites

    Photo
    Photo
    Photo

    We like them...

    SINA-Health
    International Health Policies
    CGD

    Archives

    Septembre 2019
    Juin 2019
    Avril 2019
    Mars 2019
    Mai 2018
    Avril 2018
    Mars 2018
    Février 2018
    Janvier 2018
    Décembre 2017
    Octobre 2017
    Septembre 2017
    Août 2017
    Juillet 2017
    Juin 2017
    Mai 2017
    Avril 2017
    Mars 2017
    Février 2017
    Janvier 2017
    Décembre 2016
    Novembre 2016
    Octobre 2016
    Septembre 2016
    Août 2016
    Juillet 2016
    Avril 2016
    Mars 2016
    Février 2016
    Janvier 2016
    Décembre 2015
    Novembre 2015
    Octobre 2015
    Septembre 2015
    Août 2015
    Juillet 2015
    Juin 2015
    Mai 2015
    Avril 2015
    Mars 2015
    Février 2015
    Janvier 2015
    Décembre 2014
    Octobre 2014
    Septembre 2014
    Juillet 2014
    Juin 2014
    Mai 2014
    Avril 2014
    Mars 2014
    Février 2014
    Janvier 2014
    Décembre 2013
    Novembre 2013
    Octobre 2013
    Septembre 2013
    Août 2013
    Juillet 2013
    Juin 2013
    Mai 2013
    Avril 2013
    Mars 2013
    Février 2013
    Janvier 2013
    Décembre 2012
    Novembre 2012
    Octobre 2012
    Septembre 2012
    Août 2012
    Juillet 2012
    Juin 2012
    Mai 2012
    Avril 2012
    Mars 2012
    Février 2012
    Janvier 2012
    Décembre 2011
    Novembre 2011
    Octobre 2011

    Tags

    Tout
    2012
    Accountability
    Aid
    Alex Ergo
    Assurance Maladie
    Bad
    Bamako Initiative
    Bénin
    Bruno Meessen
    Burkina Faso
    Burundi
    Civil Society
    Communauteacute-de-pratique
    Communauté De Pratique
    Community Of Practice
    Community Participation
    Conference
    Cop
    Course
    Couverture Universelle
    CSU
    Déclaration De Harare
    Divine Ikenwilo
    Dr Congo
    économie Politique
    élections
    équité
    Equity
    Fbp
    Financement Basé Sur Les Résultats
    Financement Public
    Fragilité
    Fragility
    Free Health Care
    Global Fund
    Global Health Governance
    Gratuité
    Gratuité
    Health Equity Fund
    Health Insurance
    ICT
    Identification Des Pauvres
    Isidore Sieleunou
    Jb Falisse
    Jurrien Toonen
    Kenya
    Knowledge-management
    Kouamé
    Leadership
    Mali
    Management
    Maroc
    Maternal And Child Health
    Médicaments
    Mise En Oeuvre
    Mutuelle
    National Health Accounts
    Ngo
    Niger
    Omd
    OMS
    Parlement
    Participation Communautaire
    Pba
    Pbf
    Plaidoyer
    Policy Process
    Politique
    Politique De Gratuité
    Politique De Gratuité
    Post Conflit
    Post-conflit
    Private Sector
    Processus Politique
    Qualité Des Soins
    Qualité Des Soins
    Quality Of Care
    Recherche
    Redevabilité
    Reform
    Réforme
    Research
    Results Based Financing
    Rwanda
    Santé Maternelle
    Secteur Privé
    Sénégal
    Société Civile
    Uganda
    Universal Health Coverage
    User Fee Removal
    Voeux 2012
    Voucher
    WHO

Powered by Create your own unique website with customizable templates.