Financing Health in Africa - Le blog
  • Home
  • Bloggers
  • Collaborative projects
  • Join our COPs
  • Resources
  • About Us
  • Contact Us

An online debate on "Performance-based financing in low- and middle-income countries: still more questions than answers"

9/24/2012

1 Commentaire

 
One of the two editorials of the August issue of the WHO Bulletin was dedicated to Performance-Based Financing (PBF). Fretheim et al. reported the main findings of their Cochrane review published a few months earlier. In the weeks that followed – despite many experts being on holidays and the summer “torpor” – a discussion developed on the online discussion forum of the PBF Community of Practice (CoP). The discussion benefited from contributions by two authors of the review (Atle Fretheim and Sophie Witter). This blog post summarizes the main points of the discussion. It tries to be as objective as possible; you can access the whole exchange of emails on the Resources page of this blog.

Collection, editing, titling and introduction: Bruno Meessen, September 12th 2012.

Launch of the debate

On August 9th, Emmanuel Ngabire (School of Public Health, Kigali), co-facilitator of the CoP, shared the article on the Google group. The first response came two hours later, from Stefaan Van Bastelaere (Belgium Technical Cooperation, Brussels). He expressed his concern on the possible impact of the editorial and his frustration: “The authors reduce PBF to a strategy that generates ‘quantities’, which in my opinion is unfair.”   

Bruno Meessen (Institute of Tropical Medicine, Antwerp), lead facilitator of the CoP, then invited members of the discussion group to read the paper and share their views.

Longin Gashubije (Ministry of Health of Burundi, Bujumbura) questioned the narrow definition of PBF used by Fretheim et al. in the editorial: “I think PBF is more than the transfer of money; when well implemented, it allows to transform the whole health system”. He also explained why no randomized study was done in Burundi: the mere observation, through the monitoring system, of improvement of quantitative and qualitative indicators (that had never improved before the implementation of the PBF) turned out to be sufficient to persuade decision-makers to scale up the pilot experience. 

Authors of the review are willing to engage in dialogue

On August 10th, Atle Fretheim (Norwegian Knowledge Center for the Health Services, Oslo) joined the group. After introducing himself, he recommended members of the discussion group to read the Cochrane review as well, not only the editorial. He expressed his willingness to engage in a dialogue on how PBF schemes might be evaluated in a practical and feasible, but robust way.

On the same day, Bruno Meessen reacted. The accumulated frustration was palpable – much of his long mail concerns the researchers’ lack of interest (so far) in setting up a real dialogue with PBF implementers. As far as the systematic review is concerned, a better connection with the field and the implementers would have prevented some misunderstandings. 

Start of a discussion on the Cochrane review technique applied to health system reforms and interventions

In the same mail, Bruno also shared his personal view (as a researcher) on the limits of systematic reviews. However, he acknowledged that the discussion will be beneficial to strengthening the impact evaluation program on PBF.

Still on the 10th of August, Sophie Witter (University of Aberdeen), first author of the Cochrane Review, answered Bruno.  “As a health systems person, I can fully understand your frustrations with the Cochrane process. It has its strengths and limitations, like all methods.”  In her mail, she reminded where systematic reviews fit in the body of knowledge. She acknowledged that “it may be a bit early for the evidence on PBF, but these reviews are updated periodically, so that should not be a problem in the longer term” and that “PBF is a particularly tricky topic for systematic reviews as it has been interpreted and implemented in very different ways and very different contexts. She added that “these are all issues we highlight in the review. We call for more research on systemic effects and the relationship to different contexts.” 

On August 11th, Por Ir (National Institute of Public Health, Cambodia ) – who just finalized a (non-systematic) review of evidence on RBF in maternal and neonatal health on behalf of the German Development Cooperation – shared his surprise “to see many reactions to this editorial, but not to the Cochrane review itself when it was published. Por did read the Cochrane review, and found it “well written with very transparent Cochrane recommended methods and well balanced findings and conclusions. So, if we want to challenge the findings, we better challenge the Cochrane review methods (mainly for health system research), but not the authors.”

A Cochrane review too keen on including enough studies?

The debate quieted for nearly a week, but was then re-launched by Joanne Harnmeijer (ETC Crystal, Netherlands) on August 17th. She came back on one of the points raised by Bruno, his assessment that the Cochrane review came too early. The discussion which followed provided the CoP members with some insight into the tensions internal to the systematic review. To Joanne, the review was unfair: the reviewers included data, reports or studies which did not necessarily pursue Cochrane standards and then in the next step of the assessment process, they correctly gave these studies a very low score of rigor.

On the same day, Sophie Witter reacted, providing more information on the approach taken and how it complied with Cochrane standards. Joanne responded still on the same day, reiterating her point. She emphasized Atle’s recommendation to have “a debate on how PBF-schemes might be evaluated – in a practical and feasible, but robust way” – is an important one.  Her own contributions fit in this agenda.

Joanne’s mail also triggered a response by Atle. He commented on the selection criteria. He reminded us of a few Cochrane rules that need to be respected if the review concerns a health systems intervention. He also wrote: “We do not criticize authors of the original studies. It is not our intention, anyway. On the contrary! They may have conducted the best study possible given the circumstances. I  would like to add that we are very thankful to many of the original authors. Several of them responded promptly to our e-mails and even sent us their full datasets. I think Cochrane reviewers in general struggle much more than we did when trying to engage study authors and getting access to unpublished data. So, we are very thankful, indeed!”

Still on the same day, Joanne answered Atle. The disagreement remains.

How to deal with the contextual factors in the assessment of PBF schemes? 

On August 21st, Eric Bigirimana (AEDES and BREGMANS Consulting, Cameroon)  came back on the importance of the context for the design and effectiveness of PBF schemes. He illustrated that with incisive observations made by participants of a study tour in three African Great Lakes countries. Eric is also a researcher. He reckoned an alternative research approach – the realist evaluation one – would take contextual factors better into account. In his long mail, he argued why.

Sophie, who is very familiar with the realist evaluation approach (she is currently the lead coordinator of a research project on selective free maternal health care which partly relies on this approach), responded:

“I think that the realist evaluation approach is very interesting and agree that it would be well applied to PBF. For the Cochrane methods, if there are enough robust studies, then you can look for context patterns. Unfortunately, if you only have a few (as was the case for the PBF review), then that is ruled out. But I would just note that the methodology in itself is not unable to take into account contextual differences.”

Contribution by Robert Soeters: a synthesis, some other criticisms and a way forward in terms of research

On August 22nd, Robert Soeters (SINA Health, Netherlands) sent his contribution. In his long mail, Robert provided some background on how he had been involved in the systematic review process. He felt that his commitment to transparency had not been rewarded by a similar commitment on the side of the reviewers. If field workers had been given the opportunity to give early feedback on the findings of the review, that would have been a valuable validation process for him. Robert also explained how current PBF knowledge has developed gradually over time through accumulation of experience and hinted that adoption of PBF best practices is another kind of validation process, one in terms of relevance. His assessment of the review was that “the result is a biased set of recommendations and some conclusions, which are communicated out of context.” He then developed this point by providing some more information on a few countries he has been working in over the past years. 

In the rest of his mail, Robert put forward a few ideas for a different approach on how to conduct PBF research. He raised different concerns, which matter for this future agenda. One is to find research strategies which do not harm the policy process, for instance randomization across districts (and not within) and gradual roll out of PBF “whereby at first a number of districts are included in the PBF intervention and  control districts that are not (yet) included in PBF.”

As for systematic reviews, he insisted on focusing on homogenous interventions. “ There is a quickly growing consensus in low- and mid –income countries on the definition of PBF and it should be avoided that studies are included of projects that do not qualify as PBF. It is unhelpful to study a project that is not PBF and then draw conclusions as if it is PBF. Cases in point were studies on Uganda (Palmer et al) and Zambia (Cochrane Review).” Similarly to others, he stressed the importance of context.

He concluded by endorsing the common objective of working on research methodologies. “We invite the academic world to constructively engage on the PBF health reforms and thereby to improve the research agenda.”

A rebuttal letter by Atle and a shared commitment to a strong research program on PBF

On the same day, Atle responded to Robert, point by point. However, for an unknown reason, the mail never reached the Google group. In his email, Atle reminded CoP members of some of the rules of a Cochrane review. He also provided guidance on how authors of primary studies could help systematic reviewers (e.g. provide more background information on context). In a private email, four days later, Robert thanked Atle and Sophie for their feedback in the CoP and concluded: "We are very happy that a serious discussion about how to accompany the promising PBF developments in many parts of the world with serious research is now well underway. Of course, you understand that I still beg to differ on a variety of points with you, but at least a dialogue has been opened which we really welcome. (...) We are looking forward to further debates with you and thank you once again for at least having recommended further research in this field. There, we fully agree!"

 
On August 25th, after the announcement by a member of the approbation by the Burundian parliament of an IDA grant of US$ 14,8 million for PBF, the Google group discussion veered off into another direction – on the sustainability of PBF, in French first. 

Obviously, though, the discussion on the best research strategy for PBF interventions will go on. Feel free to contribute via the comment section.


1 Commentaire
Huntingdonshire County Public link
5/21/2013 12:21:39 am

While We have comprehend the post, and also believe almost all of the writer’s propositions, the duplicate author offers offered me individually a lot to consider. We are rereading the content to be able to get more details.

Répondre



Laisser un réponse.


    Our websites

    Photo
    Photo
    Photo

    We like them...

    SINA-Health
    International Health Policies
    CGD

    Archives

    Septembre 2019
    Juin 2019
    Avril 2019
    Mars 2019
    Mai 2018
    Avril 2018
    Mars 2018
    Février 2018
    Janvier 2018
    Décembre 2017
    Octobre 2017
    Septembre 2017
    Août 2017
    Juillet 2017
    Juin 2017
    Mai 2017
    Avril 2017
    Mars 2017
    Février 2017
    Janvier 2017
    Décembre 2016
    Novembre 2016
    Octobre 2016
    Septembre 2016
    Août 2016
    Juillet 2016
    Avril 2016
    Mars 2016
    Février 2016
    Janvier 2016
    Décembre 2015
    Novembre 2015
    Octobre 2015
    Septembre 2015
    Août 2015
    Juillet 2015
    Juin 2015
    Mai 2015
    Avril 2015
    Mars 2015
    Février 2015
    Janvier 2015
    Décembre 2014
    Octobre 2014
    Septembre 2014
    Juillet 2014
    Juin 2014
    Mai 2014
    Avril 2014
    Mars 2014
    Février 2014
    Janvier 2014
    Décembre 2013
    Novembre 2013
    Octobre 2013
    Septembre 2013
    Août 2013
    Juillet 2013
    Juin 2013
    Mai 2013
    Avril 2013
    Mars 2013
    Février 2013
    Janvier 2013
    Décembre 2012
    Novembre 2012
    Octobre 2012
    Septembre 2012
    Août 2012
    Juillet 2012
    Juin 2012
    Mai 2012
    Avril 2012
    Mars 2012
    Février 2012
    Janvier 2012
    Décembre 2011
    Novembre 2011
    Octobre 2011

    Tags

    Tout
    2012
    Accountability
    Aid
    Alex Ergo
    Assurance Maladie
    Bad
    Bamako Initiative
    Bénin
    Bruno Meessen
    Burkina Faso
    Burundi
    Civil Society
    Communauteacute-de-pratique
    Communauté De Pratique
    Community Of Practice
    Community Participation
    Conference
    Cop
    Course
    Couverture Universelle
    CSU
    Déclaration De Harare
    Divine Ikenwilo
    Dr Congo
    économie Politique
    élections
    équité
    Equity
    Fbp
    Financement Basé Sur Les Résultats
    Financement Public
    Fragilité
    Fragility
    Free Health Care
    Global Fund
    Global Health Governance
    Gratuité
    Gratuité
    Health Equity Fund
    Health Insurance
    ICT
    Identification Des Pauvres
    Isidore Sieleunou
    Jb Falisse
    Jurrien Toonen
    Kenya
    Knowledge-management
    Kouamé
    Leadership
    Mali
    Management
    Maroc
    Maternal And Child Health
    Médicaments
    Mise En Oeuvre
    Mutuelle
    National Health Accounts
    Ngo
    Niger
    Omd
    OMS
    Parlement
    Participation Communautaire
    Pba
    Pbf
    Plaidoyer
    Policy Process
    Politique
    Politique De Gratuité
    Politique De Gratuité
    Post Conflit
    Post-conflit
    Private Sector
    Processus Politique
    Qualité Des Soins
    Qualité Des Soins
    Quality Of Care
    Recherche
    Redevabilité
    Reform
    Réforme
    Research
    Results Based Financing
    Rwanda
    Santé Maternelle
    Secteur Privé
    Sénégal
    Société Civile
    Uganda
    Universal Health Coverage
    User Fee Removal
    Voeux 2012
    Voucher
    WHO

Powered by Create your own unique website with customizable templates.