S/he asks a simple question: why are NGOs so slow to use monitoring data that could improve the effectiveness of their work?
The answers were spot-on and made me wonder whether – and to what extent – paying for results (often referred to as “results-based financing“) helps address some (if any) of these challenges.
I’ll just go down the list:
Reason 1: Most of the data that NGOs collect is useless, usually geared towards measuring a small/narrow number of quantitiative indicators that have little to do with the actual program.
As someone who believes in and promotes RBF, I tend to be a big fan of outcome metrics, indicators and measurement. Having said that, I completely agree that the types of metrics and indicators used by most NGOs have little to do with their actual programs and theories of change. But I think it’s important to understand the underlying reason for that: most NGOs’ M&E frameworks (and therefore the data they collect) is driven by donors agendas. Indeed, rare is the donor who disburses “unrestricted” funds; instead, most donors fund a specific project that advances their narrow strategic priorities and impose their own methods and indicators for measuring success. So, dozens of donors –> dozens of M&E frameworks –> useless data for the NGO –> few opportunities (or incentives) for actual learning.
In an RBF contract, the donor is not paying for projects per se. Instead, s/he is purchasing an outcome (i.e. a 10% reduction in the incidence of malaria in a given community) regardless of which programs – or combination of programs – it took to get them there. This should, in theory, promote the use of metrics and indicators more closely aligned with NGOs’ theories of change. A major assumption, however, is that donors are both willing and able to wash their hands clean of any programatic decision-making once an outcome has been agreed…this is where third-party intermediary organizations can add value in terms of negotiating contracts and setting the record straight.
Reason 2: Pressure to spend quickly means there is little incentive (or need) for learning.
The pervasive disconnect between funding disbursements and programmatic realities on the ground is a huge problem in development…and usually comes from the top (i.e. the donors). Again, there’s a reason for that: if U.S. government agencies don’t spend all the money they’re allocated in a given year, their agencies will get less money the following year. For foundations, there are serious tax/legal ramifications for not disbursing a certain percentage of their endowment each year. Yes, the system’s messed up. Unfortunately, RBF alone won’t provide a solution. While it’s true that paying for results in one lump sum at the end should in theory remove the perverse incentive to spend, spend, spend, if donors are unable (for legal reasons or otherwise) to set aside a pool of money that just sits there for a couple of years and does nothing, RBF won’t stand a chance. Donors need to figure out a way around this ASAP.
Reason 3: Short-term projects often leave little time for staff to really learn from M&E.
It’s true that most donors fund projects on a short-term basis…too short to enable any real focus on outcomes or learning. In an RBF contract, because outcomes are precisely what donors are paying for, they should be committing funds over longer time horizons. However, this is assuming that the donor has the legal authority/political will to actually commit – and hold on to – funding over multiple years in the first place (see previous grumblings).
Reason 4: Learning from monitoring data requires some kind of process to allow organizations to feed this learning back into performance. Donors don’t typically make this easy (em, DFID) by imposing rigid logframes that are impossible and/or take forever to amend.
Don’t get me started on logframes. Suffice it to say that in an RBF contract, the idea is that once an outcome has been agreed and the donor has committed to paying a specified amount for that outcome, donors wash themselves completely of any programmatic decision-making. That includes changes to our beloved logframes.
Reason 5: NGOs genuinely don’t have a clue about how to actually improve programs. That’s because development is complex and programs typically aim to do ridiculously ambitious things.
I agree that development programs are complex, which is why a hands-off, more flexible approach like RBF might be preferable to a rigid, linear funding model. I also agree that many NGO programs aim to do ridiculously ambitious things. In some ways, because RBF requires both donors and NGOs to focus on things they can actually measure (ironically, this is a common criticism of RBF), perhaps RBF can help bring things down a notch…and in a good way. Start small, think big.