Measuring donor performance

Development cooperation is a vast and highly complex field, in which the impacts of some actions are only felt years later. Despite daunting methodological challenges, it is essential to systematise donor performance measurement (DPM).

Significant progress has been made in the past 20 years in measuring the performance of individual projects. Monitoring and evaluation provide a clear picture of the results of specific projects and programmes. The methodology serves “bottom-up governance”, as information from specific projects has an impact on donor agency policy. Policymakers at the national, bilateral and multilateral levels can – and should – use such knowledge.

However the sum of successful projects and programmes does not determine the effectiveness, efficiency or sustainability of any single donor’s activities in any particular field. Efforts to measure the true impact of development projects on a particular sector, region or country have usually failed due to the “attribution gap” – the difference between the specific results of a project (a new road, for instance) and higher development goals (such as more business activity). Currently, most attempts to narrow the attribution gap focus on identifying plausible links between specific project results and general development indicators.

To an increasing extent, development policy is being implemented in broader programmes that address interrelated issues. This approach makes it easier to tackle the attribution gap – the goal, after all, is no longer the construction of a road, but rather the promotion of business activity. Meaningful indicators of business activity could be data on average incomes, jobs, industrial output et cetera.

Unless targets are specific and measurable, however, it remains impossible to hold anyone accountable for not achieving goals. The reverse is also true: no agency can truly take credit for improvements in the well-being of a target population. In other words: we do not know whether donor agencies are doing a good job or not.

Donor agencies – whether bi- or multilateral – depend on money from the national budgets of rich countries. Tax payers would certainly like to know whether “their money” is well spent. Without proof of results, it will become hard to justify development expenditure in the long run. DPM would, of course, also help decision makers in poor countries to pick appropriate partners. Therefore, systematic performance assessments would not make aid more donor-driven, but rather serve aid effectiveness.

It all boils down to transparency in reporting. While most donors and their agencies do not like to consider themselves “normal economic actors”, they certainly want to have a lasting economic impact in poor countries. Moreover, they are often among the strongest players in specific sectors of those countries. Accordingly, they should accept norms that apply to private-sector companies.

Quantified goals

Companies listed on stock exchanges face detailed reporting and accounting requirements. On this basis, it is possible to assess their shareholder value. In the case of donor agencies, it would make sense to assess the “taxpayer value” instead. For that purpose, donor agencies should be required to quantify their developmental goals and their intermediate targets. These goals should be expressed as specific, measurable improvements in the real situation of the population in the target countries. Relevant data, for instance, might concern
– household incomes,
– literacy,
– employment,
– health,
– hardship experienced in business, as well as the quality of air or water and many other issues.

No doubt, some appropriate indicators can be found for each and every development project or programme. Such data, in turn, could be used to assess donor agencies’ performance in terms of
– quality (comparing quality of outputs with technical standards),
– efficiency (comparing outputs with costs),
– productivity (comparing outputs with physical inputs),
– effectiveness (comparing actual results with planned results),
– economy (comparing physical inputs with costs),
– relevance (relating programme-level objectives to broader country or agency goals),
– sustainability (comparing results during and immediately after implementation with long-term results) and
– cost-effectiveness (comparing outcomes/impacts and their costs).

Skeptics will argue that it is difficult to quantify goals when dealing with important developmental issues such as governance or institution building. There is a way to rise to this daunting challenge, however. What we need is indicators for the performance of the institutions relevant to governance. Moreover, finding such indicators should be easier when assessing sector-wide approaches and multi-dimensional programmes than when dealing with small-scale projects, because there is less of an attribution gap.

An important part of DPM will be obtaining clear information on the work of other donors in any particular sector or region. This is currently done on a pro forma basis, but it does not yet result in systematic coordination. It would obviously make sense, however, to directly compare the performance of various donor agencies.

Systematic DPM would lead to more transparency. One result would be fair competition among agencies. That, in itself, should facilitate better decision-making. Benefits of DPM would include
– the setting of more realistic and achievable goals for donor interventions,
– the design of projects and programmes that contribute directly to achieving goals (instead of doing so indirectly and partially, as often at present),
– positive publicity and taxpayer trust, and
– better resource allocation guiding funds and talents to the most successful donor institutions.

The idea to assess donor performance in general is not new, and many donor agencies have established some sort of internal performance assessment. These reports, however, are impossible to compare because they are based on a wide variety of different methods. Moreover, they tend to suffer from poor credibility, as they are drafted on commission of the very agencies the performance of which they are supposed to assess.

For the purpose of systematic DPM, it will therefore be necessary to determine who orders the assessments, who receives the results and who makes what decisions based on them. Stakeholders include governments, parliaments and taxpayers. The political implications will be complex and most likely transcend national borders.

Improving governance

Twenty years ago, when project evaluations were first introduced, there was much resistance from the “development community”. Today, project monitoring and evaluation is accepted as being essential to quality control.

DPM would significantly contribute to improving donor-agency accountability, coherence in their programmes and cooperation at all levels. Moreover, DPM would certainly improve the governance of donor agencies, which, given agencies’ aspiration of improving governance in developing countries, is only logical and fair.