Scope for better assessment

Billions of dollars are spent every year on development – all too often with disappointing results. The question of how to allocate aid money most effectively is haunting donors and academics alike. While many continue to grapple with the theoretical challenges, the World Bank has adopted a clear and operational concept. Its “Performance-Based Allocation” system is based on the assumption that aid is most effective in an environment of good governance. Therefore, the bulk of concessional funds is disbursed to countries with sensible policies and well-performing institutions.

The World Bank’s instrument to rate governance is the Country Policy and Institutional Assessment (CPIA). Obviously, the question of what country gets how much aid is of a political nature. Accordingly, the CPIA is controversial too. After the assessment exercise began in the late 1970s, the World Bank has repeatedly revised the CPIA methods (see box) and also its allocation formula.

Nonetheless, criticism persists. Critics argue that the rating process is not robust, unreliable and inconsistent. Moreover, some claim that the CPIA does not measure the right factors, and thus does not capture all aspects relevant to development. The rating and allocation decisions are said to be unfair, ultimately penalising those countries that are in greatest need of help.

On behalf of Germany’s Federal Ministry for Economic Cooperation and Development (BMZ) and German Technical Cooperation (GTZ), the Global Public Policy Institute (GPPi), an independent think tank in Berlin, examined whether the CPIA in its current form makes sense. The GPPi study resulted in reform proposals (GPPi 2008). Implementation would render the allocation system more robust, more equitable and more conducive to aid effectiveness.

The CPIA is meant to identify the countries that are likely to use development aid most effectively. But does it really serve this purpose? Publicly available data show that the correlation between CPIA results and development outcomes is tenuous at best. Countries at the bottom of the list, for instance, are almost as likely to achieve extraordinary growth rates as are top performers. Therefore, the development-relevance of the CPIA seems questionable.

It is obvious that there is scope for improvement. That triggers another question: Can the current system be reformed, or is a radical overhaul necessary? The main alternative to assessing countries’ policies and institutions would be to rate countries according to their developmental progress. Such an outcome-based assessment system would reward any conceivable development strategy, as long as it was effective. In view of the disappointing results of structural-adjustment policies in the 1980s, it seems tempting to focus on results instead of policies and institutions.

Despite this reasoning, GPPi argues for a reform of the CPIA. There are several reasons:
- It makes sense for donors to not only consider whether development is taking place, but also how it is brought about.
- As reforms often only show results after a long time, an outcome-based system would reward or penalise governments for the action of their predecessors.
- It would be very difficult to take into account the effects of external influences on development.
- An outcome-based system would ultimately penalise countries that face particularly tough development challenges, whereas a system based on country policies and institutions can foster support for very poor countries with promising political and institutional developments.

As argued above, the CPIA approach is in need of reform. Therefore, GPPi proposes changes to make results more robust and reliable, increase the development relevance of the criteria and enhance the fairness of the rating and allocation processes.

More reliable results

It is true that CPIA ratings are based on a sophisticated assessment process, built on the World Bank’s vast expertise and subject to the professional judgment of World Bank staff through a strict review process. In spite of such efforts, the results are not satisfying. One reason is that the CPIA relies too heavily on subjective assessments. While it is, of course, notoriously difficult to assess transparency, accountability or corruption in objective terms it does not mean that one cannot measure these factors at all. Rather, the margin of error should be transparently acknowledged.

Credibility would also benefit from assessment teams using additional information and drawing on more diverse sources. In addition to the views of the host governments, those of other donor agencies, academia and civil society should be solicited.

Finally, the CPIA process should become more transparent. So far, the assessment is largely an internal World Bank affair. It is, however, not enough to only publish the CPIA results. The factors considered in making the assessment should not remain confidential. Obviously, external scrutiny could help discover mistakes and omissions, and would thus be crucial for results to become more reliable.

Boosting relevance

Currently, the CPIA is based on economic growth theories. These theories assume that, besides economic and structural policies, social, environmental and governance issues also play a role. However, after decades of research and debate, scholars still do not agree on what factors are essential, and even less on their relative importance. Therefore, it is inherently controversial to base a tool like the CPIA on these theories.

It would be better to focus on internationally agreed development goals, like the Millennium Development Goals and others. The assessment would then no longer try to determine merely whether a country is on course to achieve economic growth, but also whether it is promoting education, public health, gender equality, environmental sustainability, human rights and other matters relevant to alleviating poverty.

Reorienting the CPIA in this sense would have two distinct advantages:
- There is a broad international consensus on what are desirable development goals. Determining, for example, whether a country is implementing an adequate policy to promote literacy is much less controversial than establishing whether it has the right institutional conditions for economic growth.
- A focus on this broader set of goals would give countries more scope to draft development strategies that fit their special needs and circumstances.

Fairness matters

Aid allocation must not only maximise effectiveness, it must also be fair. Therefore, external factors that are not under a government’s control must be taken into account. So far, the CPIA primarily assesses a country’s policies and institutions. However, some criteria (financial sector, gender equality and property rights/rule-based governance) also concern outcomes. It should therefore be ensured that CPIA ratings are not influenced by external factors but only reflect the given country’s own policies and institutions.

Moreover, the CPIA methods should be sufficiently flexible to take into account country-specific circumstances. Fragile states, for instance, face particular conditions such as a high risk of violent conflict. The CPIA, so far, does not consider measures relevant to reduce the potential for strife. It would therefore make sense to add to the CPIA criteria issues such as respect for human rights, protection of minorities and empowerment of communities.