• Print Edition
• Archive
  • Editorial
  • Monitor
  • Focus
  • Media
  • Tribune
  • InWEnt Forum
  • Debate
• About us
• Index
• Links
• Subscription

 

• RSS
• Twitter

 

[ Impact research ]

From snowball to avalanche

Monitoring and evaluation matter in national politics, but play an even greater role in bilateral and international cooperation. It is therefore astounding how little attention is paid to choosing reliable indicators for impact measurement in the planning stages of interventions. A capacity-building project run by InWEnt in the field of information technology (IT) illustrates that it makes sense to examine what works – and how as many people as possible can be reached.


[ By Balthas Seibold, Winfried Kalhöfer and Markus Wauschkuhn ]

The most prominent example of a problematic selection of indicators is surely the Millennium Development Goals (MDGs). Very few countries have reliable data for the base year 1990, against which the desired effects are supposed to be measured. Monitoring data tend to be insufficient as well. Unfortunately, it is a similar story with many national Poverty Reduction Strategy Papers (PRSPs), on the base of which donors make their funding decisions.

The selection of meaningful indicators is crucial for any M&E system. It is very difficult to correct an inappropriate selection once a programme is underway. InWEnt promotes data collection in accordance with – and on the basis of – existing statistical systems and not to create additional, often donor-driven parallel systems.

It makes sense to assess various details in M&E. How does a single IT trainer succeed in teaching 200 members of a cooperative to handle a computer? The key question in every capacity-building project with a view to dissemination and “training the trainer” is: How can training materials be distributed widely and sustainably? How does a snowball become an avalanche?


“it@coops”

The InWEnt programme “Information Technology for Asian Cooperatives” – it@coops for short – is an example of how this can be done. The programme supported small Southeast Asian cooperatives in training and imparting IT know-how, improving local IT centres and forming national and regional networks. InWEnt and the Asian Women in Co-operative Development Forum (AWCF) conducted an analysis of the programme in retrospect, and discovered snowballing effects:
– The newly created pool of 70 IT instructors trained more than 5,000 members of cooperatives within a single year. Additional qualified instructors were appointed.
– Local IT business development centres in Indonesia, the Philippines and Thailand reached more than 21,000 members of the target group.
– Small and micro businesses belonging to cooperatives in rural areas gained access to the Internet and were connected to important IT services.
– Almost two thirds of the new users of information technologies were women. The newly acquired skills improved their prospects, boosted their self-esteem as well as their productivity, and gave them access to sources of knowledge.

In short, the project fulfilled the brief given by the German Development Ministry (BMZ), which had commissioned it@coops. To what extent has impact monitoring, which was conducted throughout the programme, contributed to success? In physics, the rule applies that you cannot measure the position of a particle without influencing it. In the natural sciences, this effect is often considered to be troublesome, but to programmes such as it@coops, it is most useful. In this case, local partners identified with the objectives because they were involved from the outset in working out and developing quantitative and qualitative indicators.

This approach motivated the people involved and raised their awareness. They carried out their own survey to find out how many women benefited from the computer training course run by newly qualified trainers. This kind of dynamism, of course, is only possible if indicators and benchmarks are worked out participatively and can be adjusted as the programme progresses.


Innovation thanks to IT

Information technologies open up new possibilities for impact measurement. Training material can be placed under an “infectious” open source licence, for example, which is what was done in this case. This means that the material, including any improvements, remains publicly accessible. In principle, it can also be used by any number of people. In the long run, the actual coverage is kept track of by means of an identity number – similar to the way “digital watermarking” works with music and videos.

it@coops was set up as a brand, proving that the approach works. National congresses have already been held under this name, and a Google search using this term brings up over 3000 websites. The snowball has launched an avalanche.

Of course, it is important to constantly check what works and what doesn’t. There are several obvious factors which contributed to the success of it@coops:
– It is important that the participants are selected according to criteria relevant to development.
– The people trained to train others should be members of the respective organisations.
– Emotional ties through national and regional networking are useful.
– The involvement of partners in the field is fundamental – and financial contributions are welcome.
– The training courses have to be tailored precisely to the needs, which requires local expertise.
– The focus of the cooperatives – especially of the AWCF – on the interests of poor women was helpful.

So far, however, the contextual conditions for success are not very well understood. Therefore, the entire programme will be examined by external consultants too. It is already clear, however, that any similar measure in future should pay even more attention to the conditions under which participants work day-to-day and what role their superiors play.

The insight that a successful programme cannot simply be repeated is commonplace. Experience shows, however, that it is worthwhile to set clear objectives and to observe closely how an initiative progresses.

Successful transfer at the workplace

The International Leadership Training (ILT), which is organised by InWEnt in Germany, annually prepares 400 people from developing and emerging-market nations for leadership positions back home. The programme focusses on the needs of the participants’ employers. Last summer, an evaluation proved the programme’s worth. In future, evaluations will take place on a regular basis.

During their year in Germany, the participants (ILT) expand their theoretical and practical competencies. The core topics
are management skills and the implementation of sustainable change. The programme includes an internship of several months at a German company or government body.

The main purpose of ILT is to strengthen the foreign institutions that assign participants to the courses. InWEnt acts in accordance with the goals and sector-specific strategies of the partner countries.

Of course, the skills an individual person acquires are only effective when actually put to use. ILT participants should therefore apply their newly learned knowledge skills at work. To InWEnt, a successful transfer of the competence means that participants
– act as multipliers, passing on their expertise to colleagues in presentations, internal essays, seminars or by other means,
– give advice on administrating of their sending institution or other parties, and/or
– change work-flows by either improving existing methods or introducing new ones.

In order to successfully use their newly-acquired knowledge when returning home, all ILT participants prepare “transfer-pro­jects”. These projects are implemented after returning home from Germany.

After two and half years of experience with ILT, InWEnt gave Bonn University’s Centre for Evaluation and Methods the assignment to carry out a tracer study. In order to evaluate the success of the ILT and its participants, the following factors were taken into account
– the relevance of the training for further
careers,
– changes at the personal, institutional
and systemic levels and
– personal networks.

Roughly half of the 900 ILT-participants from 67 countries were contacted via e-mail. 55 % filled in the questionnaire. Moreover, 177 employers and sending-organisations were contacted, of whom 43 % responded.
The data shows that nearly all participants (93 %) were able to make use of programme content at their workplace. That view was shared by the employers. Moreover, 88 % of participants indicated that colleagues had indirectly profited from their training, and 39 % were able to fully implement their transfer-project, while 40 % did so in part.

In regard to an improvement of working and decision-making procedures in their institution and in regard to the cooperation with international partners, nearly two-thirds of the participants found the programme useful. Positive effects at the institutional level were also noted by the employers.

Of particular interest is the result of ambitious transfer-projects. A highly relevant example is an energy-efficiency concept that is now being used as a publicity tool by a national environment programme. In another case, health-insurance companies are now assessing quality controls at health-care facilities. More than half of the respondents stated that other institutions had benefited from ideas they had brought home. The inmpact beyond the sending institution is an aspect that should be examined more in depth in future evaluations.

Many participants rated their networking experiences as positive. Three-quarters of the respondents used their contact to other ILT-participants on a professional basis, and 43 % kept in touch with staff members at the training facilities in Germany. Nearly the same number stayed in contact with the institutions in which they had done their internship.

From InWEnt’s point of view, independent evaluations do not only provide for accountability, but are also important for effectiveness and sustainability. The results help InWEnt’s management steer capacity-building programmes. Positive achievements as well as criticisms and suggestions for improvement stimulate further strategic planning.

The retrospective tracer study was a pilot project. In future this instrument will be made a standard device, so former participants will be questioned regularly in order to obtain an even clearer picture of the ILT impact on institutions and careers.

Claudia Kornahrens

D+C, 2009/01, InWEnt Forum, Page 37-39