Jeffrey Sachs and Angelina Jolie: advocates for the Millenium Villages project.

Michael Clemens likes evaluating impact. In fact, it might be more accurate to say that he lives impact evaluation; such is the intensity of his knowledge – theoretical and specific – on this particular avenue of development economics. Clemens’ recent and probably most high-profile work has focused on how impact assessment has been carried out on the Millenium Villages project. He addressed a joint RAS/ODI audience in London on Wednesday 3^rd July to talk about the Millenium Villages project and impact assessments

The Millenium Villages project is a joint venture of The Earth Institute at Colombia University (home of Jeffrey Sachs) and the United Nations Development Programme (UNDP). The project is most closely associated with Sachs, author of such rock star development books as The End of Poverty (not uncontroversial in itself – see William Easterly’s review) and on occasion to be spotted consorting with actual rock stars such as Bono and actress Angelina Jolie.

So, projects conceived under Sachs’ brand of Big Plan solutions can be an easy target for development naysayers. However, Clemens isn’t really one of these, he professes that his critical assessment of aspects of the Millenium Villages Project reveals “an incredibly sad story” where “no one wins” – the implication being that Clemens isn’t making criticisms of the project because he wants to, but rather because, as ODI Director Alison Evans pointed out, “this matters – this is a lot of money.”

What is Clemens saying?

The Millenium Villages (MV) projects constitute 14 different locations in rural Africa where poverty is being fought at village level towards achievement of the Millenium Development Goals (MDGs). The project’s own literature states that it fights “˜the root causes of extreme poverty, taking a holistic, community-led approach to sustainable development.’ Clemens contention is that impact assessment has been carried out with such a lack of rigour or basic independence of those conducting assessments, that any claims for the success of the programme are at best dubious, and at worst false.

Impact assessment in the development “˜industry’ is nothing new. According to Clemens “˜randomisation’ is all the rage right now, but randomisation – essentially comparing a group of people who did receive the benefits of a particular development programme with a group who didn’t – does not mean rigour. In the case of the MV programme, no “˜control’ villages that were totally without the benefits of such developmental inputs were studied. There seems to be some contention over whether “˜control’ villages would be possible, or ethical, with the MV project originally asserting that they were not.

In addition, a major contention of Clemens is that a comparison with the base-line of development in the Millenium Villages ie before the projects began, fails to take in to account that many African countries have experienced high levels of economic growth in the last 2 decades, which has been the major cause of improvement in development indicators, notably child mortality. Despite some claims, later withdrawn by the Millenium Villages project, it was found that child mortality in the Villages was declining at about the same rate as it was in the broader districts/regions and countries.

The riposte of Sachs and fellow Millenium Villages advocate Prabhjot Singh stated that “with communities, there are no true controls – life changes everywhere, in the MVs and outside them.” Which when it was published in October 2011 was described by David Mackenzie – Lead Economist, Development Research Group, World Bank – as “˜baffling.’ For a more detailed piece on the academic back-and-forth as it took place in October last year, there’s a good piece on the development blog View from the Cave.

So, why is this seemingly technical discussion between development geeks interesting/relevant to people with a wider interest in contemporary Africa? And by extension, why did the RAS and ODI invite Michael Clemens to talk about his views on MV and impact evaluation? As mentioned before, for projects that cost vast amounts of money, and in a climate where there isn’t a never ending pot of aid money to spend, it matters how we use it. Therefore it matters that we assess the effectiveness of projects properly. Would aid donors be better off spending their money and political capital on something that is proven to work eg bed nets to stop transmission of malaria by mosquito?

Impact assessment should also force us to assess what impact development assistance is having in relation to the broader economic changes that are at present constantly championed by optimistic commentators.

Clemens’ criticisms of impact evaluation and the MV projects have seemingly created some tension and bad-feeling between a number of economists with a stake in either camp. As was evident at the RAS/ODI meeting, much of this comes down to whose figures you find most credible, which for the non-specialist, can become somewhat bewildering.

To read Michael Clemens’ presentation in full click here

To watch a video recording of the event click here

Magnus Taylor is Managing Editor, African Arguments Online