Rwanda’s latest survey suggests poverty reduced 6%. A more meaningful comparison using the same data suggests it’s actually increased 6%.

President Paul Kagame at the Transform Africa Summit 2013 in Kigali, Rwanda. Photograph by Rwanda Government.

Some time ago, I congratulated the Rwandan government on the progress shown in the latest Integrated Household Living Conditions Survey (EICV4), particularly in terms of the reduction of poverty and inequality. I’m afraid I now have to withdraw my kudos.

Both President Paul Kagame and Minister of Finance and Economic Planning Claver Gatete claimed a substantial progress in poverty reduction, from 44.9% in 2010/11 to 39.1% in 2013/14, a spectacular drop of 6 points. This is indeed what can be found in the survey.

However, a closer reading of EICV4 shows that the results simply cannot be compared to previous EICVs since changes were made in the methodology on rural and urban classifications, consumption definitions, the definition of the poverty line, and consumption thresholds. In other words, due to significant changes in the methodology, poverty headcounts in EICV4 are simply not comparable to the previous three surveys. And yet, both the government and the report itself (e.g. in the summary table on page vi) do compare the incomparable. They find progress where it cannot be found.

A more genuine comparison between EICV4 and previous EICVs can however be made, and it leads to completely different observations.

The change in methodology in the recent survey is discreetly mentioned on pages 1 and 17 of the report – “As for EICV4 carried out in 2013/14, it has been deemed necessary to update the poverty line” – without further explanation of how this was done and how it affects comparability. A crucial sentence can though be found on page 33, which reads: “If the objective is to obtain a minimal basket [of goods consumed by the poor], then replace rarely consumed or extremely costly goods with more common, less expensive items within the same category”.

What the EICV does, however, is much bigger than this. It also massively reduces the quantities (by 70% or more) of three traditional staples (sweet potato, Irish potato, banana) and increases other items such as fermented cassava, sorghum, and maize. There is no ground for this adaptation, which is then translated in “Adjusted scaled up” data (in terms of Kcal and Kg) on table B4 on page 37, but what it does is artificially reduce the value of the basket and thus decrease the poverty line by as much as 19 points in 2013/14 as compared to 2010/11.

Using different poverty lines from EICV3 to EICV4 means direct comparisons lack meaning, but the knowledge of how the poverty line was changed for 2013/4 allows us to work out a more meaningful comparison of poverty between the two periods.

We can do this by starting from the 2010/11 poverty figure of 45%, but then adapting the 2010/11 poverty line in the same proportion (-19 points) as the 2013/14 adjustment. Based on the EICV3 micro data and lowering the poverty line by 19%, the actual poverty figure is then 33% in 2010/11. There is nothing wrong in principle with a working out a new poverty line for EICV4 – as “many changes in the socio-economic structure of the country have taken place [since the first EICV in 2000/1]” – but even if the change is accepted, the 2013/14 report should measure change in poverty using a comparable poverty figure in 2010/11, namely 33% instead of 45%.

In other words, the new line needs to be applied retrospectively to past surveys. The genuine comparison from 2010/11 to 2013/14 therefore is not from 45% to 39%, but from 33% to 39%. What this suggests is that rather than a decrease of 6 points in poverty, what the EICV4 actually shows is an increase of 6 points. The above calculation is admittedly crude, and more sophisticated research would be needed to derive more precise figures to show exactly what went wrong. Ideally, one would recalculate the 2010/11 poverty level based on the micro data using the 2013/14 methodology. However, the overall picture is clear, and it shows an increase in poverty.

Omerta

The question then becomes whether the Rwandan government deliberately fools the world and its own population. There are strong indications this is indeed the case. While EICV3 acknowledged the support of the Oxford Policy Management (OPM) team and two of its researchers mentioned by name, EICV4 merely expresses gratitude to “the advisory team of national and international experts for their advice”, without further specification. This is understandable in light of the fact the OPM, in the words of one of its researchers, had a “difference of view” with the National Institute of Statistics of Rwanda (NISR) and that the whole OPM team withdrew from the poverty analysis and all other work with NISR. I don’t know what the “difference of view” was about, but one may assume it had to do with methodology and comparability of data across the successive EICVs.

This entire story raises a serious problem as Rwanda is keen on showing strong “development” measured, among other things, against reduction of poverty and inequality. Indeed, the international community accepts a trade-off between “development” and repression. But if “development” is not based on evidence, as appears to be the case now, what is left is just repression (for which the evidence is overwhelming).

When soliciting comments on my findings from involved actors, omerta is what I encountered. Two of the international partners that supported EICV4 – the UK’s Department for International Development (Kigali) and the World Bank (Rwanda Country Programme Coordinator) – declined to comment. Meanwhile, OPM stated that “our contract has a Confidentiality Clause that prevents us for disclosing any information about the work that we have done for Stats Rwanda”. Despite repeated requests NISR too failed to comment.

It is surprising that the international aid community (that surely must know what I know) does not seem to be bothered by major flaws in the evidence on Rwanda’s achievements in two major pet areas of donors: poverty and inequality. Yet the Rwandan government is known to have cheated on other issues. There is even a kinyarwanda word for it: tekiniki (from the French techniquer, which means “to doctor data”). So the aid industry appears to have decided to soldier on, even in the face of this blatant attempt by the Rwandan government to play foul. This makes clear again that donors and recipients need each other. Donors need “success stories”, recipients need money, and neither wants to rock the boat.

Maybe the aid community should heed the warning of the 2015 Economics Nobel Laureate Angus Deaton, who wrote: “By providing health care for Rwandan mothers and children, [President Kagame] has become one of the darlings of the industry and a favorite recipient of aid. Essentially, he is “˜farming’ Rwandan children, allowing more of them to live in exchange for support for his undemocratic and oppressive rule. Large aid flows to Africa sometimes help the intended beneficiaries, but they also help create dictators and provide them with the means to insulate themselves from the needs and wishes of their people”.

Filip Reyntjens is a Professor of Law and Politics, Institute of Development Policy and Management, University of Antwerp