Lies, damned lies and statistics: Poverty reduction Rwandan-style and how the aid community loves it
Rwanda’s latest survey suggests poverty reduced 6%. A more meaningful comparison using the same data suggests it’s actually increased 6%.
President Paul Kagame at the Transform Africa Summit 2013 in Kigali, Rwanda. Photograph by Rwanda Government.
Some time ago, I congratulated the Rwandan government on the progress shown in the latest Integrated Household Living Conditions Survey (EICV4), particularly in terms of the reduction of poverty and inequality. I’m afraid I now have to withdraw my kudos.
Both President Paul Kagame and Minister of Finance and Economic Planning Claver Gatete claimed a substantial progress in poverty reduction, from 44.9% in 2010/11 to 39.1% in 2013/14, a spectacular drop of 6 points. This is indeed what can be found in the survey.
However, a closer reading of EICV4 shows that the results simply cannot be compared to previous EICVs since changes were made in the methodology on rural and urban classifications, consumption definitions, the definition of the poverty line, and consumption thresholds. In other words, due to significant changes in the methodology, poverty headcounts in EICV4 are simply not comparable to the previous three surveys. And yet, both the government and the report itself (e.g. in the summary table on page vi) do compare the incomparable. They find progress where it cannot be found.
A more genuine comparison between EICV4 and previous EICVs can however be made, and it leads to completely different observations.
The change in methodology in the recent survey is discreetly mentioned on pages 1 and 17 of the report – “As for EICV4 carried out in 2013/14, it has been deemed necessary to update the poverty line” – without further explanation of how this was done and how it affects comparability. A crucial sentence can though be found on page 33, which reads: “If the objective is to obtain a minimal basket [of goods consumed by the poor], then replace rarely consumed or extremely costly goods with more common, less expensive items within the same category”.
What the EICV does, however, is much bigger than this. It also massively reduces the quantities (by 70% or more) of three traditional staples (sweet potato, Irish potato, banana) and increases other items such as fermented cassava, sorghum, and maize. There is no ground for this adaptation, which is then translated in “Adjusted scaled up” data (in terms of Kcal and Kg) on table B4 on page 37, but what it does is artificially reduce the value of the basket and thus decrease the poverty line by as much as 19 points in 2013/14 as compared to 2010/11.
Using different poverty lines from EICV3 to EICV4 means direct comparisons lack meaning, but the knowledge of how the poverty line was changed for 2013/4 allows us to work out a more meaningful comparison of poverty between the two periods.
We can do this by starting from the 2010/11 poverty figure of 45%, but then adapting the 2010/11 poverty line in the same proportion (-19 points) as the 2013/14 adjustment. Based on the EICV3 micro data and lowering the poverty line by 19%, the actual poverty figure is then 33% in 2010/11. There is nothing wrong in principle with a working out a new poverty line for EICV4 – as “many changes in the socio-economic structure of the country have taken place [since the first EICV in 2000/1]” – but even if the change is accepted, the 2013/14 report should measure change in poverty using a comparable poverty figure in 2010/11, namely 33% instead of 45%.
In other words, the new line needs to be applied retrospectively to past surveys. The genuine comparison from 2010/11 to 2013/14 therefore is not from 45% to 39%, but from 33% to 39%. What this suggests is that rather than a decrease of 6 points in poverty, what the EICV4 actually shows is an increase of 6 points. The above calculation is admittedly crude, and more sophisticated research would be needed to derive more precise figures to show exactly what went wrong. Ideally, one would recalculate the 2010/11 poverty level based on the micro data using the 2013/14 methodology. However, the overall picture is clear, and it shows an increase in poverty.
Omerta
The question then becomes whether the Rwandan government deliberately fools the world and its own population. There are strong indications this is indeed the case. While EICV3 acknowledged the support of the Oxford Policy Management (OPM) team and two of its researchers mentioned by name, EICV4 merely expresses gratitude to “the advisory team of national and international experts for their advice”, without further specification. This is understandable in light of the fact the OPM, in the words of one of its researchers, had a “difference of view” with the National Institute of Statistics of Rwanda (NISR) and that the whole OPM team withdrew from the poverty analysis and all other work with NISR. I don’t know what the “difference of view” was about, but one may assume it had to do with methodology and comparability of data across the successive EICVs.
This entire story raises a serious problem as Rwanda is keen on showing strong “development” measured, among other things, against reduction of poverty and inequality. Indeed, the international community accepts a trade-off between “development” and repression. But if “development” is not based on evidence, as appears to be the case now, what is left is just repression (for which the evidence is overwhelming).
When soliciting comments on my findings from involved actors, omerta is what I encountered. Two of the international partners that supported EICV4 – the UK’s Department for International Development (Kigali) and the World Bank (Rwanda Country Programme Coordinator) – declined to comment. Meanwhile, OPM stated that “our contract has a Confidentiality Clause that prevents us for disclosing any information about the work that we have done for Stats Rwanda”. Despite repeated requests NISR too failed to comment.
It is surprising that the international aid community (that surely must know what I know) does not seem to be bothered by major flaws in the evidence on Rwanda’s achievements in two major pet areas of donors: poverty and inequality. Yet the Rwandan government is known to have cheated on other issues. There is even a kinyarwanda word for it: tekiniki (from the French techniquer, which means “to doctor data”). So the aid industry appears to have decided to soldier on, even in the face of this blatant attempt by the Rwandan government to play foul. This makes clear again that donors and recipients need each other. Donors need “success stories”, recipients need money, and neither wants to rock the boat.
Maybe the aid community should heed the warning of the 2015 Economics Nobel Laureate Angus Deaton, who wrote: “By providing health care for Rwandan mothers and children, [President Kagame] has become one of the darlings of the industry and a favorite recipient of aid. Essentially, he is “˜farming’ Rwandan children, allowing more of them to live in exchange for support for his undemocratic and oppressive rule. Large aid flows to Africa sometimes help the intended beneficiaries, but they also help create dictators and provide them with the means to insulate themselves from the needs and wishes of their people”.
Filip Reyntjens is a Professor of Law and Politics, Institute of Development Policy and Management, University of Antwerp
Statistics when used by individuals or associations with descriptive purpose can be contrived to provide a response particular in favour to the originating people and or associations. Statistics can be skewed as the numeracy is fungable.
When using statistics, it is usually advisable to provide a textual perspective so as to enhance the ontologic integrity of the statistics being advanced in projecting ideas or programs or values.
Paul Kagame seems to be a most benevolent fellow compared to so many others on that lovely dark continent.
Better perhaps to hold them to the same standard as that of the lying Obama administration eh? With that comparison he looks a saint!!!!!!!
Look at the Obama administration’s unemployment stats….for example. They pretend that employment has actually improved under his horrid mismanagement and abuse of the American people. In fact 95 MILLION WORKING AGE Americans are OUT OF THE WORK FORCE…a stunning record in modern times…Statistics lie, but statisticians lie far more!!! and Democrats are consummate liars!!!!!!!!
All that, all I feel sure of is that the Oxford consultants group disagreed with the government over some part of their method. Assuming for the sake of argument that the manipulation is correctly described and was done on purpose, I just don’t see it as that unusual. Many countries do it, and corporates routinely change reporting procedures to make a minus look like a plus. Which doesn’t make it alright but it’s just not sinister or food for polemic unless people want to make it look that way.
Reaction to NISR’s rebuttal of challenges to its EICV4 poverty data
http://statistics.gov.rw/about-us/news/nisr-refutes-wrong-allegations-its-latest-poverty-report-france-24
1. I have not argued that NISR has changed the calorie threshold from 2500Kcal/d. In fact, I believe that it would be absolutely acceptable for NISR to lower the calorie threshold, if it deemed that appropriate. My claim is that if NISR does that, or makes any other similar fundamental change to its methodology, it would need to apply that change retrospectively to previous surveys in order to enable meaningful comparison between surveys. NISR has not responded to this fundamental and uncontroversial methodological point regarding the comparability of 2011 and 2014 figures.
2. NISR’s claim that the change in the consumption basket was motivated by a desire to ensure that it reflects current consumption patterns of Rwandan households is contradicted by the fact that it arbitrarily removed three main staples (bananas, potatoes, and sweet potatoes) from the final basket, after updating it for current consumption patterns. Unless NISR can show that Rwandans have stopped consuming these three staples, its justification for the new poverty line does not hold. Actually, the unadjusted data in NISR’s poverty line spreadsheet is the updated consumption basket of the bottom 40%. NISR gives the change in the consumption basket as the justification for updating the poverty line, therefore the logical thing would have been to stick with this basket. To do otherwise suggests questioning the data.
3. If NISR were interested in ensuring that the basket reflects current consumption patterns, it should have used the quantities listed in the column titled “Scaled up quantity” (3rd column from the right) in table B4 (page 37 in EICV4 poverty profile), which are updated for changes in consumption patterns. Had NISR done so, the poverty line would have been Rwf 197,000 (this can be computed from table B4), and the poverty rate would have been at least 12 percentage points higher than the one claimed in EICV4 (i.e. 6% higher than in 2011).
4. I must therefore conclude that NISR’s motivation for changing the poverty line was not only to update the consumption basket based on changes in consumption patterns, as it claims, but also, and more importantly, to change the type of poverty line used to a (lower) minimum poverty line, as clearly stated on page 33 of the report. While it is NISR’s right to use whatever type of poverty line it deems fit, it needs to make sure that comparisons between surveys are made using comparable poverty lines, as stated under point 1 above.
5. Since NISR has invited scrutiny of its findings, I would suggest that it publishes the datasets and associated materials used to compute the poverty rates, so that these can be objectively verified by independent third parties. Failing that, I invite interested experts to check the facts for themselves. The comparable poverty rate can be estimated from the following information which is currently publicly available: (a) the 2011 datasets are available on the NISR website (micro-data) and (b) the 2014 poverty lines (adjusted and actual) are detailed in table B4 on page 37 of the EICV4 report. It is therefore possible to check what the comparable poverty rate would have been in 2011 if NISR had consistently applied its own 2014 methodology retrospectively to the 2011 data.
Filip Reyntjens
Institute of Development Policy and Management
University of Antwerp
Hahahaaa haters haters haters.i am rwandan and i can assure you that the statistics the Rwandan government released are true.say all the craps you have in your empty head but know that:”dogs bark but the caravan goes by”.