Poor Numbers: The Politics of Improving GDP Statistics in Africa – By Morten Jerven
Last week African Arguments published a story that Prof Morten Jerven, author of “˜Poor Numbers: How We Are Misled by African Development Statistics and What to Do about It‘, had been blocked from presenting his research on African statistical capacity at the UNECA. This was due to opposition to his ideas from, notably, the South African Statistician General, Pali Lehohla. The speech Jerven intended to give to the UNECA can be read here. Morten Jerven responds below.
Discussing economic statistics and GDP estimates of African economies is clearly important, but it’s also sensitive. Pali Lehohla and his self-proclaimed union of “˜African Statisticians’ are allied in a self-defeating campaign.
My book Poor Numbers has created an unprecedented argument for investing in the statistical capacity of African countries. Why would Lehohla and his silent supporters go against this? The answer is simple. Pali Lehohla and his counterparts are doing well in the current system. Any change to the status quo in the political economy of statistics in Africa is considered a threat.
The allegations that I am a “˜hired gun’ or “˜that I have not done my research’ are of course ridiculous and entirely false. With Lehohla putting his emphasis on “stopping Jerven in his tracks“ before he “hijacks the African statistical agenda” the immediate danger is that good initiatives will be suspended and cancelled. In the long term, statistical offices in the region may struggle for survival.
Any observer of this debate is likely to draw two conclusions: The quality of the statistics are probably not very good and some of the leaders seem to react very aggressively rather than doing something about it.
When Bill Gates read Poor Numbers he concluded that it was time to invest in better GDP statistics. If he or other stakeholders read this particular debate, I don’t think their next logical step is going to be – “˜And I am going to put my money on the angry guy in who is trying to censor public debate’. I fear that the public statements made by Lehola and his supporters seem to observers that statistical offices are part of the problem and not the solution.
I still think any agenda that attempts to provide the public with better statistics must engage with statistical offices and their interests. As I describe below, these are not the only conditions that make it hard to have an objective and open debate on African statistics.
***
When I first set out to investigate African GDP estimates in 2007, I didn’t think the work would get much of a reception. As I document in Poor Numbers, actual practitioners of national accounts were surprised that a researcher would take interest in their work. Actual data users in the countries concerned, such as Central Banks, were happy that such concerns were being raised, and the consultants were relieved to talk about some of their long-standing worries and the problems they experience.
Data disseminators, on the other hand, were usually quite reluctant to share information, and did not (sometimes understandably) always want to provide information on the methods used to produce the numbers. This was generally justified on the basis of “˜ownership’ of the data and a belief that they shouldn’t share the processes surrounding the production of the final data because of “˜confidentiality’ (which can be frustrating, or as one reviewer put it “[Jerven] relates chilling tales of how his attempts to access raw data behind international institutions’ statistics met with evasion.”
The initial response from many economists working on Africa varied between, “˜so what?…we already know this’, “˜we don’t trust or use official statistics on Africa anyhow’ and “˜I know but what is the alternative?’ Many more scholars in African studies and development studies, who were generally concerned with the long-standing use of numbers on Africa as “˜facts’, were relieved that there was finally someone who sought, not only to unveil the real state of affairs, but genuinely wanted to answer some of the problems that users face when trying to use the data to test their scholarly questions.
My original intent was never to write a book about this. Initially I was genuinely puzzled by some of the discrepancies in GDP data on Africa and, after some deliberation, decided that I should try to systematically collect information on how some African economies are being measured. I conducted the research between 2007 and 2010, and wrote up the manuscript in 2011.
As is documented in the book, my research methodology has been to combine archival work and secondary literature with broad surveys and in-depth interviews. As I document in the book, there is simply no way you can just walk in the door and demand: “˜How poor are your numbers?’ Moreover, one of the key problems is a complete lack of documentation of methods and sources – theoretical manuals do not always match up with the practical process at the office. It has however been possible to collect the information needed by using a combination of methods – evaluating documents like an historian, reading numbers like an economist and collecting metadata through personal observations, triangulation and conversations (like a good journalist would.) In the introduction of Poor Numbers I describe these methods as an attempt to do a “˜political ethnography’ of African economic statistics.
I had an invitation and introduction to all the offices I visited. Date and place of interviews at statistical offices, central banks and donor missions are documented. Between 2007 and 2011 I wrote letters, emails and phoned all statistical offices in Sub-Saharan Africa repeatedly in order to verify information, request access and set up interviews. As anyone who has tried something similar can attest, the response rate is extremely low. For all the places I did go to I had a response, a contact and an invitation.[i]
Upon arrival at all these places I went through the dissemination office to clarify my purpose and research. At all those offices I also requested an interview with Directors and senior management and in every case these requests were ignored. Moreover, as anyone who has done a similar project can tell you, you cannot access these places without an invitation and a name. When permission to enter is given your name is duly recorded in the big book that sits at the front desk of official buildings. I have never needed to sneak in through any backdoor. I was welcomed in at the main gate.
When Pali Lehohla claims “˜that I have not done my research’ it is not only completely wrong, it also stands in striking contrast to the position of the Director in Zambia, who penned a lengthy reaction to Poor Numbers making exactly the opposite argument. His problem is that I have done my research and that I have done it well. I am sure that in retrospect the Director wishes that he had paid closer attention to national account statistics in Zambia so that he might have noticed or answered my responses.
Note that neither Pali Lehohla, nor the Zambian Director, nor any of the other reports that replicated the study I conducted in Poor Numbers, has found any cause for disagreeing with the diagnosis. To understand how African statistical institutions find themselves in the current situation we need to understand the interplay between states, donors and consultants – and how the demand and funding for data affects the quality of data supplied.
From Poor Numbers to statistical tragedy and damned lies
The potentially controversial content of the book and its material became clear to me when I presented part of the work in South Africa in 2011. I documented the upward revisions in Ghana and elsewhere and showed the very uneven application of methods and data in the African region. In response to the talk, Shanta Devarajan, Chief Economist for Africa at the World Bank, declared Africa’s “˜Statistical Tragedy‘. It was a bold statement, it was not entirely accurate, but I think it was instrumental in bringing the debate forward.
This declaration, together with the news stories emerging about GDP revisions in Ghana, and forthcoming revision in Nigeria, made it abundantly clear that not only was the knowledge problem bigger than many had thought, these numbers also really matter.
In 2012, I was asked by African Arguments to write a summary of my argument, and explain in layman terms how one country, like Ghana could go “˜from being one of the poorest countries in the world one day to an aspiring middle-income one the next’. My intent with the piece was to demystify the process and to lay bare the basic discrepancies between global standards of measurement and local challenges of availability of data and resources.
That worked to some extent. However, when the Guardian reprinted the story, they smacked the headline: Lies, damn lies and GDP on it. As anyone who has read that piece or my book will know, I go to some lengths to dispel the beliefs that there was a hidden political agenda behind this revision. Indeed, one of the things that has struck me in the course of discussing my book during the past year is how data users manage to maintain an inherent suspicion of any “˜official number’. Meanwhile, critical skills often seem to fail when it comes to thinking about the basic problems of converting a complex reality into simple numbers. A similar misguided gut reaction exists among scholars, who may never trust a number from Sudan, Ghana or South Africa, but would not hesitate to use the same number if the World Bank had recycled it.
The story travelled across to France, where an interview with me was published with the headline “˜Le-Grand-Mensonge‘. At this stage it is perhaps not surprising that some actors started to respond to this as a publicity problem. As I just stated, one problem of providing statistics is that you always have to defend your numbers in public – otherwise your institution might suffer a serious credibility problem.
In the case of the African Development Bank, I think some individuals were getting concerned that if too many questions were asked surrounding the accuracy of the GDP numbers, then this might negatively affect the decisions of investors. And rightly so – during this period I got frequent calls from investment banks who wanted to know how big the Nigeria’s GDP really was. In response to the review of Poor Numbers in the Financial Times, Mthuli Ncube, Chief Economist and Vice President, African Development Bank Group, wrote to the newspaper that Africa’s rise was real, and despite the uncertainty around the numbers “For investors, for visitors, for Africans themselves, seeing is believing. The growth is tangible. Come and see for yourself.”
I am somewhat sympathetic to his position. However, the reason we produce statistics is in order to be able to make informed decisions about the magnitude and pace of economic growth. Without these statistics it is impossible to properly analyse the distribution of income or the growth effect on poverty.
It is not often that a book gets mention in the reports of international organizations. Poor Numbers did not only make its way into the references, but the IMF’s Regional Outlook Reports for Africa team, as well as UNECA and AfDB, actually carried out replication studies of my information on the comparable statistics in African countries. The AfDB report spent most of its time responding to the stories in the media, but also found fundamentally the same patterns when it comes to painting a picture of the current situation in provision of economic statistics.
The survey work done for Poor Numbers was undertaken in June 2011, and double checked in November 2011. I spend 14 pages (pp. 123-137) detailing the provenance of each and every observation in my table in Chapter 1. It is remarkable to read the reports in the IMF and AfDB (published in May and June 2013 respectively) and see the differences in information reported. The AfDB has only managed to report information on base years in 34 countries (in the IMF report there is information on 45 countries) but the information in the IMF table and the AfDB report does not cohere.
It is manifest to the knowledge problem on African development statistics that such key disseminators of information cannot agree upon the simple facts in the metadata on national accounts in Africa. However, these reports, Poor Numbers, and even the otherwise so wildly defensive, incorrect and paranoid statements from the directors of statistics in Zambia and South Africa, do not actually at any point disagree with the diagnosis. Their concern is who has delivered it and what implications it will have for their own personal future.
The politics of investing in statistical capacity
When Bill Gates wrote a review of Poor Numbers he said: “[I]t is clear to me that we need to devote greater resources to getting basic GDP numbers right. As Jerven argues, national statistics offices across Africa need more support so that they can obtain and report timelier and more accurate data.” This should be wonderful news, but the road ahead is not that easy. It requires more than simply increased resources – the main problem is the incentives and the political economy surrounding the provision of statistics.
So while Poor Numbers launches the idea of improving the current situation, it does point out that it is not only a question of funding – it is about incentives and global governance of the demand for data. There is a lot of diverging interest among international organizations. The World Bank wants more credit for its new lending programme STATCAP. Paris21 advocates National Statistical Development Strategies. The African Development Bank wants to find funding for a large round of economic censuses, while UNECA wants to oversee the implementation of the newest statistical methods for calculating GDP. These are just a few of the programs out there, and these organizations have sought to engage Poor Numbers to their benefit. Quite naturally, gaining support for “˜their’ program or “˜agenda’ sometimes comes before actually getting the job done: improving measurements.
If you thought that there was a data revolution taking place, with the world united towards achieving a better measurement of development, then I am afraid you are misguided. Lack of firm facts leaves convenient room for negotiation of the numbers when it is needed. Currently the data, if they are available, are not timely or of the quality needed, and hamper any serious agenda for economic governance or development planning. As any reader of Poor Numbers will know, the big story in the book is governance by ignorance.
As I have pointed out, this stands in striking contrast to the demand for data in the development community. The most extreme version of “˜evidence based policy’ comes from those who suggest that we tie financial rewards directly to statistical evidence of success. The trend is that donors are increasingly demanding monitoring and data in return for funding. In this situation, donors can ask states to provide the data for them – with the huge gaps in data combined with blatant incentives to distort it, the average outcome will be “˜policy driven evidence’ rather than “˜evidence based policy’. Alternatively, donors disengage completely from statistical offices and fund their own data collection, evaluation and dissemination. There are marked moves in this latter direction. For example, the USAID funded Demographic Health Surveys are increasingly being viewed as the “˜gold standard’ and the data is increasingly used by economists.
Conclusion
I am concerned about this trend. I worry that while we demand evidence for policy, we forego the opportunity to invest in accountability. I think it is a mistake to think of data as a technocratic search for facts – it has to be viewed as an exercise in building institutions. For all this talk of “˜institutions matter’ and “˜governance’ in development circles, there has been a surprising gap in analysing the statistical office.
We need to rethink the demand for data and how we invest in data in Africa and beyond. My focus has been on Africa because the problem is particularly striking there. To fix the gaps we should first re-think the MDG and other donor agendas for data and do a cost benefit analysis – what are the costs of providing these data and what is the opportunity cost of providing these data? The opportunity cost is often ignored. Local demand for data needs to come into focus. A statistical office is only sustainable if it serves local needs for information. Statistics is a public good, and we need a good open debate on how to supply them.
It is in this light that I am so concerned about the reaction of Pali Lehohla and his silent (or not-so-silent) allies. Ultimately, this may undermine statistical offices in the region. What I am arguing is precisely that we need to strengthen the statistical offices and connect them closer to local needs for information. When one flies in a team of consultants that hire local staff to fill in the questionnaire and the only lasting legacy of the survey is the dataset. At that institution the staff will be waiting for the next payday in the form of a new survey. Maybe Lehohla and other statisticians are worried that I am standing between them and another payday. That is exactly what the Director of Statistics in Zambia is accusing me of in his public letter – he worries that I am trying to distort per diems provisions from him and his colleagues towards foreign consultants.
I can assure him and others that I have no such agenda. My “˜agenda’ has been to draw attention to the unevenness in statistical capacity, and open up a discussion about how to invest in local statistical offices in a way that is suited to local needs – designing incentives and institutions in a manner that matches local institutions. Unfortunately, the recent events may bring support to those who are pessimistic about the possibility of bringing transparency and real reforms to bring better data for development in the future. I accepted the invitation to present my work at PARIS21 in May and at UNECA in September as a part of the public service a scholar can contribute to the exchange of ideas. It is frustrating when other agendas stand in the way of such an exchange. In the meantime, I take some comfort from the many expressions of support that have been relayed to me. I hope, when the dust settles, that we are ready for a free and open debate where all parties feel able to take part.
Morten Jerven is Associate Professor at the Simon Fraser University, School for International Studies. His book Poor Numbers: how we are misled by African development statistics and what to do about it is published by Cornell University Press.
[i] These countries are Ghana, Nigeria, Malawi, Uganda, Tanzania and Zambia – in Botswana I had no invitation so my study on Botswana is based on archival work. My survey had responses from Burundi, Cameroon, Cape Verde, Guinea, Lesotho, Mali, Mauritania, Mauritius, Morocco, Namibia, Mozambique, Niger, Senegal, Seychelles and South Africa.
[…] a href=”https://africanarguments.org/wp-content/uploads/2013/09/MortenJerven-e1380186364989.jpg”>; Morten Jerven: “allegations that I am a ‘hired gun’ or… […]
To anyone familiar with the way things actually work in the African context nothing could be more obvious than Mr. Jerven’s warning. Here EVERYONE knows the numbers are shaky. It’s assumed however that the statistical people are slowly getting it together and that good numbers will someday result. Its also assumed that that process is often slowed by official statistician’s resistance just as in any system where personal interests are in play. Those statisticians might argue the media storm must be generating a nice sales spike for the Mr. Jerven’s book. In the end this issue IS important but the discourse we are having about it isn’t helping very much. We need a discussion on agreed common ground from where we can work outwards and build consensus. What the Poor Numbers debate really shows is how difficult it is to make progress under the present model which seems to be two fellas arguing it out under the banyan tree.
The defensive reaction of Pali Lehohla and others like him is indeed cause for concern. At least, the scientific community across board is or should be grateful for Jerven’s Poor Numbers. As a researcher, I am of the opinion that statistical data have to be approached with a degree of suspicion, not just in Africa. In any country, numbers can be ‘fictionalised’ to serve political purposes. The problem with African statistics goes beyond political manipulation of numbers; it is a serious issue of technical competence and a disregard for the epistemic value of statistical data.
People who try to silence others, rather than answering them with provable reasoning or evidence, create the impression that they have a lot to hide. African economic progress can only benefit from honest discussion of data quality and how to improve it. Please see Devarajan, S. (2013) Africa’s statistical tragedy. Review of Income and Wealth Online, 2013. At http://onlinelibrary.wiley.com/doi/10.1111/roiw.12013/full and M. Lipton, ‘Africa’s national-accounts mess’, ‘Africa’s National-accounts Mess’, Journal of Development Studies, forthcoming 2013, and available from Taylor & Francis Online at http://www.tandfonline.com/doi/abs/10.1080/09662839.2013.822668 Michael Lipton http://www.michaellipton.net/
I have observed and read with interest the reaction to the position that Africa took on Poor Numbers by Professor Jerven. None of the persons who responded attempted to read the documented evidence on Africa’s numbers which includes amongst others deep cutting institutional reforms and reviews of legislation based on fundamental principles of official statistics. The only chorus in this song is about Professor Jerven’s novel, instead of empirical evidence of numbers as they come up out of African countries. The chorus is largely innuendo about what others know about Africa.
There is a level of practice expected in academia. One such is to bring evidence that you are alive to global practice in official statistics, yet Prof Jerven appears to be oblivious of what has brought international and domestic criticism to Canada’s own practice at exactly the same time he was preparing for the launch of his book. A concerned intellectual on matters official statistics would have not missed the sharp criticism and correspondence and protestations of to the Prime Minister of Canada by a renowned scholar and practitioner who led StatCan to its glory days of practice for more than five decades and was at the helm for at least twenty three years. Ivan Fellegi stays true as a scholar and practitioner and gives hope that global practice enjoins us all in measurement. Professor Jerven ows it to the intellectual community to display this high level of alertness to official statistical practice and above all this is a matter of intellectual honesty. Myopia of this magnitude at his own door step baffles the mind.
Without imputing what might not be the case, I dare say I would be derelict not to state that in Vancouver the paper by the African delegation was titled Statistical Renaissance in Africa and it so happens that this title is designed for a book that we are writing and Professor Jerven is fully aware of this project. Whilst we do not have copyright to the title of the book, Statistical Renaissance, surprisingly Professor Jervens intended address to the ECA was incidentally titled “From Statistical Tragedy to Statistical Renaissance.” The two points I have raised plus others that require a much longer treatise leave bad intellectual taste on what the intellectual intent of the book Poor Numbers was.
[…] I had an invitation and introduction to all the offices I visited. Date and place of interviews at statistical offices, central banks and donor missions are documented. Between 2007 and 2011 I wrote letters, emails and phoned all statistical offices in Sub-Saharan Africa repeatedly in order to verify information, request access and set up interviews. As anyone who has tried something similar can attest, the response rate is extremely low. For all the places I did go to I had a response, a contact and an invitation.[i] […]
So, Mr Pali Lehohla, the Canadians are doing something wrong. Does this mean that what we are doing is right? Your response seems to be aimed at deflecting attention from the issue at hand, the numbers in Africa. If Professor Jerven has got it all wrong in so far as South Africa is concerned, it’s my opinion that you need to take some of the responsibility for this. Why did you not engage openly with him when he was conducting his research instead of keeping your doors closed to him? Perhaps you could have demonstrated that your methods are sound. As things now stand, I’m left suspecting that you are trying to hide something.
Also, Jerven is Norwegian. Not sure why Canada, or nationalism in general, is relevant. Agree that the scientific community should be grateful for Jerven’s Poor Numbers. Attempts to censor academic debate are backwards and unacceptable.
Shame on UNECA for folding to pressure from Lehohla. They should be safeguarding statistics.
Really sad to see one of our “best” statistician in Africa using extremely weak arguments to make his point. The truth is: he has no point to make and has no interest in the improvement of statistics in Africa.
1. SA likes the status quo – they are the best of the worst
2. Its all about the money – Funding for capacity building and technical assistance
3. There is no point arguing or challenging Morten research or findings, just take a plane and go to one of statistical offices – the empirical evidence is every where to see.
4. Comparing the Canadian census issue with the state of statistic offices in Africa is simply nonsense. At least Canada can count and back the numbers.
4. Typical way of doing things in Africa, we are always late in the game and then we cry like babies because Morten is getting all the attention.
The renaissance won’t come from people who have failed for the past 50 years – IMF, WB, States, Donors – you name it. Time to get past criticism and time to fix it! That’s all that matters
[…] most recent person that raised the issue again is Professor Moten Jerven. His latest book: Poor Numbers: How We Are Misled by African Development Statistics and What to Do About it. The book spoke of specific institutions as the sources of the problem. But, his first opportunity […]
I tend to agree with Prof’s findings with regards to lack of statistical knowledge in most if not all NSO’s in Africa including SA. Statisticians position are held by those with no statistical qualifications hence lack of quality of data collected by these offices hence they against the challenge from Denver to have their offices analysed to rdetermine the extent of statistical capacity and knowledge gap.
Statistical capacity is what these meetings talks about but in reality graduates from different statistical institutions around Africa roam the dusty streets in there respective countries without absorbed in these office because they pose a threat to the self-proclaimed thclaims to who continues to frustrate users with their guesses.
Pali and his many friends should start realising that behind the so-called numbers,there is humans beings and affects their lives tremendously.We’re not interested in the manyd
Books,documents,papers they write or present but on realible and accurate statistics for the development of our country and the people living in it.When will we have a situation when heads of statistical offices held accountable?
I managed Egypt’s Household Income Expenditure and Consumption Survey from 1995-1995.
I helped the Kenya Central Statistical Office write the Urban Household Budget Survey before the political situation brought the halt to the project. The World Bank picked it up later.
I worked on the Government of Tanzania’s (Public Service Management) Client Service Charter (CSC) Program, and I helped draft questionnaires for each of the stakeholders. I felt reasonably capable doing that because I lived in Tanzania from 1984-1988. I taught Econ there.
I am an employee of BEA, having been for 15 years (I do not work directly on the GDP – I am a student of its validities). I have devoted my life to bias and variance in official statistics, and refinement in the process of assigning numbers to Economic constructs. I consider myself a scientist.
I refuse to characterize Africa as monolithic by claiming to know about “statistics in Africa.†Truly, truly, that phrase in meaningless because it lacks LOCAL characterization.
No one wants to hear about mixed bag results, but frankly the “African statisticians†that I have met have been much more dedicated to telling the truth, ultimately, than the “Western statisticians†who simply go about their jobs for a paycheck. This is my OPINION, and it is a biased one.
As far as the science of estimation goes, I think it is obvious to anyone who has tried to aggregate, that conditions (data sources, transport, management, politics, access to data, opportunity costs, wariness to respond, etc., etc.) make matters of measurement more difficult in the Third World. The numbers that have been produced still indicate something, though it may not be exactly what the design intended. Try sampling a country like Egypt, with 80 million people, without making a mistake here and there!
That’s why, I really think, the only thing the First World should offer is sustainable technical Best Practices capabilities; finding fault with these statistics is unsophisticated and presumptive: as if we know what the best estimators are! As if we’re the arbiters of truth! That’s fricken Eurocentric.
plaquenil pill is plaquenil an immunosuppressant hydroxychloroquone
cialis without a doctor prescription cialis 20 mg