Posted by David Fellows[1]

Introducing the concept of “objective data”

In March 2018, we republished a short note on the use of objective data to combat corruption [2]. The piece highlighted statistical techniques being used in western countries to identify corruption by correlating unorthodox procurement practices with aberrant supplier behaviour established from factually based ‘objective’ administrative data. It was suggested that less complex approaches to the analysis of ‘objective’ data could be used to indicate the need for further forensic examination of officials, suppliers, and politicians. The emphasis was on finding workable approaches for developing countries that were compatible with the available resources.

The term ‘objective’ data refers to factual information derived from official government records. It represents data on transactions, activity schedules, and personal information, recorded through established processes, that give the information credibility. This contrasts with ‘subjective’ data which is often based on opinions or experience that is poorly evidenced and of limited application, as is the case with corruption perception surveys.

Frequent use of objective data

Objective data is checked and compared in dozens of administrative processes which can produce anomalies that may indicate the presence of corruption. For example, invoices are checked against orders and goods received notes or contract certificates, or payroll submissions are checked against timesheets. In addition, national bodies charged with the oversight of public administration – such as supreme audit institutions and public procurement commissions – are routinely engaged in the examination of objective data which can also lead to the identification of corruption.

Such findings are then included in published reports that may be used to identify process deficiencies or potentially to prosecute cases of fraud and corruption. These oversight functions can be particularly effective when they are invested with independence from government, extensive powers of enquiry, transparency of reporting, and due consideration of findings.

Developing objective administrative data systems

Apart from routine scrutiny provided by administrative processes and oversight arrangements, programs of administrative reform provide excellent opportunities for the development of systems that incorporate the automatic validation and cross-referencing of administrative data to help identify patterns of corrupt activity.

Such arrangements are straightforward, well known, and remarkably simple to put into effect but in practice they are rarely complete or well executed. Too often there is a lack of expectation that good administration will have a beneficial effect. This places a premium on those who hold relevant managerial roles, requiring them to value high standards of administrative practice; exercise oversight responsibilities courageously, insightfully and in partnership with others as necessary; and ensure that reform opportunities are used to best effect. Well prepared and committed management is a prerequisite to any well-intentioned anti-corruption initiative.

Objective administrative data applications

Some examples of objective administrative data and its use to combat corruption are included in an Appendix available here.

The use of objective data could also be developed in other ways. For example:

1. Countries could prepare anti-corruption strategies that include the use and development of objective data and staff training. Such strategies should be accompanied by operational guidance. Anti-corruption strategies and related material are often referred to as being part of the standard anti-corruption armoury but are rarely made available. In practice, however, few of these documents have been produced to a reasonable standard anywhere in the developing world, and perhaps it is time to redress this omission.
2. Additionally, collaboration between states, perhaps on a regional basis, could be helpful in developing techniques for interrogating data, preparing anti-corruption strategies, sharing knowledge of corrupt practices, and building operational cooperation between countries
3. Consideration should also be given by multilateral agencies and regional representative bodies to the development of an international systems assessment schema (akin to PEFA methodology[3]) that would indicate the efficacy and shortcomings of individual administrative systems for the purposes of combatting corruption.

This article is written with government administration in mind, but similar considerations apply to local governments and state-owned enterprises.

[1] Director, PFMConnect. The author thanks John Leonardo for his helpful comments.

[2] This blog was first published at http://blog-pfm.imf.org/pfmblog/2018/03/how-useful-are-perception-indices-of-corruption-to-developing-countries.html

[3] See https://pefa.org/sites/default/files/PEFA%20Framework_English.pdf

In a March 2018 blog PFMConnect co-principal David Fellows discussed the deficiencies surrounding corruption perception indices and outlined how objective data analysis could offer a clearer insight into the systemic nature of corrupt behaviour, thus providing a more precise indication of the corrupt parts of an administration, the number of external parties that are engaged in corruption, and features of the public financial management (PFM) system that need to be strengthened in order to combat corruption.

In a forthcoming blog “Developing Systems to Combat Corruption”, David describes how an objective data system is used in practice and how the concept may be developed. Some further examples of objective data and their use to combat corruption is available here.

Posted by David Fellows[1]

The value and limitations of perception indices 

There are numerous corruption perception indices. They provide an outsider’s impression of the prevalence of corruption across the various branches of government. Some indices focus on issues of bribery, others are more general in scope. Some indices aim to engage with the general public, and others with businesses or NGOs. Perception indices can incentivise governments to tackle corruption given the reputational damage that they can inflict.

The shortcomings of perception indices, however, have been widely recognised, including in recent studies by UNDP and the IMF[2]. Their evidential base is limited; survey samples are generally small; within the same index a variety of methodologies may apply so they can lack internal consistency; methodologies change so trends can be questionable; standardisation is difficult to achieve between or even within countries and, as a result, the ranking of countries can vary from one perception index to another.

The relevance of objective data     

Those agencies and officials responsible for preparing these indices are aware of the deficiencies and make considerable efforts to mitigate them. Their key deficiencies are unassailable, however. Perception indices are based on impression, personal experience and hearsay rather than hard fact. In a multi-faceted study of villagers’ perceptions of corruption affecting road building in Indonesia, Olken finds that perceptions are a good indicator of the presence but not the quantum of corruption. He concludes that “there is little alternative to continuing to collect more objective measures of corruption, difficult though that may be”[3]. These factors can allow governments to diminish the importance of the messages that perception surveys contain.

An alternative approach has been proposed in a recent paper by Fazekas[4]. The paper gives an account of recent research into public procurement in which legal, regulatory and administrative records have been analysed to reveal the presence of corruption. Relevant factors include: the characteristics of the tendering process; the political affiliations and personal connections of suppliers; and the location and transparency of information about the ownership of these supplier companies. Fazekas correlates these various data sets to reveal behaviour that indicates a skewing of contract awards toward suppliers with particular characteristics.

Fazekas uses the term ‘objective’ to refer to factual data that are not mediated by stakeholders’ perceptions, judgments, or self-reported experiences. Nevertheless, the data are based on provable characteristics (e.g., from suppliers and procurement agencies). This approach, however, can provide some significant challenges. Databases may not be available electronically, thus hampering data collection, and information is not collected on a systematic basis across countries. Despite these reservations, the approach can produce valuable evidence identifying areas of public administration that are especially prone to corruption, the role of officials in facilitating corruption, and the means by which corruption is being perpetrated.

Objective data analysis and developing countries

European countries and the USA have been at the forefront of this kind of work, but it also has potential for guiding administrative scrutiny and reform in developing countries. The necessary analysis could be undertaken by internal auditors, anti-corruption agencies, or other oversight bodies. These agencies could use the results to improve system design, and commission detailed forensic investigations of those concerned.

Fazekas uses sophisticated statistical techniques, but simpler methods could also be employed to measure inappropriate administrative processes, potentially illicit flow of funds between parties with close personal ties, the unexplained accumulation of personal wealth, citizens’ complaints, and other indicators of corruption. These results could then be used to identify potential levels and sources of corruption and, if acted on, lend credence to the government’s anti-corruption commitments.

The approach outlined above is relevant to national and local government, as well as public corporations where significant levels of corruption can occur at the highest levels. Such work could be enhanced through external moderation and research collaboration across national boundaries, perhaps at regional level. A recent piece by the present author, published here, discusses the growing relevance of digital media to governance reform.

The importance of national leadership

Objective data analysis can offer a clearer insight into the systemic nature of corrupt behaviour, thus providing a more precise indication of the corrupt parts of an administration, the number of external parties that are engaged in corruption, and features of the PFM system that need to be strengthened. It can provide data to support a vigilant administration that wishes to maintain pressure on corruption, complementing efforts to increase prosecutions or administrative reforms.

Whatever ideas are advanced, they will all require commitment from national leaders if they are to succeed.

[1] David Fellows is Co-principal of PFMConnect. He is an accountant and PFM specialist with significant interests in digital service development and performance management. His thanks are extended to Cornelia Körtl and Domenico Polloni for their invaluable contribution to this article.

[2] UNDPs Guide to Measuring Corruption and Anti-Corruption (2015). See also IMF 2017, “The Role of the Fund in Governance Issues – Review of the Guidance Note, Preliminary Considerations”.

[3] Benjamin A Olken, “Corruption Perceptions vs Reality” –  https://economics.mit.edu/files/3931

[4] Mihály Fazekas “A Comprehensive Review of Objective Corruption Proxies in Public Procurement”  https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2891017.

Corruption Correlations

Our blog “International Development and the Challenge of Public Sector Corruption” discusses the results of our examination of correlations for the control of corruption and government effectiveness and public financial management (PFM) performance.

Corruption and Government Effectiveness

Correlations were calculated for the relationships between the control of corruption (capturing perceptions of the extent to which public power is exercised for private gain) and government effectiveness (including the quality of public services) for 184 countries using data from the World Bank’s 2013 Worldwide Governance Indicators (WGI), together with World Bank 2013 per capita income data and Rand Corporation’s Trace (bribery) Matrix risk scores for these countries.

The Trace (bribery) Matrix risk scores have an inverse relationship with corruption control levels i.e. low Trace Matrix risk scores indicate relatively favourable levels of control over corruption whilst high Trace Matrix risk scores indicate relatively poor control over corruption. Strong relationships between WGI control over corruption /government effectiveness scores and Trace Matrix risk scores will result in relatively high negative correlation values.

Results were prepared for the total sample of 184 countries as well as the halves and quartiles of the sample.

Corruption and Public Financial Management

Correlations were calculated for the relationships between some measures of PFM performance and the measures of corruption and government effectiveness for the 39 developing countries for which Public Expenditure and Financial Accountability (PEFA) assessments were made available during the past three years from 2013 to 2015. The respective PFM performance measures used are performance indicators prescribed in the PEFA methodology applicable in 2011 comprising the initial 2005 indicator set and subsequent amendments.

Results were also prepared for this sample of 39 countries as well as the halves and quartiles of the sample.

Correlations download

The correlations are presented in a spreadsheet that can be downloaded here.