Transparency of public procurement has been one of the most important areas of government spending on the agenda of governments, civil societies and businesses around the globe in the past decade.

Initiatives such as the Open Government Partnership, which has seen 70 members making 189 Open Contracting commitments by March 2019, underline the significance of public procurement transparency. However, despite such major policy developments, we still lack specific and rigorous evidence of the impact of transparency interventions targeting public procurement.

In other words, how do we know which types of transparency matter for which actors when aiming to achieve high-quality public services, procedural justice or public sector integrity?

To answer this question, the Government Transparency Institute has conducted a research study supported by DFID and a consortium of not-for-profit organisations (Open Contracting Partnership, Hivos, The B Team) interested in improving procurement policy and impact. The research explores short-term effects of transparency reforms on corruption risks, institutional efficiency, competition and prices, and on using public procurement databases. Procurement outcomes were compared in three countries (Mexico, Paraguay and Slovakia) before and after transparency regulation changes took place to determine whether the reforms had a measurable impact.

The three selected countries all recently started publishing more information related to their tenders. The information can be found and accessed easily by users. All countries had procurement data available from both before and after these reforms, meaning each of these cases represented a data-rich environment to start with.

The selected transparency reforms mostly led to more available data in more accessible formats and were accompanied by some publicity, workshops, and trainings. This made it possible for the researchers to study the effects of the changes in the information that was published. In theory, a data publication depends on: what kind of new data is being published about what area of procurement, and are stakeholders willing and able to use the information published.

The selected countries with distinct transparency reforms targeting public procurement:

Overall, the findings show some individual transparency reform effects in all countries.

In Mexico, the level of competition increased in tenders that were already competitive, but it also increased the share of high-corruption risk, single-bidder contracts in the very short-run. In Paraguay, the results are mixed. On the one hand, the level of competition slightly decreased – fewer bids were submitted and the share of new companies winning decreased. On the other hand, the average submission period length got longer by 8 days which may be due to increased scrutiny thanks to better availability of data. In Slovakia, the share of single-bidder contracts decreased by 19 percentage points, and the number of bids per contract increased by two on average. However, interviews confirmed that these differences are at least partly due to changes in government staff, a host of new regulations, and an overall shift in public contracting norms.

In sum, these findings do not reveal a consistent picture of systemic impact, as no statistically significant effect of substantive size was identified in the three countries’ public procurement datasets. So what did we learn from this research exercise?

From a methodological perspective, data quality remains a challenge even in countries with good quality data by global standards. Public procurement data needs to be of high quality throughout the whole comparison period both in terms of its scope, the availability of data fields and the truthfulness of the recorded information. Transparency reforms improving the scope and quality of public procurement data are highly valuable on their own as open data is as good as the data going into it. However, evaluating such reforms will remain a challenge as the reform influences the data used to measure outcomes and it is hard to isolate a reform from other developments in procurement governance happening at the same time.

Our research has revealed that increasing the amount and accessibility of data publication in public procurement is unlikely to lead to short term improvements in procurement outcomes in countries with considerable data transparency at the outset. Nevertheless, we see some early signs of a potential impact in selected cases where a longer time window or more investment into promotion, skills, practice change, and data use may lead to robust, sustained, systemic change. These deserve further investigation, e.g. adding further data sources and combining them with interview evidence, document reviews and surveying data users – for example, public buyers – could reveal whether and how open data is used, and how it could be made more useful.

The detailed research report by the Government Transparency Institute can be found here