Publications

The contributors come from a broad range of disciplinary perspectives including law, public administration, applied philosophy, data science, and artificial intelligence
1: Algorithmic Regulation: An Introduction, Karen Yeung and Martin Lodge
Part I: Normative concerns
2: Why worry about decision-making by machine?, Karen Yeung
3: Machine decisions and human consequences, Teresa Scantamburlo, Andrew Charlesworth and Nello Cristianini
4: Digital discrimination, Natalia Criado and Jose M Such
5: The ethics of algorithmic outsourcing in everyday life, John Danaher
Part II: Public sector applications
6: Administration by Algorithm? Public Management meets Public Sector Machine Learning, Michael Veale and Irina Brass
7: The Practical Challenges of Implementing Algorithmic Regulation for Public Services, Alex Griffiths
8: Reflecting on Public Service Regulation by Algorithm, Martin Lodge and Andrea Mennicken
Part III: Governing algorithmic systems
9: Algorithms, regulation and governance readiness, Leighton Andrews
10: Legal practitioners' approach to regulating AI risks, Jason D Lohr, Winston J Maxwell and Peter Watts
11: Minding the machine v2.0: The EU General Data Protection Regulation and Automated Decision Making, Lee A Bygrave

The EU's new approach to policy evaluation is characterised by a focus on closing the policy cycle (linking ex ante and ex post appraisal) and by applying evaluation to all types of policy intervention, whether expenditure or regulatory policy. This article analyses the main features and challenges of this new approach. It first studies the conceptual and interdisciplinary challenge of such an encompassing approach to evaluation. It then assesses the new approach in the light of four key objectives of ex ante and ex post appraisal; ensuring evidence and learning; accountability, transparency and participation; policy coherence; and reducing the regulatory burden.

Evaluations may perform a key role in political systems as they provide a basis for parliaments to hold their executives accountable. This is equally the case in the European Union. Yet, several factors may work against the usage of European Union evaluations for accountability purposes. Members of the European Parliament work under great time pressure and executives may have little incentives to produce high-quality evaluations. This article therefore addresses the question of to what extent and when Members of the European Parliament use ex post legislative evaluations. We present an analysis of 220 evaluations, studying how many were referred to in parliamentary questions. Our main finding is that 16% of the evaluations are followed up through questions. However, the parliamentary questions hardly serve accountability aims. Members of the European Parliament mostly use evaluations for agenda-setting purposes. The main variable explaining differences in the usage of evaluations is the level of conflict between the European Parliament and Commission during the legislative process.

Governments are increasingly trying to limit the costs of regulatory compliance. One of the approaches that has been gaining ground in the last five years is the “one-in, x-out rule”, or the offsetting of regulatory costs stemming from new regulations by reducing the existing regulatory stock. This paper presents examples of regulatory offsetting approaches in selected OECD countries. By comparing the different approaches and discussing their key features, the paper provides guidance to countries considering introducing regulatory offsetting. This paper finds that there are many methodological and implementation issues that need to be resolved before a government decides to use a one-in, x-out approach as part of its regulatory policy. Key suggestions for countries introducing regulatory offsetting include i) ensuring a solid methodology for calculating regulatory costs; ii) linking the responsibility for finding offsets to the “owners” of regulation; iii) setting up quality oversight mechanisms; iv) securing strong political commitment and support and v) implementing regulatory offsetting as a complement to other regulatory management tools

About ex-post review of legislation and better regulation
I. The ex-post review of legislation is a key part of the Commission’s Better Regulation
policy. It is aimed at facilitating the achievement of public policy objectives at minimum cost
and improving the added value of EU interventions. In 2015, the Commission strengthened
its better regulation policy by launching the Better Regulation Agenda.
How we conducted our audit
II. In our audit, we assessed whether the EU system of ex-post review of legislation had
been properly planned, implemented, managed and quality-controlled, thereby contributing
effectively to the Better Regulation cycle.
III. The audit covered ex-post reviews of legislation carried out between 2013 and 2016 by
four directorates-general of the Commission as well as all legislation and ex-ante impact
assessments within the remit of those directorates-general adopted between 2014 and
2016.
What we found
IV. Overall, we concluded that the Commission’s current ex-post review system compares
well to the situation in the majority of Member States. Regarding more specifically the
evaluations, the Commission has designed a system which is, as a whole, well-managed and
quality-controlled, thereby contributing effectively to the Better Regulation cycle. However,
when it comes to reviews other than evaluations, we identified weaknesses.
V. We found that review clauses and, to a lesser extent, monitoring clauses are widely
used in EU legislation. However in the absence of common inter-institutional definitions and
drafting guidelines, their content and therefore their expected outputs are not always clear.
VI. While evaluations are generally carried out in line with legal requirements and good
practices, this is less the case for the other reviews, to which the Better Regulation
guidelines did not apply until 2017. We also identified shortcomings in the presentation of
the methodology used and in the recognition of data limitations when applicable.
7
VII. We also found that ex-post reviews are publicly available and accessible and that the
vast majority of them provide a clear conclusion and indicate next steps to be taken. The
Commission systematically forwarded its reports on the ex-post reviews to the co-legislators
(European Parliament and Council); the latter, however, seldom react to the Commission
directly. Also, the ex-post reviews are not always used by the Commission when preparing
ex-ante impact assessments. The inter-institutional agreement between the European
Parliament, the Council and the Commission on better law-making, which provides provision
on the review of existing laws, is not binding.
VIII. Finally, we found that the rationale of the REFIT programme is unclear, as are the
criteria by which individual initiatives have been labelled as REFIT. At the same time, the
guidelines present REFIT as a specific programme. This raises questions as to its current
nature and added value.
What we recommend
IX. On the basis of these observations, we make several recommendations to the
Commission and one to the Regulatory Scrutiny Board.

The Regulatory Scrutiny Board (RSB or ‘the Board’) was set up under the Commission’s 2015 better regulation policy. It scrutinises the quality of impact assessments, fitness checks and major evaluations for
the Commission.
The Board is a procedural safety mechanism governed by a mandate (1
). It acts during the early stages
of preparing legislation, and it helps to protect Europeans against poorly conceived laws. It also helps
to find solutions for evaluations and impact assessments that are meant to inform policy decisions. It
allows work that is in good shape to progress quickly with its stamp of approval, but is also able to halt
the preparatory process. Any political decision to override the Board requires a public explanation why.
Fast facts:
• The Board is independent and reports to the President of the Commission.
• There are seven full-time members, both from inside and outside the Commission.
• The Board scrutinises all impact assessments, fitness checks and major evaluations.
• All opinions are published.

Last 12th June, the OECD launched a consultation on its “best practice principles” on regulatory impact assessment (RIA). The paper extends and elaborates the 2012 “Recommendation of the Council on Regulatory Policy and Governance”, with the aim of providing member countries with a practical instrument to better design RIA systems. Although not particularly innovative in content, the “principles” outline some of the most recent trends in the international debate over RIA, such as a focus on governance issues and on the evaluation of non-economic impacts. (Siriana Salvi, Rassegna Osservatorio AIR, n. 4/2019)

As behavioural insights (BI) become more widely used, countries are looking to expand the application of the methodology to new frontiers of policy making.

Almost a decade after the OECD reviewed the better regulation practices in 15 EU
countries, this report presents an up-to-date analysis of the use of core regulatory policy
tools across all 28 EU Member States and the European Union. In 2017, both the
European Union and EU Member States show a strong overall political commitment to
regulatory reform. All EU Member States have adopted regulatory policies promoting
government-wide regulatory reform, covering various areas of regulatory governance.
Regulatory policy in the European Union progressed under the better regulation agenda,
which played a crucial role in shaping the current Commission’s regulatory processes.
For the first time, the OECD has assessed stakeholder engagement, regulatory impact
assessment (RIA), and ex post evaluation systematically across all EU countries and the
European Union, drawing on its composite indicators of regulatory policy and
governance. EU countries have, on average, invested less across the three assessed areas
than non EU countries. The difference is particularly striking for the area of ex post
evaluation, indicating that EU Member States have yet to develop effective systems to
review existing regulations. While therefore more progress is needed, many EU countries
have significantly improved their regulatory management practices over the last decade.
For instance, substantive investments have been made by EU Member States to seek
input on draft laws from affected parties, especially via electronic communication.
Furthermore, nearly all EU Member States have embedded RIA as a core part of their
regulatory management toolkit. Although the European Union as an institution scores
favourably compared to most of the Member States, the implementation of regulatory
management tools can still be improved by all.
EU countries do not usually facilitate the early engagement of their citizens in the
European Commission’s regulatory proposals. The Commission uses a range of different
tools to engage with stakeholders at various points during policy development. EU
countries and their citizens thus have opportunities to participate, provide evidence, and
improve EU laws, including at early stages of their development. Many member
countries, however, do not sufficiently inform their stakeholders of these opportunities to
provide input. Instead, most stakeholder engagement with EU laws occurs after the
Commission has made a regulatory decision via individual transposition procedures. At
this stage, the focus of consultation is generally on the implementation of EU directives,
rather than on their expected societal impacts. To ensure that EU laws benefit fully from
stakeholder consultation, Member States should provide better evidence and information
during regulatory design to complement the existing practices of the European
Commission.
Where Member States’ regulatory practices are poor, the potential benefits of EU laws
will be reduced. For example, if EU laws are implemented in a piecemeal manner, the
resulting regulatory burdens will be higher than they should be, hampering investment
and reducing competition, as well as posing a risk to the single market. Where EU
countries include additional regulatory measures in excess of those provided in EU laws,
16 │ EXECUTIVE SUMMARY
BETTER REGULATION PRACTICES ACROSS THE EUROPEAN UNION © OECD 2019
it is important that these measure be subject to appropriate consultation and impact
assessment as part of their design, to ensure that the anticipated gains from EU laws are
realised.
With respect to domestic law-making in Member States, stakeholder engagement often
takes place too late in the policy development process to be of real value. Perhaps more
striking is that, in some Member States, stakeholder engagement is still not sufficiently
broad. The vast majority of EU countries have heavily invested in the establishment of
online government portals to better communicate with affected parties when developing
laws. While praiseworthy, these investments are not yielding their full potential benefits.
For example, stakeholders are generally not informed by policy makers about how they
have helped to shape and, ultimately, improve regulatory proposals. This may lead to
disenchantment among stakeholders, and possibly to the rejection of laws, and to
diminishing voluntary compliance and engagement in future stakeholder consultations.
Greater accountability for the results of consultation is not only needed in EU countries
but also in OECD countries more broadly, the 2018 Regulatory Policy Outlook found.
Many EU countries are also not reaping the full benefits of using regulatory impact
assessment to aid domestic law-making. They do not systematically assess alternatives to
the proposed regulatory option, and where a triage procedure exists, it tends to focus on
the cost to business only when determining a proposal’s potential impact and the
corresponding level of assessment required. The RIA process still begins only after
regulatory proposals are developed and decided upon by governments. Furthermore, there
is no real incentive to change practices, as there are little or no consequences to producing
poor quality RIAs. Despite its instrumental role, oversight is still one of the least
developed features of regulatory policy in many EU countries.
All EU countries and the European Union itself remain more adept at making laws than at
ensuring they continue to deliver benefits to communities. Laws are not systematically
subject to ex post evaluations in almost all EU countries, creating a risk that obsolete laws
remain in force. This represents a substantive waste of resources to the economy: to
governments, in terms of unnecessary inspections and enforcement; to businesses, in
terms of excessive regulatory burdens; and to citizens, in terms of reduced choice,
increased prices, and exposure to potential risks when regulations do not keep pace with
societal changes. Where ex post evaluations are conducted, they tend to be unstructured,
and do not systematically allow for public consultation or impact analysis. Worse still,
ex post evaluations do not systematically assess whether regulatory goals have been
achieved—something of vital importance for establishing whether laws remain
appropriate. These findings are in line with the findings for OECD countries.
The better regulation agendas of EU countries and of the European Union need constant
attention – a ‘set and forget’ model does not work, just as it does not work for laws
themselves. Countries need to strengthen their regulatory processes and the institutions
involved. At a time of fiscal stringency and heightened global uncertainty, regulatory
policy remains a key government tool for ensuring the safety and well-being of citizens
while stimulating innovation and economic growth and prosperity. Despite some
improvements, much work remains to be done to reap the rewards of better regulation.

After a modest recovery, the economy is weakening
Italy continues to suffer from long-standing social and economic problems
A comprehensive reform package holds the key to stronger growth and social
inclusion
In-work benefits and a moderate guaranteed income scheme would boost
employment and reduce poverty
More effective regional development policies and strengthening capacity at the
local level would help to narrow the regional divide