Documents
Better Regulation
OECD (2019)
Closing the regulatory cycle: effective ex post evaluation for improved policy outcomes (forthcoming)
1. The stock of laws and regulations has grown rapidly in most countries. However
not all regulations will have been rigorously assessed, and even where they have, not all
effects can be known with certainty in advance. Moreover, many of the features of an
economy or society of relevance to particular regulations will change over time. The
OECD Recommendation on Regulatory Policy and Governance (2012) therefore calls on
governments to “[c]onduct systematic programme reviews of the stock of significant
regulation against clearly defined policy goals, including consideration of costs and
benefits, to ensure that regulations remain up to date, cost justified, cost effective and
consistent, and deliver the intended policy objectives.”
2. The OECD Regulatory Policy Outlook 2015 found that the use of evidence and
analysis in the regulatory process has to date mostly involved ex ante assessments. Given
that the stock of regulation is much larger than the incremental flow, there is large
potential for improving the existing regulatory framework through more systematic
ex post evaluations of regulations.
3. The 9th OECD Conference on Measuring Regulatory Performance was concerned
with the appropriate institutions and processes to effectively implement ex post
evaluation. Discussions focussed on institutionalising ex post evaluation as an integral
part of the regulatory cycle, the results of which feed back into the regulatory process
itself.
4. The following considerations emerged from the discussions as central to the
effective implementation of ex post regulatory reviews:
Ex post reviews need to go beyond a “costing exercise” to focus on whether the
regulation’s underlying policy objectives have been achieved. In addition,
evaluations should take into account any side-effects of regulations and include a
consideration of possibly better alternatives.
A mix of different approaches to evaluation will generally be required, depending
on the context, including reviews of regulations triggered by “sunsetting” clauses
and statutory requirements, and ad hoc reviews. Comprehensive in-depth reviews
or programme reviews that look at the mix of regulations and policy instruments
in certain policy areas or sectors can identify options for more broad-ranging
reforms. Evaluations triggered by “sunsetting” clauses or automatic review
requirements might also be usefully packaged if they address similar or
overlapping issues. Finally, ongoing ‘management’ of regulations, e.g. through
stock-flow linkage rules or red tape reduction targets, can achieve significant
reductions in administrative burdens
Evaluations that are built into the regulatory regime from the outset create
a coherent link between ex ante and ex post evaluation to support evidence-based
policy making throughout the regulatory cycle. This approach can ensure the
setting of clear objectives and enables the early identification of an appropriate
methodology and data needs.
While there is no “one-size-fits-all” institutional setting for ex post evaluation,
independent bodies can play a key role in conducting reviews and providing
4 │
Closing the regulatory cycle: effective ex post evaluation for improved policy outcomes
oversight of evaluations. The more ‘sensitive’ the regulatory area, and the more
significant its impacts, the stronger the case for an ‘arm’s length’ review process.
Less critical evaluations will normally be conducted by line ministries and
agencies themselves. These can also benefit from some form of independent
quality control. Regulatory reviews should not be carried out in isolation but pay
attention to previous evaluations prepared by other institutions, including
parliament and independent bodies.
Consultation with stakeholders is crucial to make sure evaluations are targeted
appropriately and can be informed by the real-world impact of regulations.
Consultation with stakeholders also helps ensure recommendations for
improvement are practical and user-centred. At the same time, those responsible
for evaluation need to account for the diversity of knowledge, resources and level
of organisation of different stakeholder groups.
Getting the timing of evaluations right can greatly increase the impact of resulting
recommendations within the policy-making process. Depending on the political
system, an evaluation conducted at a time when parliament plans a new legislative
initiative in a related policy area may have more impact than one conducted
during an election campaign when results face a greater risk of becoming
politicised. Evaluations may also help inform party programmes or coalition
agreements when published at the right time.
The existence of stock-flow linkage rules such as ‘one-in x-out’ can provide an
incentive and discipline for keeping regulatory costs in check. To be effective,
such regimes should motivate proper evaluation of existing regulations proposed
for removal including assessments against their objectives and broader public
policy objectives.
In order to ensure the quality of ex post evaluations, governments need to invest
in the required skills and capacities within the public sector. A number of
academic disciplines are potentially needed, depending on the area of regulation
being reviewed, but the key skills in common relate to the identification and
measurement of impacts, whether social, environmental or economic