Original article here
Last Friday, September 23rd, 2022 took place the second activity of the periodic Seminars called Del Derecho y del Revés tecnológico: Dialogues between programmers and jurists at the Faculty of Law of the University of Barcelona, in the framework of the research project PID2020-115774RB-I00, entitled PERSONALIZATION OF PUBLIC SERVICES, BIASES AND ARTIFICIAL INTELLIGENCE: TOWARDS THE CONSOLIDATION OF DIGITAL RIGHTS IN PUBLIC ADMINISTRATIONS, whose Principal Investigators are Juli Ponce and Agustí Cerrillo.
On this occasion we had as special guest Dr. Javier de la Cueva. Javier de la Cuevahas a degree in Law, a Master in Advanced Studies in Philosophy and a PhD in Philosophy from the Complutense University of Madrid. He works as a lawyer, as a lecturer at the University of the Instituto de Empresa, and as a researcher. As a lawyer, he specializes in Information and Communication Technologies (ICT) Law. He lectures and writes books on his specialization. He has been a GNU/Linux user since 1998 and a system administrator under this operating system since 2003. He is a trustee of the Civio Citizen Foundation.
The purpose of the activity was the analysis of the Spanish judgment of December 30, 2021 (Central Court for Contentious-Administrative Proceedings No. 8). This judgment has analyzed for the first time in Spain the automated decision making and the obligations of transparency with respect to these, especially with regard to the source code of the program called BOSCO used by the Spanish Administration. Javier de la Cueva himself, in collaboration with Ana Valdivia, has reported on this court decision in English in the German open discussion forum Verfassungsblog.
The background of the judicial decision very briefly explained is as follows. In accordance with Directive 2009/72/EC of the European Parliament and of the Council of 13 July 2009 concerning common rules for the internal market in electricity (art. 3.2), the Spanish parliament passed a law providing for the granting of a social bonus (Law 24/2013, of 26 December, on the Electricity Sector, art. 45, amended several times and developed by various subsequent regulations, included in the judgment we will comment on, which has given rise to high litigation). This voucher is a social benefit aimed at protecting certain consumers with specific personal circumstances and low income («vulnerable consumers») consisting of applying a discount on the price of electricity consumed in their usual home, to prevent them from suffering the so-called energy poverty, which affects many European citizens.
Fundación Civio, a private foundation committed to transparency and good governance in Spain that does extraordinarily valuable work, detected that the lack of information from the administration and the complexity of the process meant that thousands of people were not applying for the new model on time. The Civio Foundation denounced this fact and at the same time created its own app to help vulnerable groups to apply for the aid. Dozens of municipalities, social organizations, groups and media inserted it on their websites, reaching many thousands of people.
The use of that Civio application highlighted that something was not working properly: using the Civio application and the Bosco program at the same time, the results were not identical, with the second one denying applications from people who were entitled to the social bonus, according to the current regulation. Fundación Civio then collaborated with the Spanish Government to try to alleviate the problems detected and requested in 2018 access to the source code of BOSCO, in accordance with Spanish transparency legislation, because there were indications of errors in its configuration that prevented the correct application of the legislation.
The Foundation did not receive any response from the Spanish Administration. Faced with this administrative silence, an «aberrant practice» of maladministration, the Foundation, again using Spanish transparency legislation, went that same year to the Council for Transparency and Good Governance, an administrative body created specifically for the protection of the right of access to information.
The Council partially agreed with Civio in 2019 and obliged the Administration to grant access to the technical specification of said application, to the result of the tests carried out to verify that the BOSCO application complies with the functional specification and any other element that would allow to know how the application works.
However, it denied access to the source code, since the Council considered that the limit contained in Article 14.1 letter j) of the Spanish transparency legislation, relating to intellectual property, was applicable.
Civio then decided to go to court. That same year 2019, it filed a lawsuit requesting access to BOSCO’s source code. The aforementioned judgment of December 30, 2021 resolved said request, denying it and closing a first phase of the conflict, which is going to continue, since an appeal has already been filed before a second court.
What were Civio’s arguments and why did this judicial denial occur?
Javier de la Cueva explained that in the face of Civio’s allegations of violation of the principle of prohibition of arbitrariness (art. 9.3 of the Spanish Constitution) and the absence of legal protection of intellectual property in a program developed by the public administration itself, the judicial body was impervious.
Regarding the first, Civio argued that the impossibility of accessing the source code of the programs causes the law to become an automated application through a binary code that is impossible to read, thus removing not only the principle of legality, but all kinds of hermeneutic, argumentative and ethical possibilities that can be used only and exclusively in the results generated, but that cannot enter into any of the causes and, much less, in the system of rationality on which the decision making is based, which violates the right to know the motivation of the resolutions.
However, the court decision holds that by recognizing the right to the bono social the Spanish Administration (translations are ours):
«It adjusts its actions to the aforementioned regulations, issuing the corresponding administrative act. And for this purpose, it uses a computer application, called «BOSCO information system», which is inserted in a phase of the administrative procedure, the purpose of which is to verify compliance with the requirements previously established by the aforementioned regulations. This being so, it cannot be considered that the administrative act is dictated by a computer application, but by an administrative body, and in the event that the addressee of said act is not in agreement with it, he/she can challenge it through administrative and judicial channels. Therefore, the legality of the administrative act is not justified by the computer application that is instrumentally used in a phase of the corresponding administrative procedure, but by the regulations governing the matter».
According to the judgment, from the evidence at trial:
«The allegation of the appellant entity regarding the existence of failures or calculation errors in the application, particularly for certain groups, is therefore unfounded. And in the cases of «impossibility of calculation», due to the lack of sufficient information of the applicant in the databases connected to the application, the applicant is required to provide the documentation, and based on this, it is determined whether the right to the social bonus exists or not. These are cases in which the automatic calculation through the application is not possible, but not cases of denial of the social voucher«.
Therefore, the judgment makes a formalistic argument, reaching the conclusion that «Based on the foregoing, it must be considered that the denial of access to the source code of the computer application does not imply a violation of the principle of legality, since, ultimately, it will always be possible to verify whether the applicant meets the requirements for the social bonus to be granted«.
Regarding the second allegation that there is no violation of the limit of the Spanish transparency legislation that prevents access to information if there is an infringement of intellectual property, the ruling states that:
«The source code of the aforementioned computer application does not fall within the exclusions of intellectual property, mentioned in article 13 of Royal Legislative Decree 1/1996, of April 12, 1996, on intellectual property, a precept invoked by the appellant entity, since said code is neither a norm nor an administrative act».
The sentence insists that «it must therefore be considered that there is no rule that imposes on the Administration the development of open-source applications or the acquisition of free software«. Finally, the claim formulated by Civio to be granted access to part of the source code is rejected, as this is a new claim, not formulated in the lawsuit. In addition, the sentence adds considerations on the impossibility of granting access to the source code, since doing so would create problems of vulnerability of the program and affect public security (?).
Javier de la Cueva considered the context and content of the judgment, explained above, in the first part of his presentation. He pointed out that although jurists have been using algorithms for hundreds of years (e.g., later law repeals earlier law), the issue of algorithmic systems used in artificial intelligence raises epistemological problems related to judicial ignorance and the lack of attitude to integrate this necessary knowledge. He also underlined how the Spanish judicial decision departs from the solutions given in other European countries to similar cases.
In his opinion, access to the source code is essential to know if the program works correctly or if there are programming errors. For De la Cueva, the documents to which access has been given (remember, the technical specification of the application and the result of the tests carried out to check that the BOSCO application complies with the functional specification) are insufficient. He used a comparison: it would be a case similar to that in which a news item is written about a judicial sentence, commenting on it, without providing the sentence itself. It is essential to have the text of the sentence to be able to know if what the journalist says in the news item is correct or not.
Regarding the problem of intellectual property, Javier de la Cueva alluded to Spanish legislation on intellectual property and the fact that sentences, rules and regulations and the acts, agreements, deliberations and opinions of public bodies do not have intellectual property rights (art. 13). Moreover, there are works protected by copyright that the Administration has no problem in offering to the public, such as cartography or statistical tables, for example. The interpretation of the Intellectual Property Law must be done systematically: copyright only refers to private activity.
On the other hand, in De la Cueva’s opinion, the sentence commits a mistake in the interpretation of the facts, since, despite what it states, there is no official to make the decision because it is an automated decision.
Pending the decision of the court (Audiencia Nacional) that will review the sentence, De la Cueva explained how the Council for Transparency and Good Governance, which in this first litigation appeared alongside the State Administration, has decided, has now decided that it does not now oppose the appeal filed by Civio to the sentence «as it understands that said sentence is not in accordance with the law», according to a letter from the new legal representation of the Council. In fact, the Council, faced with a different claim filed by a citizen in 2021 on the access to the source code used in the computer application through which proceeded to the draw of courts associated with selective processes of an autonomous community grants it, denying that intellectual property prevents such access.
As De la Cueva pointed out, just by facilitating this change in the Council’s position, the litigation has already had positive consequences.
After Javier de la Cueva’s presentation, Professor Joost Joosten made a presentation on the same, highlighting some points. First, Professor Joosten emphasized that he agreed that there are many different types of algorithms and that in each case it is necessary to specify which of them we are dealing with. An important distinction for Joosten is the distinction between statistical algorithms (machine learning, etc.) and rule-based algorithms such as BOSCO, a distinction that does not prevent us from speaking of artificial intelligence in both cases, as stated, for example, in Annex I of the draft EU Regulation on artificial intelligence.
Professor Joosten pointed out that at the beginning of the case that has given rise to the judgment, the problem of possible program errors arose. This is a relevant issue, since it is possible and necessary to advance in the detection and avoidance of possible errors in algorithmic systems, in order to avoid problems such as those generated by BOSCO.
Professor Joosten stated that he does not believe that having access to the source code is the solution to problems such as the one that has arisen, since such code may be incomprehensible. He considered that in any case, access to the source code of the BOSCO program should be given, since it is a program developed by the Public Administration that should serve the citizens and with the use of public resources. He pointed out that progress should be made towards the use of a language that is understandable to non-technicians in relation to AI. He also mentioned the existence of an application that can check that what is stated in the technical specifications document (which for him is very important) coincides with what the program actually does in each case, although it is a verification system that is still in the process of being perfected.
Professor Joost concluded with a question about the ruling, from a non-lawyer’s perspective, related to what might be the background that may exist in such a ruling.
A lively round of interventions was then opened among those present in person and online. Among other issues, it was pointed out that there could indeed be a background to the judgment that is not strictly legal, but linked to sociological aspects, which could oscillate between the objections to showing a defective source code that could embarrass the Administration and the attitude of the judge, who, faced with a lack of knowledge of the subject matter, would have preferred to adopt the position of supporting the Administration. This support could arise from a certain understanding of the judicial function and the role of judges, some of whom consider themselves to be public employees rather than members of a branch of the State responsible for controlling the Administration and who, when in doubt, opt for the administrative arguments because they consider that the public interest is represented by the Administration by default.
However, as was emphasized in the debate, it is necessary to bear in mind that judicial deference cannot derive in judicial indifference to the rights of citizens; that the Public Administration should serve the general interest objectively, but it is the judicial responsibility to control that in each case this service really exists and that good administration has been respected; and that the presumption of validity enjoyed by the Spanish Administration´s decisions, like other European administrations, does not mean that there is a presumption iure et de iure of the legality of the administrative activity, but rather that, once the burden of appealing against the administrative decision has been exercised, it is the Administration that must justify that the decision adopted is in accordance with the Law and the appellant can destroy the presumption of legality.
On the other hand, the debate also raised the question of the principle of legality and the reservation of certain decisions to the law as a norm approved by democratically elected representatives. The decision to automate administrative acts, and how to do so, should not be given by law as a blank check to administrative offices. This blank remission of the law should not occur in cases where important citizens’ rights such as this one are affected. Furthermore, the Administration approved the algorithmic system without prior citizen participation or publicity of any kind, which is not acceptable, whether the algorithmic system is considered a source of law or not. In short, what BOSCO does is to codify in computer language a previous written legal regulation, which starts from a law of the Spanish parliament, which should have foreseen automation and established minimum guidelines, and goes through several regulations. What guarantees are there that the translation into computer language is correct and that the existing legal framework has not been altered?
Furthermore, in the case of the BOSCO program, the regulatory decisions that designed the administrative procedure for granting or denying the social bonus gave an apparently excessive protagonism to private companies in the sector (called reference marketers), which seems to lead to a privatization of the administrative procedural activity, by entrusting to them, with a simple regulation, the development of administrative functions such as checking, denying or approving and notifying. This would violate the impossibility of attributing administrative powers to private parties (contained in the Spanish legislation referred to in the Basic Statute of the Public Employee, art. 9.2) and thus privatize administrative procedures, which has been declared unlawful by the Spanish Supreme Court.
The session was closed with the intention of following the case with interest, paying attention to the second judgment that will be handed down on the BOSCO case, which we hope will be of better quality and much better argued than the first one, giving rise, finally, to the proper access to the source code.
The session can be viewed on video here. (AfL%6OBQ)
PROF. JULI PONCE SOLÉ
Full Professor of Administrative Law at Universitat de Barcelona
webpage