Transparency materialised: how registers can regulate access to documents?

Abstract

Proactive transparency in the form of electronic provision of documents is required by law in the EU. It has long been acknowledged in law and technology studies that digital technology can have legal consequences when implemented to perform a legal function. Consequently, the technological design of document registers has the ability to limit as well as enhance access to documents. When technology can have such regulatory powers, incorporating it into a legal function requires closer attention as to how or why it is so. This article will provide a close analysis of the European Commission’s main Register of Documents (RegDoc) to study the implications of technological design for access to documents. Transparency is approached through a procedural view, highlighting its mechanisms. The article uses a HCI based walkthrough method for the case-study artefact critique of the RegDoc. The main findings suggest that there are two specific affordances of the RegDoc that limit access, especially for users who do not have pre-existing knowledge of the documents they are searching for. These affordances are, first, the scope of the dataset and, second, searchability. Overall, designing technology for legally relevant functions should take into account the wider legal framework that the technology aims to cater for. Attention should be paid to the affordances that can make a legal difference in a technology created to perform a legally relevant task.

1. Introduction

In 2019, the European Ombudsman found maladministration in an access to documents case regarding text messages between European Commission President Ursula von der Leyen and the CEO of Pfizer on a Covid-19 vaccine deal. ¹ Evidence had suggested that such messages existed, ² but the Commission denied such claims. ³ The Commission had only consulted the internal archiving system on whether such messages had been recorded into the system, and because the archives did not provide the messages, the response to the access to documents request was that such documents did not exist. ⁴ The Commission was asked to look again and not to confine the search to documents that had been recorded. ⁵ It responded by stating that it would follow its established practice and the documents had not, to date, been found. ⁶ Since the Commission had only looked for documents that were recorded in the archives, its ability to find the documents was largely hindered by its internal archiving rules and the technology employed for the task.

Digital technology employed for or to assist in the performance of a legal function can have legal effects. The Pfizer case shows that even if the messages are documents in accordance with the applicable law, ⁷ the technology used to archive the documents played a major role in preventing them from being considered for access. Technology used for transparency has become part of the legal function of access to documents.

The principle of transparent governance is a long-standing ideal of democracy. Indeed, the public’s ability to obtain knowledge about the governing institutions is a necessary condition for accountability and legitimacy. While transparency in academic debates has many faces, the practices of transparency guide how the principle is manifested in the real world. The EU’s transparency efforts are centred around access to documents. ⁸ Direct access to documents through the institutions’ document registers play a central role in the ability to access documents and consequently in gaining information on the institutions. This is also something that transparency as a fundamental principle in the EU aims at. ⁹ However, when digital technology is integrated as a mechanism for transparency, it can limit the access as much as it can foster it. ¹⁰ This should not be confused with the transparency of the technology. Together with the technology used for transparency, the institutional working processes can govern transparency practices and legal rules alike.

Document registers in the EU have been built to ‘make it easier for citizens to exercise their rights’ ¹¹ and ‘make citizens’ rights [to access to documents] effective’. ¹² When the purpose of the registers is to foster transparency, it is not enough for the document registers to merely exist; they must be able to provide information through the provision of relevant documents. Therefore, the real-world effects of the technology must be examined to understand how the document registers are able to influence access to documents and consequently govern transparency efforts. ¹³ In other words, the purpose is to look at the grassroots level: how the technology used for transparency can shape access to documents both in terms of the legal right of the public to obtain information and the institutions’ obligations to provide it.

Compared to theoretical research on transparency, mechanisms of transparency have received little attention from scholars. While it has been acknowledged that transparency is not merely shedding light on something but actual performance in providing access to documents and consequently to information, the mechanisms of transparency require closer scrutiny. These mechanisms can include institutional practices, tools used for the task of provision of documents, and even the application forms for access to documents (and what institutions require to be included in them). This article is constrained to the tools, ie document registers.

While law and, more specifically in the EU context, the Transparency Regulation play a central role in guiding access to documents from the procedural perspective, it seems to have left the practical adaptation of direct access to documents via document registers mostly vague. Document registers are not thoroughly defined in the legislative text. ¹⁴ The law in essence merely requires the document registers to exist, consequently leaving the responsibility of designing and constructing them largely to internal guidelines and the creators of the registers. More specific references to what each document should contain in the register (such as subject matter) are listed in the law ¹⁵ but that is not always followed in practice. Institution-specific guidelines on recording and archiving documents have been enacted to close the gap between vague legal requirements and the everyday practice of the officials. ¹⁶ Sometimes the institutional practices gain rather strong standing in relation to technologically related questions regarding access to documents, as the Pfizer case shows.

Thus, even though the legislation creates obligations for EU institutions to provide public access to document registers, ¹⁷ the functioning of the register inevitably guides how proactive transparency is manifested in practice. This article will show how the digital tools used for transparency, the document registers, have regulatory powers. The article will concentrate on the European Commission’s Register of Documents (RegDoc) as a case study. I will show how the RegDoc can be considered to regulate access to documents in its own right through the way in which it has been constructed and maintained. ¹⁸ The broader theoretical claim is that the mechanisms used for access may have a great effect on the actual practice of transparency and thus affect the desired consequences. In other words, the digital tools used for transparency have regulatory powers. By showing this, I argue that once this regulatory ability is recognised, it should be considered, and the negative effects minimised when designing legally relevant technology such as document registers.

Since the aim is to unravel how the document register is able to influence access to documents, that necessarily requires a closer look at the technology itself. Consequently, computer science, and more specifically human-computer interaction (HCI) studies, will be used to understand the technology, while keeping the research focussed on the legal processes and effects. Similar bridging of the gap between law and technology has been conducted in socio-legal research of technology for some decades now, ¹⁹ establishing the theoretical grounds from which questions related to law and technology can be observed. Empirical, more nuanced studies are still in the minority. ²⁰ It should not come as a surprise, then, that computer science and HCI studies to date have moved to far more complex systems than the technology examined in this article, ie, search engines. However, multiple factors ‘taken for granted’ by computer scientists can be shown to be relevant when the technology is examined through a legal lens. For example, the assertations that technology is always socially situated and human decision-making is constantly involved even in the most extensively autonomised systems are recognised in HCI research ²¹ and remain normative assumptions that underlie this Article.

In the following, the Article is divided into five sections. In section two, I build on critical transparency studies that emphasise the mechanisms of transparency and place the document registers in context using a funnel metaphor. I continue to present the methodological premise of the following critique of the artefact – the RegDoc – by drawing from HCI based walkthrough approach. In other words, in section two, I place the case study of RegDoc in the context of transparency through the funnel metaphor and then present the method of the artefact critique.

In section three, I will conduct the walkthrough and reveal various aspects relating to the technology and its design that are able to limit access to documents in their own right. The analysis concentrating on the RegDoc as a case study is a necessity, since I dive deep into the technology of the specific document register. RegDoc was chosen due to it being the main document register of the Commission.

In the fourth section, I draw on HCI studies to discuss different user groups of the document register and link it to how the document register functions. I show from the technology’s perspective how it is designed caters to certain users, which can be considered problematic from the perspective of transparency. To be sure, aspects relating to the technology and the users are intertwined, and therefore the separation of the topics comes with limitations on the continuity of the argument. However, due to the complex nature of the technological aspects as well as the user’s interaction with the technology, the division was necessary for the sake of clarity. Before concluding, in section five, I show how limiting the usability of document registers can have problematic ripple effects on direct access to documents requests.

In the third and fourth section, user interviews are used to illustrate experts’ personal experiences with using the RegDoc. Also, institutional interviews are included to get a better understanding of the technology. The interviews were conducted in a semi-structured way and held in an online meeting service (Zoom), with the exception of one case in which a phone call was used instead. They were recorded with the consent of the interviewee. The interviews were transcribed and all who requested a copy of the transcript received it. The data are held and used in semi-anonymised fashion, including a broad category of affiliation and a title. The institutional interviews included one interview with the ombudsman’s office, an official who does not represent the opinion of the European Ombudsman, and two Commission officials. The user interviews followed a purposive sampling method where the users were chosen as experts who work closely with the Commissions’ documents regularly, or at least have done so. They included two journalists, one academic, and five personnel from several Non-Governmental Organisations (NGOs). Since the NGO personnel comprised more than half of the users interviewed, this creates a bias in the dataset. Due to the uneven representation and the fact that only eight expert users were interviewed, the sample cannot be considered as fully representative of experts who work with EU documents, and they should not be considered as primary source material for the argument. Overarching themes from the expert user interviews are included for illustrative purposes.

2. An approach to studying mechanisms of transparency

Transparency – a central concept for democratic systems – has been characterised in critical transparency studies as messy, ²² paradoxical, ²³ and dependent on the governing body’s hermeneutic convictions on transparency rhetorically as well as practically. ²⁴ In general, information disclosure is a necessary condition for transparency. However, mere information provision is not sufficient if the goal is to make the governing entity’s actions and decisions visible and comprehensible to the public. ²⁵ If transparency were considered reductively as merely shedding light on something, it would leave out the complex processes which occur around the transparency efforts and necessarily affect the levels of transparency. ²⁶ The aspects linked to procedural transparency are necessarily contingent on the level, type, and form of information provided.

Transparency as a concept simultaneously means both the administrative procedures and the mindset which underpin the actions for transparency. Transparency and information provision can be understood not only as conduct, but also as an attitude, including the commitment to operate openly and under the scrutiny of the public. ²⁷ Both the information as a starting point and the processes as the focus are essential features in the understanding of transparency, and allow for an examination of the issues from a broader perspective: the processes before, during and after providing information. The procedural and operational background of transparency may affect its consequences, sometimes in significant ways. Therefore, when evaluating transparency, the factors considered matter not only on the respective level of existence but also on the level of functioning.

Transparency laws are central in securing transparency, but we cannot stop there. Believing that what is written down is how things are leaves out the extensive interpretative element of law, as well as the discrepancies that lie between law-in-books and law-in-action. Transparency should be understood not only as talk but also as action. ²⁸ Clearly, there first needs to be a will to legislate on the subject, which then yields the relevant laws. ²⁹ But the laws do not produce outcomes corresponding to the demands if they are too vague, not implemented, or if they are somehow circumvented. The implementation of the laws as well as the processes and mechanisms surrounding transparency can be considered the actions of transparency, which are built upon the speech of transparency. The promises, normative calls, and, to a lesser extent, the law are nothing more than an idealistic utterance of the preferred state of affairs as long as we cannot see an effect in the real world. ³⁰ Therefore, we must also look beyond the black-letter rules governing transparency and see how access to information is operationalised in practice within the institutions.

Technologies equipped for transparency can produce legal effects as objects that mediate the realisation of the legally relevant matter (transparency). A key aspect is that some of these legal effects stem from the construction and maintenance of the technology. Thus, the document registers are not merely objects of the law, ³¹ but also contribute to making a legal meaning. ³² The idea is to understand the legal phenomenon of transparency as also including the role of materials such as document registers in shaping and mediating interactions relevant to the law. ³³ This role in shaping the legal relations that contribute to making a legal difference is what makes materials such as document registers specifically legal materials. ³⁴ They take part in the production of the legality of proactive transparency precisely based on the parameters they have been made to function within. ³⁵ As legal materials, ³⁶ document registers are part of the legal processes of transparency. In the next section, I will present a way to systematise transparency by using a funnel as a metaphor for the multiple levels that practically limit information provision in the process of proactive access to documents.

A. Funnel of opacity

As mentioned, the basic goal of transparency is to make the actions and decisions of the governing entity visible and comprehensible to the public. In practical terms, it would be impossible to expect accountability or legitimacy without both visibility and comprehensibility. To that end it is important to understand the context within which the transparency laws function and what other aspects may limit access. The funnel of opacity is created to do just that, concentrating on proactive transparency within the EU legal framework.

In the EU, access to documents is governed by the Transparency Regulation. ³⁷ The regulation provides a right to access to documents unless the Article 4 exceptions apply for the particular document. ³⁸ In other words, only Article 4 exceptions can legally limit access. However, that is not always the case in practice. Scholars have recognised limitations on access to documents beyond the Article 4 exceptions, such as unofficial institutional practices. ³⁹ I bring forward the argument that from the point of view of proactive transparency, the document registers can further hinder the ability to gain access to documents. What makes this matter significant is that technology is available to mitigate these limiting effects and the end product is always a choice (intentional or not). ⁴⁰

Figure 1 provides a visualisation of the transparency pipeline, concentrating on how the accessibility and understandability of the documents diminishes towards the bottom of the funnel. Although the Figure is a simplification, it aims to provide an image of proactive access to documents beyond the Transparency Regulation’s promises. All documents in box 1 are the documents mentioned by Article 2 of the Transparency Regulation: all documents that relate to policies, activities, and decisions within the institution’s responsibility, ⁴¹ to which therefore there is a right to access.

Figure 1. Proactive transparency pipeline – funnel of opacity.

The only exceptions to that right are codified in Article 4. Therefore, box 2 represents the smaller pool of documents which should be accessible when the documents that fall under the Article 4 exceptions are removed. Box 3 presents an even smaller number of documents that can be searched through the document registers. This is where the Article concentrates on. What happens in that box? How and why can (and do) the document registers further hinder the ability to gain access to documents?

To be sure, the funnel of opacity does not stop there. What further hinders the ability to gain access to documents, and especially to understand them is, for example, the language used. That is represented in box 4. Unofficial leaks do not pass through the pipeline but are generally leaked by a private party who has been involved in the issue. The unofficial release of documents has been a topic of a Court of Justice of the European Union (CJEU) decision, in which it was ruled that the fact that a document has been released unofficially does not equate to the official release, where the searcher or requester can confirm the officiality of the document. ⁴² The official means to gain access to documents is of great importance.

It becomes apparent that mere concentration on the laws and their promises is not enough, but we will need to focus on the processes and mechanisms ultimately being an essential part of the space in which transparency functions. ⁴³ When technology is used for pursuing something ‘legal’, such as access to information, the characteristics of the technology as a ‘mediation tool’ are exposed as visualised in Fig. 1. Technology has been equipped between the text of the law (right or obligation) and the outcome of the action done to pursue the right or obligation. ⁴⁴ How the mediation technology is built and presented inherently influences the ability to exercise the right or obligation that arises from the law. Thus, the document register presented in box 3 is the context of this article. In the next section, I draw from an HCI-based walkthrough method to execute the critique of the artefact (document register) in section 3.

B. The walkthrough method

Technology’s materiality simultaneously means that the technology generates affordances. ⁴⁵ In the most basic terms, the concept of affordances encapsulates the idea that a material artefact, such as a website, limits or allows certain kinds of behaviour. Affordances can be also understood as actions that are facilitated by the technology’s design. In legal materials, these affordances can make a legal difference. When the technology is created to perform a legal function, such as access to documents, the affordances of the document register require closer scrutiny. One way to do it is through a walkthrough method created by Light et al. ⁴⁶

The walkthrough method was originally created to study apps, but its flexibility and transformability to context-specificity allows its use as a guideline for studying document registers. ⁴⁷ In essence, the aspects to consider are the environment of expected use, and the technical walkthrough. ⁴⁸ The environment of expected use for the document registers stems from the law. It includes wider normative calls for transparency, linking it to democratic principles and the more specific legal requirements from the Transparency Legislation. Since the document registers are part of the administrative functions, the environment also includes institutional aspects such as manpower and budget.

The rest of the aspects are more technology-specific, and that is where I will concentrate in the next section. I will ‘walk through’ the Commissions’ Register of Documents and analyse the affordances from the perspective of how they can affect the right to/obligation of access to documents. For the purposes of analysing the Commissions main document register, I have found two separate technological aspects that are considered as having influential affordances. In other words, these two dimensions of the technology shape the ability to gain access to information and limit the capability to use the document register for such purposes. These two aspects are the scope and the searchability of the register. Furthermore, the user – a person aiming to gain access to documents through the document register – is a central actor in the access to documents landscape. Therefore, after walking through the technological aspects that limit access to documents, the user’s role is explored in section 4.

3. Walking through the Register of Commissions’ Documents (RegDoc)

A. The Commission’s registers and the RegDoc

The Commission currently maintains 11 sources of documents, most of which are document registers (Fig. 2). In theory, proactive publication of documents is a common practice in the Commission, limited only by the types of documents included in the archives and the trust that the documents will be registered by individual officers. ⁴⁹ Each register contains documents of a specific type relating to infringement proceedings, comitology, and historical information, for example. ⁵⁰ All 11 sources of documents are distinct when it comes to the user interface, search functions and in some cases, the archive. ⁵¹ Thus, the registers have been built according to several technological structures. ⁵² As the registers are independent, they rarely share information between them, ⁵³ meaning that a search in one register may show zero results when in reality the document is searchable from another register.

Figure 2. List of Commission’s document registers. Source: Commission, ‘How to access commission documents?’.

All 11 sources of documents are currently operational, but they are to be unified within a single register in the future, in accordance with the New Register of Commission Documents Reform. ⁵⁴ The aim of the project is to progressively update the registers’ underlying technology (some of which is aging or already outdated), ⁵⁵ link the document archives, and ultimately create a one-stop document register which would include documents from all the separate registers. ⁵⁶ The project was begun in 2018 and RegDoc was opened for the public on 17 May 2021 with a revamped visual design and searchability functions. ⁵⁷ The current phase of the project includes the integration of all the separate registers into one. ⁵⁸ Although this phase should have been concluded by 2023, ⁵⁹ at the time of writing, such integration has not yet happened.

B. Scope of the registers – RegDoc and only adopted documents

What can be searched for from a document register necessarily depends on what documents the register has in its database. At the same time, the decisions on what to include in the database shape what is perceived as ‘reality’ through the search results from the register. ⁶⁰ In technology studies, it has long been clear that technology, data, and their construction are not necessarily neutral. Considerations of importance, for example, shape the choices about what to include in the database. ⁶¹ In addition to the choices of the creators, internal guidelines on the requirements of relevant data influence what should be included. The decision-making on what is included and what is not affects the outputs from the register. ⁶² As Parmiggiani and Grisot have argued, value is co-created from the data through influencing what the databases contain. ⁶³ The underlying data creates new visibilities as well as invisibilities by choosing what the technology is able to show to the user. ⁶⁴ Choosing to leave something out decreases the direct value of the data through concealment and consequent issues to bring it to light. ⁶⁵ For example, documents related to unofficial decision-making can be left out of the database without publicly disclosing that this has been done. A document register cannot present anything that is not in the database.

The RegDoc is considered to be the ‘main register of Commissions documents’. ⁶⁶ The website states that it ‘includes various types of Commission documents’ followed by a list of examples. ⁶⁷ Thus, the list of different types of documents presented in the website is not exhaustive. Since the Commission upholds 11 different sources of documents, of which the register of documents is the ‘main document register’, the fact that it is not entirely clear what documents can be found in the RegDoc makes searching for relevant documents a challenging task.

From a technological perspective, the types of documents included in the RegDoc are more precisely limited, albeit this information is not available on the website. Technically, for a document to be recorded in RegDoc, it has to be ‘adopted’. ⁶⁸ This is because of the technological construction of the register: it retrieves the searchable documents from an internal archiving system called DECIDE. ⁶⁹ DECIDE only includes documents that have been adopted. Consequently, internally archived documents which have not been adopted, such as informal meetings, meetings without adopted summaries, text messages, or emails cannot be found in RegDoc. If these types of documents have been released sporadically, they may be searchable elsewhere, such as from the Press Corner. Crucially, if that is the case, they are not subsequently archived into any of the registers. ⁷⁰ There are also inconsistencies with the types of documents the RegDoc includes because confirmatory applications for document requests have not been included in the register even though they are official decisions of the Commission. ⁷¹ Overall, the categorisation of what is included happens due to the interaction of the archiving system DECIDE and the RegDoc. ⁷²

Even though examples of the types of documents included in the RegDoc are listed on the website, it becomes evident that it is not entirely clear which documents are included in the register and which are not. This was also mentioned in the user interviews. Some of the users interviewed highlighted that what is included in the register is uncertain to them and therefore there might be some difficulties in using the register. ⁷³ A journalist wished in their interview for more information on how the registering of documents happens, who registers them, and why. ⁷⁴ Some reported that even if they knew the exact document number, the search did not always produce relevant results. ⁷⁵ It could have been the case that the documents searched for in the latter example were documents that had not been included in RegDoc. An interviewee from a legal service in an NGO verbalised their concerns as, ‘the fundamental problem of what is actually being put on the public register’ ⁷⁶

The scope of the registers regarding what is included, and consequently, what can be retrieved from them, is limited by the design choices of the technology. Importantly, the Commission controls what they proactively make public and what not. The internal document registering guidelines drafted by the institution influence the ability to access documents of which there are no traces in the registers. For example, electronic communication, such as emails and instant messages, are not adopted documents, and hence are not included in RegDoc; it simply is not built to include such documents, nor is there a separate register for electronic communication. ⁷⁷ Furthermore, the internal archiving systems seem to function so that such communication is not usually available in the Commission’s internal registers and is sometimes not registered at all. ⁷⁸

If such documents are not registered, it is much harder for the institution to look for the documents when they are requested. Similarly, when electronic communication is not included in any of the registers, even as metadata, it is hard for the public to know what documents even exist in the first place. ⁷⁹ The archiving rules and the technology available limit the ability to review whether a document could be made public. The control over the guidelines as well as the practice of individual officials who register such documents in the internal databases highlights the ability to influence the information that is given out. It is through the technology that this control becomes particularly visible. For documents that are not registered in the database, unofficial leaks and insider knowledge become the means to gain access to the desired documents. ⁸⁰

This problem is not confined to electronic communication but applies to all documents that are not included in public document registers or internal archives. What is registered in the internal archiving systems can hinder the transparency of the institution. If documents are not recorded, they are not searchable from the internal archives, nor is it possible to make them searchable from a public register, even if the register included such document types. A researcher from an NGO verbalised this problem as follows.

‘I think they [document registers in general] are necessary precisely for the reason [. . .] that if you don’t know something exists, it’s very hard to look for it.’ ⁸¹

C. Searchability – from the interface to the metadata

Searchability is an essential aspect of the usability of document register. Searchability means the different ways in which documents can be searched for from the register as well as the ways in which the search can be limited. Limiting a search is an essential feature in order to prevent a search resulting in massive number of documents, which can be considered information dumping.

Limiting a search through technical means is intrinsically linked to the characteristics of a document as well as the digital information that accompanies it. The documents are published in the RegDoc in full, including the actual document and its metadata (name of the document, publisher, date, document number) or as metadata only. ⁸² In the latter case, a person interested in gaining access to a document could, based on the metadata, request access to it. The metadata of the documents are a key feature for searchability of the documents.

When searching for documents from the register, the searcher inputs a search word into the search bar (Fig. 3). How the system retrieves the information and presents it is dependent on the search terms, what information is included in the documents or accompanies it, and the construction of the register. ⁸³ An equivalent function to rank documents can be made in several ways, and the solution reached is ultimately a decision. There are ways in which information can be concealed (consciously or not) by adopting an algorithm that obfuscates the results derived from the database, as there are also ways to create it in a way that makes the results appear logically. For example, if the document is accompanied with metadata, ⁸⁴ the system can be built to ‘prefer’ documents whose metadata corresponds to the search word and present those documents at the beginning of the list of all documents. At the same time, the search word inserted into the search bar can be analysed by the system in several ways that are decided by the user, such as the whole word, the whole sentence, or just part of the word or sentence. ⁸⁵ A document register generally is made to function according to the latter if the search word does not correspond to any metadata.

Figure 3. User interface of the RegDoc search section. Source: Commission, ‘Register of Commissions Documents’.

The first part of the RegDoc reform was completed in March 2021 and included changes in the searchability functions of the register. ⁸⁶ The functions now include the option to conduct a word search and limit it to the title and/or the content of the document (Fig. 3). ⁸⁷ The RegDoc also allows the user to limit the search terms to one word at least, all words, or exact phrase (Fig. 3). ⁸⁸ Thus, a searcher can limit their search on the RegDoc website, which enhances their ability to search for relevant documents.

The metadata information linked to each document, namely the document name, the publisher, the date, and the document number ⁸⁹ are the datapoints based on which the document can easily be searched for in the ‘keywords’ section, as explained above. Of these four identifiers, the name and document number are specific to a given document. Knowing the specific metadata accompanied with the document, the four identifiers, increases the probability of finding the relevant document.

Thus, the metadata that accompanies the document is an essential aspect that fosters the ability of a searcher to find the relevant document. The Publications Office of the European Union has noticed this importance in relation to EUR-Lex. Metadata’s relevance in improving the overall visibility of relevant content of searches in EUR-Lex was discussed in the publications office’s management plan from 2021. ⁹⁰ Importantly, RegDoc does not include the subject matter or the key words of the document as metadata. This is problematic, given that including subject matter is an obligation from the Transparency Regulation, but it was left out from the more specific guidelines. ⁹¹ Lacking subject matter in the metadata is one major factor that limits the meaningful use of RegDoc.

A Commission official stated that due to the lack of manpower and budget and the vast number of documents that are included in the RegDoc, it is impossible for them to include the subject in the metadata. ⁹² As a result, the search is based only on the words that are mentioned in the document or in its title. Without the subject in the metadata, especially with the occasional counterintuitive naming of documents, ⁹³ the ability to search for relevant documents is reduced. On the other hand, if a searcher knows the document number or other specificities of the document they are searching for, RegDoc is better equipped to provide the relevant document. This shows that a high level of knowledge is often needed to be able to use the register successfully.

Examination of the search options in RegDoc and the consequences of leaving out the subject matter metadata shows that while the ability to limit the search decreases unwanted search results, a necessary condition is again that the user has at least some knowledge of what they are searching for. Lacking subject matter metadata means that knowing other metadata information, such as the document number, is more important in using the register successfully. This type of information is highly expert knowledge that the public cannot reasonably be expected to know. The high requirement of pre-existing knowledge needed, not only on the institution and its competences but also the specifics of the documents that are considered to include information that the searcher wants, was brought up in the user interviews. A researcher and a project leader from separate NGO’s explain their views as follows:

ʼThere is one main problem, which is that the entry level of knowledge you need to have to approach this register is really quite high, in order to find something that is useful to you. So, for instance, if you only look for certain words or if you do a word search, for example, I think that in my experience at least, what you will come up with is a trillion documents that [are] quite difficult to navigate. So, you might be able to narrow down the search if you look for, for example, a legislative file, if you have a code or [something] more specific or you know the branch of the Commission that was most involved or so on.ʼ ⁹⁴

ʼI’m in a work environment [with] a lot of academics and of course they did sometimes know those complicated numbers. But just imagine someone who’s not used to searching for stuff [. . .] like knowing these numbers, those titles [. . .] I think it must be horrible.ʼ ⁹⁵

4. Users – what is supported and what is expected

Two main issues arise in relation to the user. Firstly, the target user is always embedded into the design (intentionally or not) of the technology. ⁹⁶ This is the case in all technologies that are built to have an interactive function. Secondly, some abilities are always presumed from the user. ⁹⁷ This is in a way a continuation of the fact that the user is always embedded into the design. Next, I will expand on these points from the access to documents perspective, concentrating on RegDoc.

In designing a digital system that a user interacts with, the attributes assumed of the user of the technology are configured into the system. As there are different types of people, there are necessarily different types of interactions with the technology. ⁹⁸ Most of the document registers are concerned with document representation and ranking, but the users should also be considered in order to cater to their needs meaningfully. ⁹⁹ In creating a document register, the fundamental question to answer is: what abilities are presumed of the users? The answer to this question guides the design of the document registers.

The users of document registers are a varied group of other than institutional actors who wish to gain access to a document, and consequently to knowledge. The users’ representation in the digital sphere can be differentiated from the physical people if we emphasise the type of search conducted from the document register. In other words, the emphasis is on the types of searches that can be conducted, irrespective of the user being a member of the public or from a specific field, with different motives for gaining knowledge. Therefore, the analysis below builds on ways in which documents can be searched for from the document register rather than specific groups of people. Thus, it is not a presentation of ‘a user’, but rather of search styles.

There are two main types of searches that are conducted in search engines, including document registers. ¹⁰⁰ Firstly, a ‘lookup search’ refers to a search in which the users have basic knowledge of the subject and know clearly what they are looking for. For instance, they have specific knowledge about the document’s title or reference number. Conducting this type of search is generally unproblematic, since most document registers are built to find documents based on specific parameters. In relation to RegDoc, knowing the specific metadata accompanied with the document, the four identifiers, increases the probability of finding the relevant document. This type of search commonly brings better search results since the user is able to confine their search to certain specificities.

Secondly, an ‘exploratory search’ occurs when the user does not necessarily know in detail what they are searching for and therefore the search itself is broader. Consequently, the user may use key words of broad applicability, which often results in vast numbers of results from the document register. Being able to limit the scope of the search helps to minimise the number of documents presented in the results of such a search, which is incremental for users who do not have specific knowledge on the documents they are searching for. ¹⁰¹

In RegDoc, a large number of results can be expected from general exploratory searches, as the same word may be present in multiple documents. This is so especially since the subject matter (the main topics of the document) is not included in the metadata of the documents in RegDoc. ¹⁰² A lack of subject matter in the metadata can result in a large number of search results as the register may bring up documents that merely include the search words even though the document might not be about the subject. Comparatively, subject matter has been included in the metadata of the documents in EUR-Lex, which results in the documents being easier to find. In the case of EUR-Lex, insertion of subject matter has been done manually, one document at a time. ¹⁰³

Typical practice is to use a single type of document register for both kinds of searches. ¹⁰⁴ However, the different types of searches would either benefit from different types of document registers optimised for the distinct search styles, or from a document register that is adaptable to each style. ¹⁰⁵ The technology is available for both, but the lookup type document register is more commonly used. In those cases, when a searcher lacks specific knowledge on what they are looking for, the likelihood of finding a relevant document decreases. One reason is the way in which the technology is built to accommodate users who know specificities better.

The fact that RegDoc supports look-up search inherently better than exploratory search means that it has been built (again, consciously or not) to cater better to the needs of users who have greater knowledge of the documents that they are searching for. One major self-standing reason for that is the lack of subject matter metadata. Thus, there are ways to improve the ability to conduct an exploratory search, such as the inclusion of subject matter metadata, which the Transparency Regulation already demands.

Furthermore, since the ability to exercise one’s right to access to documents is limited, how does that fit in with the legal principle of transparency? The legal right to access documents, which is, according to the Transparency Regulation, only limited by legally defined exceptions, should not rely on the abilities presumed of users. My right to access to documents should not, in theory, be dependent on my personal abilities. I have the right, irrespective of my abilities. Furthermore, the Transparency Regulation is intended for citizen access, as shown by several provisions in the regulation. ¹⁰⁶ That being said, it is undoubtedly the case that the abilities of the users are a central issue which determines the ‘success’ ¹⁰⁷ of using a document register. This can be mitigated through the design of the technology, however. In the case of RegDoc, as I have shown, its design choices have limiting effects on the right to access documents. Only through a closer look at the technology employed for the task will these limitations come to the fore.

To be sure, it is not only the technology that may hinder the ability to gain access to documents, or the information contained therein. On top of digital literacy, and accessibility for special needs, it is also a question of language and understandability, as I presented with the funnel of opacity in section 2. All of these aspects are linked to the abilities of the users. I will provide two examples concentrating on language, because they become visible from the technology ¹⁰⁸ : firstly, the naming of the documents and secondly, the language used within the documents.

When documents are named intuitively, they are usually easier to find. This is the case for the RegDoc especially since the subject matter of the document is not included in the metadata and the main ways to confine the search as examined in section 3C. Naming the documents intuitively does not always happen and some interviewees reported that it has hindered their ability to search for the documents. ¹⁰⁹ The naming of the documents is linked to the institution’s internal working language and derivable from its complex nature.

Once the document is found, an aspect that may limit transparency is the peculiar institutional language or jargon used in the working environment and therefore also in the documents. For a layperson, it could be difficult to know what different document names or institutionally specific words mean.

These two examples on the language used in the documents show how the practice of transparency cannot be limited to what the law or policy documents say about transparency. The institutional language used in the documents is not directly related to the technology, but still serves as a good example of the many ways in which transparency is materialised.

5. Consequences to access to documents requests

While the document registers often act as a first gateway in searching for documents that the institution holds, ¹¹⁰ it is also possible to access documents through direct access to documents requests which could result in being provided with access to the official document. However, access to document requests have own limitations. ¹¹¹ Furthermore, the incompleteness of RegDoc has a further impact on the knowledge required to provide direct access to documents requests.

If a search in the registers does not provide any results, the next step is usually to send a direct access to documents request. In relation to exploratory searches, which are not really catered for by the document registers, a general access to documents request would be directed to ‘all documents related to X’ due to the applicant not knowing the exact documents containing the information required. ¹¹² ‘All documents related to X’ types of access requests are time-consuming and laborious for the institutions, ¹¹³ but the direct access to documents request means that an official search for the relevant documents. Comparatively, the register is only able to search for documents according to what the premade algorithms allow. As the EU institutions are legally obliged to make an attempt to identify the relevant documents, they are expected to undertake a more thorough search than the document register’s algorithm can produce. They may be better equipped to conduct these broader searches than the registers, because it is possible to ask the originating institution for the document if the institution has not registered the document in the archives. However, as the Pfizer case shows, gaining access to non-recorded documents is still problematic. ¹¹⁴

Using the registers to find what documents even exist in the first place is fundamentally linked to direct access to documents requests. This is apparent from the CJEU’s finding of ‘presumption of lawfulness’, a legal principle that was found to apply to access to documents requests. ¹¹⁵ In practice, ‘presumption of lawfulness’ means that when an institution claims that they do not have the document(s) requested, the burden of proof shifts to the applicant to present ‘sufficient and consistent evidence’ of the likely existence of the said document. ¹¹⁶ Applicants are arguably at a disadvantage in these situations due to the difficulty in proving the existence of the desired unpublished document. This directly ties in with the proper archiving of documents and maintenance of the document registers for the applicants to show the existence of an unpublished document. Cases in which the Commission first claims that the document does not exist only to come back later saying they have found the document happen sporadically, ¹¹⁷ which highlights the problematics of the presumption of lawfulness.

Shifting the burden of proof to the applicant to show the existence of a document exemplifies the importance of proper achieving guidelines and data management, as well as the consequences for the legal rights and obligations related to access to documents. In extreme cases, issues with archiving coupled with the presumption of lawfulness might result in situations in which a document exists, but due to it not being included in the archives, it is almost impossible for the applicant to prove its existence. Unofficial means, such as leaks of documents, would allow the applicant to demonstrate the existence of a document, ¹¹⁸ but the transparency of a governing institution should not rely on unofficial leaks. In such cases, the knowledge of the governing institutions would be confined to a specific circuit of people who know who to ask about leaks. Still, unofficial leaks, through a private party who has been involved in the issue, are a common practice for gaining access to a document. ¹¹⁹ The exclusionary nature of transparency is once again highlighted, begging the question of whom the institution’s transparency is directed to in practice.

6. Conclusion

Proactive transparency in the EU’s governance constellation is regulated not only by the legal frames from the Transparency Regulation but also by the mediation technology – document registers – and the users’ abilities. A walkthrough of RegDoc has revealed two main affordances of the technology that affect access to documents via the register. Firstly, the scope of the register is not only ambivalent but also largely limited through the adopted digital design. Secondly, the searchability of the register is meaningfully limited by the lack of subject matter metadata even though it is a requirement imposed by the Transparency Regulation. I have shown that access to documents is largely dependent on the pre-existing knowledge of the searcher due to the technology accommodating such search style (lookup search).

Implicitly or explicitly, technology is built for a certain audience, so the audience of practical transparency is carved into the technology. In other words, technology is always designed (consciously or not) for a certain use. This means that the design encapsulates specific types of users to accommodate. It remains a question how this deeper logic of the technological design can be reconciled with the legal principle, which in practice should be equal for all (rule of law).

Some of the still persistent practical issues of the document registers were already mentioned in the Commission Green Paper (2007), which voiced the need for attention to, for example, the scope of the register, number of documents included, and user-friendliness. ¹²⁰ The Commission stated that it had started to look for ways to improve these issues, ¹²¹ a statement that remains questionable 15 years on and considering the issues still experienced with the registers today. The ongoing reform has the potential to improve proactive access to the Commissions’ documents. For example, allowing users to search for other than ‘adopted’ documents from RegDoc by connecting the databases of all the Commission document registers (as the reform entails) would widen the scope for results. However, if parallel improvements such as including subject matter metadata accompanying the documents are not undertaken, then it is hard to see how the mere augmentation of documents would impact the practical ability to gain knowledge. One major self-standing improvement which would benefit all search styles (and users) of RegDoc would be to include the subject matter in the metadata of the documents, as became apparent in section 3. Large costs can be expected if it is done manually, but the technological advancements of the 21^st century allows it to be done using cheaper, computational means as well. ¹²²

The technological solutions materialising transparency are not related only to RegDoc. It seems that in some cases narratives on technology are used to postpone or circumvent political promises. For example, a Joint Legislative Portal of the Commission, Council, and the European Parliament has been discussed since 2016 along the same political agenda on ‘better regulation’. ¹²³ The Joint Legislative Portal aspires to be ‘user-friendly’ and to allow ‘non-specialists’ to gain knowledge about ongoing legislative procedures easily. ¹²⁴ While the creation of such a portal has been on the political agenda frequently for almost ten years now, it does not exist yet. It seems that the archiving procedures and systems are so distinctive that creating a cross-institutional register is easier said than done. At the same time, the technology for the creation of such a portal exists, which begs the question of where the political will for the creation of (the promised) portal has stumbled. Furthermore, for example, the European Ombudsman’s inter-institutional access to documents as well as related document-registering practices have been part of a court case, ¹²⁵ which indicates that the Ombudsman’s document registering practices would also benefit from closer scrutiny. Overall, the EU institutions and bodies would benefit from, first, looking at the technology employed for the purposes of access to documents through its capacity to contribute to making a legally significant difference, and second, by emphasising these affordances when designing a system for accessing documents.

For transparency to be closer to the effects desired, such as accountability or increased democratic legitimacy, we should begin to emphasise the tools’ functionality and usability. As I have shown, RegDoc, through its functioning and the consequent limitations from the technological decisions, contributes to making a legal difference to the Commission’s transparency. This is where greater emphasis is needed, not only on the laws and existence, but practices and functionality.