Bringing researchers together with IIIF

Abstract

Based on the open-source IIIF image viewer Mirador 3, the Specialised Information Service Art, Photography, Design - arthistoricum.net, in cooperation with NFDI4Culture, the consortium for research data on material and immaterial cultural heritage, provides a platform for collaborative exploration of image resources. Within the virtual research environment, users can create their own workspaces and image collections individually or in groups, share them for collaborative efforts, and jointly create persistent annotations. This article outlines the development and integration process of this web-based research platform and highlights future areas of improvement.

Introduction

In the realm of digital research within art history and image-centric sciences, the significance of image annotations and collaborative work cannot be overstated. These processes play a crucial role in collecting, mapping, storing, and publishing information related to digital image resources. While existing technologies and standards facilitate the enrichment and global exchange of image-based data, the cultural heritage sector demands reusable and user-centric annotation tools that specifically cater to the needs of research groups. The IIIF API system, defined in 2011, has proven to be a promising, prospectively viable and, above all, sustainable basis of digital tools in the context of arthistoricum.net due to its level of standardisation, the numerous users and the large, international developer community. Since 2018, all image data integrated in the discovery search of arthistoricum.net are available in the IIIF format and receive standardised metadata via the manifests. Image resources can be opened and further processed in a customised version of the open-source, browser-based, multi-window viewer Mirador, which seamlessly integrates with the platform's dedicated image search. The viewer is being continuously enhanced by arthistoricum.net in close exchange with NFDI4Culture with the goal to provide a user-oriented and needs-based platform for collaborative work on shared image corpora. This concerted effort reflects a commitment of arthistoricum.net and NFDI4Culture to advance the capabilities of sustainable, open-source digital tools in the pursuit of enriched and synergistic research experiences within the field of art history. The overarching aim is to furnish the research community with a sophisticated digital research environment, which fits the needs of researchers at all stages of the research process.

A brief introduction to arthistoricum.net and NFDI4Culture

arthistoricum.net is the digital platform for research on European art and art history including photography and design.¹ It provides a cross-source discovery system curated according to subject-specific criteria which enables qualified research across the entire art historical spectrum, while facilitating participatory participatory opportunities for projects and networks. The service started as an integrated virtual library of art in January 2012 and is since operated in close partnership by the Saxon State and University Library Dresden and the Heidelberg University Library. Since 2014, the existing offerings have been expanded to create the Specialised Information Service Art, Photography, Design (Fachinformationsdienst Kunst, Fotografie, Design) as part of the Specialised Information Services programme funded by the German Research Foundation (DFG).² Its goal is to enhance the service function of both libraries with respect to the academic community, by making available research-relevant materials that extend beyond basic library resources. arthistoricum.net provides information infrastructure and a variety of digital services, tailor-made for a growing art historical community of universities, free research institutes, museums and collections, independent researchers, art academies, colleges and schools. The current portfolio of services and tasks ranges from needs-based acquisition and consolidation of digitally available resources, extension of the subject-specific information infrastructure, support and development of new services for open-access electronic publishing, digitisation and subject-cataloguing of resources relevant for art history to biographical services and authority control for research. The orientation and conception of arthistoricum.net are accompanied by a scientific advisory board. This advisory board, together with the two operating libraries, define the guidelines for operating and further developing the art information service and ensure continuous quality assurance.

Since October 2020, arthistoricum.net has been part of the National Research Data Infrastructure (NFDI) funded by the DFG.³ The aim of the NFDI is to systematically manage scientific and research data, provide long-term data storage, backup and accessibility, and network the data both nationally and internationally. NFDI comprises a coordinated network of 26 consortia, which are associations of various partner institutions from research and infrastructure that can be assigned to specific subfields of the science system, ranging from cultural studies and humanities to social sciences, engineering, life sciences and natural sciences. Within the consortium NFDI4Culture, which adresses the subjects areas art history, architecture, musicology, performing arts, film and media studies, arthistoricum.net acts as a close partner, supporting the sustainable development of needs-based research tools and data services for specialist communities, which are to be provided due to their high relevance for an extended circle of users.⁴

Image search

In addition to the main cross-media search of arthistoricum.net, historic and contemporary images, which have been curated according to subject-specific criteria, can be researched in a dedicated image search environment. It provides more than 3.2 million resources aggregated from a variety of image sources like the Deutsche Fotothek, heidICON, Foto Marburg, Bibliotheca Hertziana, Kunsthistorisches Institut Florenz (KHI), ETH Library's Image Archive and the Rijksmuseum. Moreover, since one of the task areas of the Specialised Information Services programme is the retro-digitisation and cataloguing of research-relevant sources such as historical literature, images of specific retro-digitised resources and collections are available for querying - besides full-texts and metadata search. In this regard, the online collection of illustrated magazines of the Weimar Republic serves as a significant example as it includes 10 of the most important German-language magazines of the period with roughly 650 issues and 75,000 printed pages.⁵ With an image corpus of more than 50,000 illustrations, which can be researched fully equipped with standardised metadata via the image search, the collection offers innovative access to historical publications as well as to the contexts of use and the journalistic reception of individual photographs. Furthermore, researchers can use the image search to explore the collection Digitale Fotoliteratur (digital photo literature), which contains nearly 20,000 printed images from more than 500 significant photobooks by famous German photographers like Karl Blossfeldt and Albert Renger-Patzsch.⁶

IIIF middleware

As part of our journey to a digital research environment, the arthistoricum.net team had to ensure that all images in our catalogue would be available as IIIF manifests. This proved to be no easy task at the time. Keep in mind that catalogue ofarthistoricum.net contains more than 3 million image resources related to the topic of art history from numerous different image sources located in Germany, Italy and the Netherlands.⁷ Each of these image archives, libraries and museums has its own organisational structures, agendas and budgetary constraints, which is why not all of them have the necessary infrastructure and resources to provide IIIF manifests for their digital objects. After taking all that into consideration, our team opted for a more comprehensive solution. Since 2019, arthistoricum.net offers IIIF middleware as software as a service (SaaS) for data providers contributing image resources to our catalogue. As part of our infrastructure, the middleware retrieves the IIIF manifest request and queries the catalogue Solr for the image resource to gather all required information from the image object metadata to create the manifest. In addition, the corresponding image will be fetched from the image source, written in a Network File System (NFS) Share and finally accessed by a Cantaloupe Image Server in order to deliver the image data (fig. 1). The advantages of this solution for our partners are obvious. Since the whole process of manifest generation is provided free of charge by arthistoricum.net, data suppliers require no additional infrastructure and can simply integrate the generated manifests into their existing system. This allows cultural institutions, especially smaller ones that would otherwise be unable to use the IIIF standard, to disseminate and share their collections as widely and cost-effectively as possible.

Fig. 1. Schematic representation of IIIF middleware

Selection process and Mirador integration

In addition to the implementation of the middleware, arthistoricum.net required a viewer capable of displaying all the IIIF image data from the catalogue while fulfilling the sophisticated needs of the art history community. Fortunately, there is a whole universe of IIIF-compliant viewers, which are easy to use and developed or maintained by different institutions or communities. The most common and popular viewers are the Universal Viewer, Mirador, OpenSeadragon and Tify.⁸ All of them bring different features and benefits to the table, thus allowing developers and institutions to choose from a wide variety of options. For our purposes, we needed a highly configurable and easy-to-integrate image viewer that could be expanded and developed to meet the requirements of researchers and research groups. After careful evaluation, our team decided in favour of the Mirador viewer software for integration into the IIIF environment of arthistoricum.net. Mirador is an open-source, web-based, multi-window image viewing platform based on HTML5 and JavaScript initially developed at Stanford University that has grown through a multi-institutional collaboration with Harvard University and various contributors from around the globe.⁹ Besides viewing and browsing digital image resources, Mirador allows for comparing objects from various IIIF-compliant repositories worldwide as well as continuous zooming in high-resolution images. Other functionalities relate to temporary image manipulation, including image rotation, change of brightness, contrast and saturation, inversion of colours and greyscale. Furthermore, the viewer can be extended with a large number of community-made plugins to add useful features and tools such as image annotation and sharing. While our first implementation was still based on version two, which has been around for a number of years and is still in production in many institutions, we quickly updated to the newer version 3. Released in 2020, Mirador 3 is still being actively modified and improved by the community. This makes is the ideal basis for our digital image research environment and our ongoing development efforts.

In order to fully integrate the Mirador viewer into the already existing web services of arthistoricum.net, in particular the dedicated image search space, the SMArT.Pictures toolset was created. With regard to the overall user experience when working with digital image files, the service facilitates the handling of IIIF manifests by bringing together useful functions at central locations of the arthistoricum.net image search. For example, when browsing specific query results in the gallery view, users can open manifests in the provided Mirador viewer, copy manifest links to the clipboard to open them in an arbitrary viewer or use the drag-and-drop option by simply hovering over a specific image and selecting the preferred action in the opening menu (fig. 2). All of these features are also available in the detailed image view via a sophisticated sidebar.

Fig. 2. Image search results gallery view with SMArT.Pictures toolset

Workspaces and annotations

Although vanilla Mirador is already a powerful tool for working with image corpora and creates a digital space for research, education and discourse in close alignment with the art community, various needs for improvements had been identified. Firstly, to meet researchers’ expectations in overall design and functionality, numerous detailed adjustments to the user interface were made. In this respect, one of the biggest modifications is the use of a wrapper application that adds a management layer to Mirador, thereby providing additional features for working with workspaces, user management and authentication as well as a help dialogue.¹⁰ Furthermore, to improve usability and to make the user interface even more intuitive, valuable functions such as switching between the elastic and mosaic workspace have been highlighted and placed in the menu bar. At the same time, the functionality of the resource list has been expanded and the option to delete existing list entries, which was previously not available in Mirador, has been implemented.¹¹ Besides the mentioned changes, our team integrated already existing plugins into the framework, such as the image tool plugin, which allows users to easily manipulate images.¹² With the reimplementation of the bookmark plugin from previous Mirador 2, which replaces the similar but less user-friendly function of Mirador 3, we have once again made it possible for researchers to create a snapshot of their current workspace and share their image collection with others simply by sending a link.

Secondly, in line with the goal to further enhance user interactions with IIIF image corpora at the image detail level, improvements to the already existing annotation tool were initiated. The annotation tool is one of the plugins that can be used to extend the basic functionalities of Mirador. Users can zoom in on a specific image section to the desired view and mark the image section to make annotations. In this way, text content can be precisely assigned to an image section. While the plugin offers basic annotation tools, it lacks an overall compelling UX-design, making it difficult to use and leaving users overwhelmed and frustrated. Therefore, we opted for a much more intuitive and user-friendly interface in the enhanced version. The core idea behind the optimised and altogether clearer and tidier interface design is a new workflow for creating annotations, which utilises step-by-step processing alongside the structural elements of the Web Annotation Data Model recommended by the W3C¹³. This interoperable standard was the basis for the further development of the annotation data model previously used in Mirador, which has been made more powerful and future-proof with regard to the requirements of a future publication workflow for annotations as micro-publications.¹⁴ Besides adding text content to image sections, users can now contribute associated tags or keywords to annotations and assign authorship of the annotation via a new author field. Within a workspace, annotations can be assigned to individual users, which facilitates communication when working together on image corpora.

While the community-built plugin stores annotations in local storage, persistent annotations require implementing an IIIF annotation server. Fortunately, the developer team of the Saxon State and University Library Dresden was able to create cross-institutional synergies by re-using the annotation server of the Heidelberg University Library, which had been created for the annotation module heiANNO, focusing primarily on annotating digital objects such as digitised publications and manuscripts.¹⁵ Since the Heidelberg University Library is one of the service operatros of arthistoricum.net, a close partnership had already been established, which facilitated communications with regard to requirements and further enhancements of the infrastructure.

Lastly, it was important to us to make the collaborative potential of the IIIF standard fully usable and re-usable for both individual researchers and research groups. The new protected workspaces offer a virtual working environment for collaborative use within Mirador to conduct art historical research directly on the image, to work on the same image material in parallel and exchange information about images in groups via image annotations.¹⁶ In this way, researchers can collaboratively create, edit and annotate image corpora from IIIF resources in a permanently available workspace and share research statuses with other members of a research group via a direct URL to the workspace (fig. 3). Annotations and resource lists are directly linked to the respective workspace and are therefore only visible to authenticated users who have a link to this workspace. Different processing statuses of a collaborative workspace are aligned by an integrated synchronisation function, which, in addition, provides feedback about the changes to the user. Furthermore, it is possible for researchers to set up several workspaces if required, to name them distinctively and to equip them with different resources and annotations. User authentication is provided by Shibboleth, a method for distributed authentication and authorisation for web applications and services that is widely used in cultural and educational institutions.¹⁷

Fig. 3. User interface of the customized Mirador 3 with advanced annotations-plugin

What's next?

The current release of the arthistoricum.net Mirador viewer in conjunction with the improvements presented, already offers a wide range of options for creating and exchanging annotations, thus further strengthening research processes when working with digital image corpora.¹⁸ However, there are a number of areas for improvement to increase efficiency and benefits for researchers, especially with regard to data publication and data availability. To address the needs of academic users at all stages of the research process, the following innovations are currently prioritised by our development team and should be rolled out by the end of 2024.

Workspace management: Enhance workspace management tools to improve organisation and collaboration among annotators. Implement a user interface for managing multiple workspaces and establish role-based access control to regulate user permissions within workspaces.

Annotation enrichment: Enrich annotations with licence assignments and DOIs to provide comprehensive metadata and facilitate attribution and discoverability.

Annotation as publication: Establish annotations as a form of publication, enabling their dissemination as micro-publications. Develop an editorial workflow for reviewing, approving, and publishing annotations to ensure the quality of annotations and streamline the process of registering and disseminating them.

Annotation searchability: Integrate search functionality for annotations into the online catalogue of arthistoricum.net, allowing users to discover and utilise published annotations effectively.

These steps affirm arthistoricum.net's commitment to providing a user-oriented and enriching digital working environment for researchers in art history and other image-centric sciences navigating the vast landscape of digital art history.