Sharing South Dakota's cultural heritage: harvesting digital collections into the Digital Public Library of America and beyond

Abstract

The Digital Public Library of America (DPLA) enables the discovery of digitized content held by U.S. cultural heritage institutions by aggregating metadata contributed from participating organizations. The DPLA differs from other resource sharing networks by providing not only the locality of an item from a catalogue such as WorldCat but offers easy access to the digitized item itself. Particularly for smaller libraries, archives, and museums, including content in the DPLA makes that content much easier for users to discover, access, and contextualize than it would be otherwise. The DPLA uses what they call the Hub Model made up of Service Hubs and Content Hubs to aggregate metadata from their partners and contribute it to DPLA. This allows state and regional collaborations to onboard small institutions, adding online texts, photographs, manuscript material, artwork and more.

Introduction

The DPLA seeks to connect people to the riches within America's cultural heritage institutions that are free and immediately accessible. To do so, DPLA aggregates over 35,000,000 metadata records of digital items found in libraries and museums from across the United States in one easy to find portal at dp.la. All items found in DPLA are free to view, and link back to the original provider's website, providing discoverability for small libraries’ rich collections alongside collections of well-known cultural heritage institutions such as the Getty, Smithsonian, the Library of Congress and many more. The DPLA states that their ‘aggregation provides access to more than 47 million images, documents, videos, and other cultural heritage artifacts from more than 5,000 libraries, archives, and museums across the United States’.¹

Hubs

It would be difficult and take too much time for the DPLA to harvest data from every US institution that has a digital library. The DPLA's vision is to strengthen and connect existing state or regional infrastructure and it does this with the Service Hub system which aggregates data from libraries, museums and archives.² Service Hubs serve as an on-ramp for institutions in a state or region to participate in the DPLA network. Service Hubs offer standardized services such as digitization, metadata help, data aggregation, as well as locally hosted community outreach programs that bring users in contact with digital content of local relevance.³

In addition to Service Hubs, Content Hubs are large digital libraries that work directly with the DPLA. Large digital content producers like the National Archives and Records Administration, Harvard, the Getty, the New York Public Library and the Smithsonian work with the DPLA one-on-one to identify and prepare their collections for aggregation by the DPLA.⁴

Metadata and aggregation

The DPLA is a metadata aggregator that brings together collections of metadata from organizations across the US and presents them in a single-entry point at its website (dp.la). In addition to a search interface, the DPLA makes its aggregated data available through various application programming interfaces (APIs).⁵ This metadata is provided with a Creative Commons CC0 designation, placing the metadata in the public domain. (Creative Commons CCO).⁶ Europeana – Europe's digital library – also releases its metadata into the public domain using CC0.⁷

DPLA's online collections use descriptive metadata enabling users to discover, locate and view the resources on the original provider's website. Subject metadata provides users with general entry points for all resource types that are often grouped into topical, form, chronological and geographic terms. In addition to formal controlled subjects – for example, Library of Congress Subject Headings (LCSH) or the Getty Art and Architecture Thesaurus – many organizations make use of uncontrolled keywords, tags or categories to create access points for their collections.⁸

Challenges in gathering metadata

What happens when metadata from different domains (e.g., galleries, libraries, archives, museums), created with different standards and schemas, are forced to interoperate semantically? There are significant interoperability issues that exist when gathering metadata at a national scale. The DPLA mitigates these issues through use of its Metadata Application Profile (MAP).⁹ In this case, the DPLA MAP is used as a lingua franca for DPLA metadata. The DPLA MAP is based on the European Data Model (EDM) used by the Europeana digital library.¹⁰

DPLA Content Hubs and Service Hubs face similar challenges in aggregating metadata. These include quality assurance, reconciliation of terms, and conforming source data to the DPLA application profile. An area receiving special attention is the clarification and mapping of rights statements. In some cases, there is no information in the record, and it needs to be supplied. In others, there may be notes with vague or irregular wording, and these need to be mapped to a controlled vocabulary in order to be useful in discovery systems (e.g., through faceting and filtering). Rightsstatements.org is helping to make this possible by providing unambiguous statements backed up by persistent URIs.¹¹

The metadata for all collaborators at a Service Hub must be aggregated and shared with DPLA through a single feed. This approach provides an on-ramp for smaller and underfunded institutions and ensures greater sustainability for the contributing institutions, the Hubs, and DPLA. An example of a DPLA Service Hub is the Minnesota Digital Library (MDL) which aggregates metadata for about 450,000 images, audio, video, newspapers, maps, documents and 3D works from almost 200 Minnesota institutions along with other institutions in the region and conforms them to the DPLA application profile. Service Hubs such as the MDL aggregate records as harvested through OAI, RESTful APIs, and data dumps.¹²

Some institutions the MDL works with might not be interested in having their content made available via MDL's Minnesota Reflections platform, but they do want the MDL's help getting their metadata into DPLA. This puts MDL in a dual role, acting as a central digital repository for many smaller organizations while still providing single-service aggregation for larger institutions that can provide repository functions for themselves.¹³

Onboarding

The Digital Library of South Dakota (DLSD) (explore.digitalsd.org) partnered with the MDL Service Hub to harvest metadata records from the DLSD's digital collections into the DPLA. Currently, smaller institutions are unable to directly share their collections with the DPLA, hence this partnership between the MDL and the DLSD. The process to harvest the DLSD's metadata into the DPLA required signing an MOU, creating and sharing OAI URLs, and adding rights statements and publishing them. This partnership in itself has dramatically increased the discovery and access to the DLSD digital collections, and once DLSD collections formally appeared in the DPLA in December of 2017, there was increased usage of DLSD digital content and special collections materials.

The Digital Library of South Dakota

The DLSD was formed in 2008 and is a collaborative effort of different organizations and universities in South Dakota to preserve its history and media collections. The DLSD digitizes documents, photographs, audio, and video formats for public access and research opportunities. Together, the collections within this digital library consortium offer unique resources, particularly in the areas of regional history and the lives and experiences of generations of South Dakotans.

Highlights of the University of South Dakota in the Digital Library of South Dakota

Through the DLSD and for purposes of teaching, learning, and research, the University Libraries at the University of South Dakota makes available and exhibits a wide range of scholarly, cultural, and historical resources that are within the collecting scopes of the Archives and Special Collections, the South Dakota Oral History Center (SDOHC), and the exhibit program in the library. Highlights include the Chilson Collection which is made up of books, journals, maps, pamphlets, and other print materials relating to local histories, South Dakota history, Native American cultures, and western expansion of the United States. Of interest is a handwritten manuscript by Zitkala-Ša, “Why I am a pagan”, which she wrote for the Atlantic in 1902 (Figure 1).

Fig. 1. Why I am a pagan, Zitkala-Ša. Original handwritten manuscript submitted to the Atlantic Monthly, 90 (1902): 801–803. University of South Dakota. University Libraries. Archives and Special Collections. DLSD: https://explore.digitalsd.org/digital/collection/chilson/id/19/rec/17. DPLA: https://dp.la/item/239bb6665bd60af0a51c46cdf660d1f4

The SDOHC collects and preserves the memories and experiences of the people of the Northern Plains from the 1890s to the present. The SDOHC collections are an especially vital and valuable record of the historical, social, and cultural legacy of the state. The American Indian Research Project and the South Dakota Oral History Project are the two primary collections under the SDOHC (explore.digitalsd.org/digital/collection/sdohc).

The Mabel K. Richardson Collection is made up of manuscript collections focusing primarily on South Dakota and the surrounding region. These papers include correspondence, photographs, journals, scrapbooks, maps, and other manuscript materials. The Richardson Collection covers a wide variety of subjects but concentrates on the people, places, and events in South Dakota's cultural, political, and economic history. An example is the Mamie Shields Pyle papers (explore.digitalsd.org/digital/collection/richardson/search/searchterm/Mamie%20Shields%20Pyle%20Papers/field/collec/mode/exact/conn/and). A pioneer leader of the women's suffrage movement in South Dakota, Mamie Shields Pyle became president of the State Equal Suffrage League in 1910, which became the South Dakota Universal Franchise League the following year. Pyle and her colleagues worked together so the women of South Dakota could claim victory in 1918, when state lawmakers and voters passed the equal suffrage amendment (Figure 2). Pyle also led the campaign for state ratification of the national suffrage amendment, which happened 4 December 1919.¹⁴

Fig. 2. Why should I vote for amendment E? “Woman suffrage propaganda posters”, 1918. Mamie Shields Pyle Papers, USD Richardson Collection. University of South Dakota, University Libraries, Archives and Special Collections. DLSD: https://explore.digitalsd.org/digital/collection/richardson/id/5056/rec/100. DPLA: https://dp.la/item/b3191e1e8f98e09065bb750d2d87b748.

Another digital collection is the juried and un-juried exhibitions hosted by the University Libraries Art and Exhibits Committee. Represented in this online collection are works from exhibitions such as the biennial Bound and Unbound: Altered Book Exhibition (Figure 3) available in the DLSD: explore.digitalsd.org/digital/collection/exhibitions/ and the DPLA: dp.la/search?q=altered+books.

Fig. 3. Wanderlust IV. Carole Kunstadt (New York), 2022. In “Bound and Unbound VII: Altered Book Exhibition”, juried by Chicago based artist Brian Dettmer. University of South Dakota, University Libraries, Archives and Special Collections. DLSD: https://explore.digitalsd.org/digital/collection/exhibitions/id/1604/rec/1. DPLA: https://dp.la/item/cffe3044a6697a8936068df19568e5b3

Future

Looking towards the future, further discussion could continue to investigate the potential of making DPLA metadata records interoperable with Europeana.¹⁵ In abiding by the EDM, a collaboration of the DPLA and Europeana could create a portal to a large portion of the Western world's digital cultural heritage artifacts and records. The newest version of the DPLA MAP has the potential to demonstrate these kinds of relationships by storing Universal Resource Identifiers from Linked Open Data sources, allowing for semantic Web interoperability.

Another current and future partnership is the DPLA's work with Wikimedia Commons. Over the last several years, DPLA has become the biggest institutional contributor to Wikimedia Commons. The Culture and Heritage team at the Wikimedia Foundation has been involved with Structured Data-related initiatives in order to engage heritage materials on Wikimedia projects.¹⁶ Their objective is to support and increase image usage across the projects, as well as to structure Wikimedia to help it reach communities globally. The DPLA has added 3.7 million images and is the main institution in the United States directly uploading files to the platform.¹⁷