Synthetic Heritage: Online platforms, deceptive genealogy and the ethics of algorithmically generated memory

Abstract

Services offered by genealogy companies are increasingly underpinned by computational remediation and algorithmic power. Users are encouraged to employ a variety of mobile web and app plug-ins to create progressively more sophisticated forms of synthetic media featuring their (often deceased) ancestors. As the promotion of deepfake and voice-synthesizing technologies intensifies within genealogical contexts – aggrandised as mechanisms for ‘bringing people back to life’ – we argue it is crucial that we critically examine these processes and the socio-technical infrastructures that underpin them, as well as their mnemonic impacts. In this article, we present a study of two AI-enabled services released by the genealogy company MyHeritage: Deep Nostalgia (launched 2020), and DeepStory (2022). We carry out a close critical reading of these services and the outputs they produce which we understand as examples of ‘remediated memory’ (Kidd and Nieto McAvoy 2023) shaped by corporate interests. We examine the distribution of agency where the promotion by these platforms of unique and personalised experiences comes into tension with the propensity of algorithms to homogenise. The analysis intersects with nascent ethical debates about the exploitative and extractive qualities machine learning. Our research unpacks the social and (techno-)material implications of these technologies, demonstrating an enduring individual and collective need to connect with our past(s), and to test and extend our memories and recollections through increasingly intense and proximate new media formats.

Introduction

Services offered by genealogy companies are increasingly underpinned by computational remediation and algorithmic power; users are encouraged to employ a variety of mobile web and app plug-ins to create progressively more sophisticated forms of synthetic media featuring their (often deceased) ancestors. Machine learning processes are inserted into the very fabric of personal archival assets such as photographs, ‘remediating’ (Bolter and Grusin 1999) memories through recursive and mechanised algorithmic logics (Kidd and Nieto McAvoy 2023). As the promotion of deepfake and voice-synthesizing technologies intensifies within genealogical contexts – aggrandised as mechanisms for ‘bringing people back to life’ – it is crucial that we critically examine these processes and the socio-technical infrastructures that underpin them, as well as their mnemonic impacts.

In this article, we introduce and interrogate these developments considering the following research questions: [1] What ethical issues do these technologies bring into focus?; [2] In what ways might the promise of synthetic ancestry support the creation of new (parallel, posthumous and technologically connected) memories? and [3] How do these algorithmically generated memories connect with the long social history of genealogy? For the purpose of this issue, we explore these questions in relation to genealogy platforms’ corporate interests, as well as, given those platforms’ reliance on social sharing, in relation to social networks’ politics of attention. Our study thus bridges scholarly debates in Critical Algorithm Studies, Genealogy Studies and (Digital) Memory Studies, unpacking the social and mnemonic implications of these technologies, and demonstrating our evident need to test and extend familial connections through increasingly intense and proximate new media formats. To do so we present a close critical reading of two AI-enabled services released by the genealogy company MyHeritage via their mobile app and web platform: Deep Nostalgia (launched 2021) (Figure 1), and DeepStory (2022) (Figure 2).

Figure 1. MyHeritage Deep Nostalgia.

Figure 2. MyHeritage DeepStory.

In this article, we understand the MyHeritage platform as a form of ‘mnemotechnology’ which showcases quite remarkably how memory emerges or is situated at the interstices of the social and (techno-)material world (Prey and Smit 2018, following Bernard Stiegler). In such a context, as we will demonstrate, memory is increasingly encoded within what Massimo Airoldi has called a ‘machine habitus’ (2022: 28); where machine learning systems ‘as socialised agents’ ‘recursively’ interact with users with a host of consequences, not least ethical ones. These include, for example, raising questions about trust and the reliability of records (Ajder et al. 2019; Rini 2019), provenance (Chesney and Citron 2019; Kidd and Rees 2021), regulation (Hepp et al. 2022) and what happens to the data that underpin these activities (Vaccari and Chadwick 2020). We pay attention to such debates in this article but are mindful too of positive accounts of synthetic media's impacts; for example, with regard to their artistic and educative possibilities, potentials for audience engagement and for the (hyper-)personalisation of experience. Simone Natale captures the ambivalence of these approaches well when he notes that ‘the true impact of AI is more nuanced and oblique than usually acknowledged’ (2021: 2).

According to Jerome de Groot, online genealogy platforms remain under-researched. This is important to address given that, as he notes, genealogy ‘was part of the early Internet and is fundamentally embedded within its current manifestation’ (de Groot 2015: 114). It would be a mistake, according to de Groot, to understand ‘tech information companies’ such as Ancestry.com ‘simply’ as genealogy companies however (de Groot 2020: 25). As we demonstrate in this article, it is becoming imperative that we interrogate their broader ‘socio-technical’ impacts (Willever-Farr and Forte 2014: 475); impacts on the very definition and construction of history and of family for example, or on how – and what – we remember. As a myriad of machine learning approaches continue to be introduced by online genealogical companies, these impacts will become more urgent, but perhaps also more difficult, to discern.

Our analysis in this article draws on studies of digital memory which emphasise the manifold ways mediated memories now have a generative function in our everyday lives; ‘creating and re-creating a sense of past, present, and future of ourselves in relation to others’ (van Dijck 2007: 21).¹ Mediated memories, according to Worcman and Garde-Hansen (2016) ‘stretch’ the very idea of the past, such that it becomes ‘continually emergent’, not least within the socio-technical parameters of platforms (Esteve Del Valle and Smit 2021: 1816). Of particular relevance for this discussion are works studying interconnections between algorithms, automation and memory (Kidd and Nieto McAvoy 2023; Cheney-Lippold 2017; Lambert et al. 2018; Sisto 2020, 2021) as well as – given encouragements to share synthetic media creations via social media – on the impacts of memory metricization or quantification within those contexts (Jacobsen and Beer 2021a). These processes can be said to ‘intervene’ in the ways the past is packaged and recalled (Lee 2020: 1). A number of studies have explored the ways platforms sort and label our past (Jacobsen and Beer 2021b), but within genealogy platforms these processes and interactions extend posthumously also, impacting the ways we are constructed, remembered and datafied after we die (Cheney-Lippold 2017; Harrington 2020; Sisto 2020, 2021). This has profound ethical implications, as we explore in this study.

‘Relations of descent’ (Moore et al. 2020), or genealogies, are a key mechanism for organising our world and helping us understand our place within it (see also Knauft 2017). As Laqueur notes, genealogy ‘is a form of communion with those we take to be ancestors’ (2023) such that we might imagine and articulate bonds with them, even after death. Genealogy offers narratives that shape us, but that are evidently not static. Instead, they are open for ‘renegotiation and reknowing’, providing opportunities to ‘grapple with past and present, self and other, personal and collective memory, and individual and group identity’ (Barclay and Koefoed 2021). Online genealogy platforms clearly provide a rich context for such grappling. Since the 1970s there has been an upsurge of public interest in and exploration of how relations of descent are ‘strung together’ (Kretsedemas 2017). This has often been in tension with professional accounts of history, not least because in asking ‘Who am I?’ genealogists segue into inquiries about ‘Who are we?’ which are profoundly political (Morgan 2021). This begins to suggest the myriad ethical considerations that work with family memory produces, especially when it relies on ephemeral traces, deals in ‘secrets and silences’ and (maybe distinctly in the context of online platforms) is negotiated, recast and legitimised ‘from below’ (Barclay and Koefoed 2021). It follows that genealogy has been celebrated for its democratising potentials (de Groot 2015; Evans 2011) where it challenges the nation-focused and state-driven patriarchal histories that tend to constitute academic scholarship (Evans 2020). In online genealogy platforms, the collaborative and socialised potential of family history is perhaps most vividly anticipated, especially where it is practised ‘self-consciously and critically’ (Evans 2020).

Accuracy and authenticity are key concerns for genealogists then, and family historians are expected to work to guidelines and agreed standards of proof (Moore et al. 2020). The surging popularity of online genealogy however (Friday 2014) has raised concerns within the family history research community about the adequacy of users’ skills in archival intelligence and artifactual literacy (Willever-Farr and Forte 2014). In a study by Willever-Farr and Forte (2014: 481), the presence of ‘‘clickologists’ and ‘half researchers’’ was lamented by family history researchers; those who they perceived as careless, or who prioritised increasing the size of a family tree at the expense of the accuracy of their work. They also noted the ‘insidious role’ of automation in the mix, suggesting that inexperienced researchers mindlessly accept automated suggestions from platforms without verification, ‘leading to family trees full of erroneous information or ‘genealojunk.’’ (Willever-Farr and Forte 2014: 481). Such concerns about accuracy and authenticity are intriguing in light of this article's concern with what have been termed ‘synthetic’ media – audiovisual content often generated or modified by AI (PAI 2023) – and the creation of new (techno-) material and mnemonic possibilities which flow from those media.

Finally, we consider MyHeritage as a site for reanimating the dead as the platform encourages and facilitates synthetic forms of ‘algorithmic afterlife’ (Lambert et al. 2018). A number of studies have explored ‘digital immortality’ within the context of social networks (Bassett 2022; Harrington 2020; Kania-Lundholm 2019; Kasket 2019; Öhman and Watson 2019; Savin-Baden et al. 2017; Sisto 2020, Stokes 2015), and there is formative work on death-tech more generally (Arnold et al. 2017; Biçer and Yıldırım 2022) and AI and death more specifically (Meese et al. 2015; Savin-Baden 2021). Within this framework of what has been termed thanatechnology (Sofka et al. 2012), cyberthanatechnology (Beaunoyer and Guitton 2021) and necro-technology (Nansen et al. 2023), our focus here is on the ethical entanglements of reviving the dead online within the context of genealogy platforms and exploring what algorithmic revivification and remediation might entail for psychosocial mnemonic practices.

In this article, with the above theoretical concerns in mind, we first turn to an overview of the MyHeritage platform, paying particular attention to the Deep Nostalgia and DeepStory applications which are the focus of this study (section ‘MyHeritage, generative AI and ethical considerations’). We reflect upon the promotion of user-generated synthetic media as a marketing strategy aimed at engaging a wider public, and interrogate the ethics of these practices. The discussion is based on a systematic critical reading of language used by the MyHeritage platform and the DI-D studio that created the underlying technology used in Deep Nostalgia and DeepStory. This has included a discourse analysis of the websites and their Terms and Conditions in relation to data use as well as their Privacy Policies (D-ID 2021a, 2023a, 2023b; MyHeritage 2022a, 2022b, 2023a, 2023b, 2023c). The discussion explores how these activities intersect with broader debates about exploitation, extractivism, bias and manipulation. Section ‘Algorithmically generated memories and (deceptive) genealogies’ responds to the second and third of our research questions, turning to some of these concerns in more detail and unpacking the algorithmic practices of ‘remediated memory’ which, we contend, are sedimented into Deep Nostalgia and DeepStory. We focus here on the homogenising tendencies of the algorithm, both aesthetically and temporally, and highlight tensions between the search for authenticity that underpins genealogy and the creative and manifold deceptions brought about by generative AI. The discussion here is based on a visual reading of the resulting short videos, including those shared by users on Twitter (hereafter X) and Facebook.² We conlude by offering some closing remarks and suggesting avenues for future research.

MyHeritage, generative AI and ethical considerations

MyHeritage is one of several crowd-sourced genealogy platforms providing users with the ability to research their family histories. Platforms like MyHeritage, FamilySearch, Ancestry and Findmypast are underpinned by billions of historical records and other data points that allow users to create family trees, find records of their ancestors and identify new and previously unknown relatives, past and present. The sharing of information and experiences is a fundamental component of these data platforms, both because of the connective nature of genealogy but also their dependency on the data submitted by users to expand their databases. MyHeritage currently has 88 million users, 5.4 billion profiles (including those of ancestors, deceased and alive) and 78 million family trees (MyHeritage 2023a).

Most online genealogy platforms use automated processes for a range of tasks, but primarily to match data across users and profiles – what Ancestry calls ‘hints’ and MyHeritage ‘discoveries’. MyHeritage's Record Matches, for example, finds relevant historical records for users’ family trees, while Smart Matches compares information across family trees to match people with unknown ancestors. DNA Matching allows users to find relatives based on their shared genetic sequences and to contact them to further collaborate on researching a possible shared family history. Users are responsible for verifying the source of every piece of information in order to accept the match and add it to their family tree. In addition, MyHeritage also uses a number of AI-enabled synthetic media tools to augment and animate users’ photographs of dead ancestors. These are our focus in this article, specifically the Deep Nostalgia and DeepStory applications which we introduce in this section.

It is worth noting from the off that the language used by the MyHeritage platform, including in the orbit of these tools, is explicitly mnemonic: MyHeritage's reenactment technology is ‘intended for nostalgic use, that is, to bring beloved ancestors back to life and hear their stories’, taking users ‘on a meaningful discovery journey that unites [their] past, present, and future’ (MyHeritage 2023c). While family history is often conceived as a way of uncovering, verifying or correcting family memories, its practice is considered as memory-work whereby affective connection to the past helps genealogists establish ‘links between the historical, the personal, and political’ to understand the present and for future and younger generations also (Evans 2020). Likewise, Deep Nostalgia and DeepStory can be read as creations about the past, but also about the present and the future, a paradox symptomatic of the ambivalence of these technologies and the nostalgia they elicit; a ‘complex emotional experience’ (Routledge 2016: 44). A crucial part of the mnemonic language used by platforms – the mobilisation of nostalgia – is of course not neutral, becoming more profitable as it becomes more visceral (Routledge 2016).

MyHeritage's Deep Nostalgia feature was released on 25th February 2021 and, according to the company, ‘is intended to bring beloved ancestors back to life’ – an ambition we revisit later in the discussion (MyHeritage 2023b). It uses deep learning technology to animate faces from historical photos, creating a short video of the result which can be easily and freely shared in social networks. The result is an animated photo where the subject can be seen smiling, blinking and turning their head. Upon its launch the Deep Nostalgia tool quickly went viral, and by 10th March 2021, 33 million people had used the feature to animate images. This number has since more than tripled, with MyHeritage now claiming (December 2023) that 111 million animations have been created (Talya 2022).

DeepStory was launched in March 2022 as an upgrade to Deep Nostalgia, taking the concept of animating photographs even further by adding audio (Esther 2022). MyHeritage described the new feature as ‘a video biography’ where ‘the narrative is told by a speaking portrait of your ancestor or relative, based on details from your family tree and manually entered text … enriched by photographs that illustrate various life events’ (MyHeritage 2023c).³ DeepStory's ‘video reenactment technology’ combines two AI and deep learning tools (MyHeritage 2023c). Live Portrait – which also underpins Deep Nostalgia – creates a high-resolution video animating the person's portrait (D-ID 2023b), whilst Speaking Portrait transforms the narrative into high-quality audio using text-to-speech technology, the face and mouth moving by simulating lip-sync to ‘speak’ the audio (D-ID 2023c). Both technologies are licensed by MyHeritage from D-ID, a company specialising in generative AI technology, including the production of synthetic media outputs. D-ID advertises this technology primarily for promotional purposes (D-ID 2023a).

An intriguing affordance of these platforms and their ‘vernacular’ (Gibbs et al. 2015) which demands further exploration is their sharing and connective logic. Users of both these features – as well as more recent ones like Time Machine (MyHeritage 2023d) – are actively encouraged by MyHeritage to share their creations via social media, specifically Facebook and X. For example:

Once your DeepStory is ready, you'll surely want to share it with your friends and family. Their reactions will be priceless! Plus, they'll get the opportunity to create their own DeepStories for all of you to enjoy. The DeepStory is a video that you can download and share in any way you'd like, or share it directly to Facebook or Twitter. You can also copy a link to the DeepStory that you've created and share it via other means (MyHeritage 2023c).

We can therefore understand these tools’ significance first and foremost through their marketing function for the company; their outputs destined within this context to be shared, ideally to attract fresh audiences to the MyHeritage platform. In order to make their own creation, a new user would then have to give MyHeritage a name and email address at the very least (MyHeritage 2022a, 2022b, 2023b, 2023c). As such, each creation becomes a ‘transaction in a data network’ (Dewdney, 2022: 24), both within an exchange relationship between a user and the company, but beyond that to other MyHeritage users and social networks.

Garde-Hansen et al. (2009) suggest that the ability and encouragement to ‘share’ has become a powerful factor in how we think about memory and its uses. Esposito agrees, arguing that image production increasingly happens for the purpose of sharing rather than storage, and as such, its aims shift from ‘temporal preservation’ to ‘social multiplication’ (Esposito 2022: 83). In the case of Deep Nostalgia especially, the frenetic social media sharing which occurred around its launch demonstrated that shift extraordinarily well, with what might have started as ‘intimate’ remembering (Hjorth and Hinton 2019: 172) transferring quite comprehensively – and in the case of DeepStory also quite audibly – into the public domain. This is significant because it requires that memories and remembering are shaped to fit the socio-technical parameters of social platforms (Esteve del Valle and Smit 2021), and it demonstrates how – through what might be termed ‘posthumous performances’ – the dead continue to be made productive, ‘generat[ing] capital’ for these companies ‘in collaboration with the living’ (Stanyek and Piekut 2010: 14).

Social media sites are of course implicated in the success of online genealogy platforms more generally. Reliant on data extraction and accumulation, the business model of MyHeritage and other genealogy companies is optimised for operation within the orbit of social networks and must increasingly consider and conform to those networks’ logics in their design as we noted above. Within online genealogy communities, dissemination of findings is widespread (Evans 2011), but connections to social networks extend these possibilities further. As de Groot notes (2015: 117); ‘The purpose of the network is the growth of the network. It tends towards openness and through that, revelation.’ The practices explored in this article further encourage the unhindered sharing and movement of information (data), and the promotion of tools to those who might not otherwise consider themselves family historians. This too is significant where it opens up new audiences for these practices, perhaps in time challenging perceptions of those interested in family history as ‘misty-eyed and syrupy’, as well as conservative in their outlook (Evans 2011).

Sharing and connecting becomes not just an ‘ethos’ then, but crucial to the company's business model in order to accumulate data, and attract new (potentially paying) users. The ethics of ‘socio-economic systems heavily dependent on the massive extraction and predictive analysis of data’ (Airoldi 2022) needs to be considered here, particularly in its exploitation of our individual and collective pasts for commercial benefits. Reliant on crowdsourcing, accessibility is an important feature of online genealogy and most platforms offer a first tier of services that are free at point of use, although you do first need to register for a user account (MyHeritage 2023a). For example, performing a search might be free but to see most records you need a paid subscription. Deep Nostalgia and DeepStory are both on a freemium plan (with a limited number of free uses before having to buy a subscription), but it is worth considering what happens to users’ data after the event (i.e. What happens to the data they have volunteered as ‘payment’). According to the MyHeritage Terms and Conditions (2022b), uploaded photos belong to users and not the platform and can be deleted at any time. The resulting enhanced photo in Deep Nostalgia or video in DeepStory also belongs to the user, and all photos are stored and manipulated on the MyHeritage servers and never shared with third parties (MyHeritage 2022a). According to the Terms and Conditions (2022b), none of the AI-based features used by MyHeritage collect or retain any biometric information or biometric identifiers from photos that are uploaded.

There are, however, caveats to these claims that raise ethical concerns about data extractivism. Several features use third-party technologies, which complicate MyHeritage's data footprint. For example, the Photo Tagger feature uses a facial recognition technology model (i.e. biometric information) that is powered by Amazon to help users tag people in their photos quickly and easily. The disclaimer notes that this feature is not enabled by default and requires the user's explicit consent (MyHeritage 2022a). DeepStory uses text-to-speech conversion which is another service provided by Amazon Polly and Microsoft Azure. MyHeritage claims that the text is deleted once converted and the audio stored by MyHeritage rather than Amazon or Microsoft, but the ramifications of this data processing (if any) are not clear to the user (MyHeritage 2022a). Similarly, MyHeritage's business model is predicated on sharing on social networks, mainly X and Facebook, which also have their own sets of Terms and Conditions. Both companies claim that the content posted on their platforms belongs to the user who retains copyright; however, in order for Facebook to provide their services, for example, they claim to need ‘a non-exclusive, transferable, sub-licensable, royalty-free, and worldwide licence to host, use, distribute, modify, run, copy, publicly perform or display, translate, and create derivative works of your content’, which is automatically accepted by the user every time content that is covered by intellectual property rights is shared, posted or uploaded (Meta 2023).

Copying and using content and data that might belong to others without their permission is at the heart of concerns raised by generative AI and synthetic media more generally. D-ID, the company that developed Deep Nostalgia and DeepStory, claims on their website that they will ‘work hard to ensure’ that their customers (in this case MyHeritage) use their technology ‘in ethical, responsible ways’ (D-ID 2021a). Alongside tech and media companies such as TikTok, OpenAI, BBC R&D and Witness, D-ID belongs to the Partnership on AI, which has recently published ‘Responsible Practices for Synthetic Media’, a framework for developing, creating and sharing synthetic media that avoids online harms such as spreading misinformation, committing fraud or bullying and harassment (PAI 2023).⁴ These are indeed actual concerns arising from the widespread use of deepfakes in the media that jump from harms to the self (such as faking memories as a form of gaslighting) to the social implications of ‘epistemic contention (such as doubt in scientific experts, fake news, deep disagreement, ‘post-truth’)’ (Rini and Cohen 2022). But they also raise more specific concerns for users engaging in the practice of creating synthetic media, such as unknowingly contributing labour or data. On whether the technology uses uploaded images to train algorithms, MyHeritage specifically says that this is not the case when uploading photos to AI Time Machine, but does not specify for any of the other AI-based technology licensed by them (MyHeritage 2023d). Even if not using the images for training the AI, the underlying technology (Time Machine e.g. uses text-to-image technology licensed from Astria and based on Stable Diffusion) has most likely been trained on data collected from others without their consent, which, in some cases, might contravene GDPR or copyright law (Vincent 2023). Prompted by the recent popularity of generative AI, the European Union Parliament made amendments to their AI Act whereby companies designing and using generative AI will have to disclose summaries of the copyrighted material used to train their models (a task near impossible), presenting a future challenge to some of the technology discussed in this article (EU 2023). Moreover, there are also questions to be asked about the circulation of synthetic media such as the digital reenactments resulting from Deep Nostalgia and DeepStory that might be collected and used to train other algorithmic models in turn (Jacobsen 2023).

Regardless of how aware of its data practices MyHeritage users are (and despite agreeing to the Terms and Conditions), the ethics of data transactions and legacies on the platform become knottier once one considers that most images animated by Deep Nostalgia or DeepStory are of users’ deceased relatives. This raises the question of what ethical considerations should inform work with digital human remains, and whether a deceased person can be considered to have consented to a (remediated) algorithmic afterlife, created posthumously on their behalf. This is heightened by the fact that this new ‘immortal’ identity on the internet will not only be managed by others but persist indefinitely; their reenactments and stories having an impact on the collective family history and, because of their inscription into an extensive and often public database, potentially on a worldwide kinship graph (Willever-Farr and Forte 2014). The manipulation of grief in this context can elicit negative emotional reactions of family and other members of the mourning community that also lay claim to the deceased relative (Willever-Farr and Forte 2014), and perhaps even the photograph. Building on Shoshana Zuboff's call to protect our ‘elemental epistemic rights’ from ‘capitalist surveillance’, Tackett (2021) wonders if this should extend to the dead also: ‘do we need a necroethics?’. Cyberthanathology is fraught with theoretical and practical ethical issues (Beaunoyer and Guitton 2021); at the very least, some argue, digital remains also deserve to be treated with dignity and have their moral rights recognised (Lambert et al. 2018), just as physical corpses do (Öhman and Floridi 2017).

These remediated memories are consistent both ‘with our own experience of a digital existence that is dissociated from our biological one’ and with ‘our habit of delegating our stories and memories to artificial agents’ (Sisto 2020: 65). However, these are hybrid practices that sit on a continuum of materiality (Galani and Kidd 2020) that also needs to be acknowledged and unpacked further. For all MyHeritage's emphasis on digitally reviving the dead, it is important to remember that, like other genealogy online platforms, MyHeritage is currently a biotech company that processes and accumulates genetic information also with many ethical ramifications. And while genetics is framed here (and possibly understood by users) as historicised, communicating ‘ethnic identity rather than health information’ (Stallard and de Groot 2020), at least in the US there are already premium health applications that can be added to the DNA testing service within the context of MyHeritage. Regardless, this addition to the practice of genealogy promises to reveal users’ ‘unique ethnic background’ by discovering ‘the specific groups’ they descend from (Ethnicity Estimates) and matching them with newfound relatives (DNA Matches) (MyHeritage 2023a). Again, users are reminded that all information collected and processed by MyHeritage is private and will not be shared with third parties – whether law enforcement, health insurers or big pharma (MyHeritage 2022a). These claims need to be qualified, as there are circumstances in which the data can be accessed; for example, if there is a warrant to do so or by duplicitous means – data breaches and the case of the Golden Gate Killer have been ethical concerns MyHeritage has had to confront in the past (St. John 2020). It has been argued that by sharing DNA with genealogy sites, users – or their relatives – might become ‘genetic informants’, with still unknown consequences (Ford 2018). Furthermore, MyHeritage uses aggregated and anonymised genetic data for research – with the user's prior and explicit consent as an opt in (MyHeritage 2022a). In creating these huge biobanks and databases, corporations have replaced the academy as gatekeepers and controllers of access to this knowledge, as well as conferring users a ‘dual status as customers and research participants’ that is fraught with ethical issues in terms of informed consent at the very least (de Groot 2020). The many ethical ramifications of using DNA to biologise ‘ethnicity’ have also been highlighted, whereby these processes of value extraction and the capitalization on affective attachments are characterised as biopolitical techniques of surveying and surveilling (Cachoian-Schanz and Schwerzmann 2021). Understanding the materiality of the practice of genealogy is crucial as the externalisation of memory on servers in the cloud ‘involves a commodity chain of extracted and mobilised material resources’ with environmental implications also (Reading and Notley 2015).

As de Groot (2020: 9) has argued, these companies present themselves as organisations enabling and empowering ordinary people. Users feel that they can ‘conquer time and space to get our dead loved ones back’ (Tackett 2021), but they also become accountable for using the technology appropriately. The responsibility for abiding by the principles and values of family history, including provenance, credibility and authenticity, is ultimately placed on the user (see also Scharlach et al. 2023). First, MyHeritage asks its users to only upload their ‘own historical photos’ and never photos featuring living people without their permission, or that of their parents or guardians if minors (MyHeritage 2022a). Second, MyHeritage also outsources to the user the responsibility to be respectful and truthful, asking users to never upload content (including in the narrative to DeepStory) that is obscene, false or offensive (MyHeritage 2023c).⁵ These responsibilities are somewhat opaque however, given the insidiousness of the virality that has surrounded these outputs (notably, for Deep Nostalgia). Once creations have been shared via Facebook or X, questions about permissions, reproduction and respectability have their parameters changed. The socio-technical infrastructures of the internet intermediaries, and the logics underpinning specific platforms – their unique ‘vernacular’ (Gibbs et al. 2015) – influence in profound ways the patterns and practices of their use, and these extensions are worthy of much closer scrutiny.

We have considered here the agency of the programmers and developers of these platforms and AI systems, acknowledging, as Natale points out, that users have agency also. The democratising claims of genealogy companies and the distribution of agency can be seen in the way users are not just asked to verify what an algorithm identifies as a potential link to a family member or historical record, but to engage in specific practices of (re)creation of mnemonic digital objects and the remediation of memories. Deep Nostalgia and DeepStory heighten users’ affective connection to their pasts, potentially reinstalling some sense of control in a context of grief (Tackett 2021). This agency is however shared and ultimately outsourced – while the decision to animate a photograph (and which photograph) rests with the user, the final looped video is directed by the algorithm and, more crucially, by the programmers and the facial movements of other datafied bodies as we will go on to analyse. The homogenising tendencies of the algorithm, however, render the creations with an uncanniness that highlights tensions between the search for authenticity that underpins genealogy, and the creative deceptions brought about by generative AI. In the next section, we explore this in more detail.

Algorithmically generated memories and (deceptive) genealogies

In the context of its use by MyHeritage, DI-D notes that the ‘technology offers today's generation a tangible link to their ancestors and proves that artificial intelligence can be leveraged to deepen one's connection to their family history’ (D-ID 2021b). In this section, we interrogate what kinds of connections these are, exploring how algorithmic remediation functions as a link to the past.

Once an old photograph has been submitted by a user, both Deep Nostalgia and DeepStory first enhance it (increasing resolution and sharpening focus), before mapping it onto blueprint (‘driver’) videos sourced from recordings of real humans, mostly employees at MyHeritage (Esther 2021). The algorithm matches each photo to a compatible driver, for example, one that has the same or similar head orientation (Esther 2021). We characterise the resultant creations as ‘remediations’ (Bolter and Grusin 1999) given that the videos produced – with or without voiceover – refashion other media including the photo and the blueprint video. In this context, remediation might be done out of curiosity, but is also clearly done for mnemonic and nostalgic purposes, often with deeply affective consequences (Kidd and Nieto McAvoy 2023).

It has been recognised that remediation has significant implications for memory work (van Dijck 2007), not least where it unsettles or recasts our individual or collective connections to the past. In previous research into mnemonic responses to Deep Nostalgia creations, we have observed this happening (Kidd and Nieto McAvoy 2023); in many cases, remediation seemed to shore up extant mnemonic attachments for users, but it was also able to support the creation of new posthumous memories, of a deceased person's smile for example. We thus saw MyHeritage's algorithmic remediations underpinning the creation of new memories and attachments, as well as (re-)scaffolding existing ones such that they became differently experienced. Both the algorithm and the logics of social media offer a distinct form of remediation that highlights the ambivalent, networked and iterative co-construction of memories. Following Bolter and Grusin (1999: 272), remediation here renders the object both transparent (making the user believe they are in the presence of the represented object – in this case their deceased relative) as well as hypermediated (where the emphasis is on the medium), not least because the inner workings of the technology are explained on the MyHeritage website (Esther 2021). References to remediation among users of Deep Nostalgia (e.g. the movement of bodies, or the authenticity of the resulting animations) are fraught with ambivalence, oscillating between understanding the video as reviving the person in the image and acknowledging that the remediation is not wholly convincing or comforting (Kidd and Nieto McAvoy 2023).

MyHeritage's promotion of these tools as a way to establish a unique and personalised mnemonic experience – ‘Discover and treasure your unique family history’ (MyHeritage 2023a) – seems to be a possibility then, but a profoundly unsettling one given the algorithm's propensity to homogenise during the remediation process; abstracting, blending and building from pre-scripted prompts, and producing a ‘synthetic’ output as a result. MyHeritage notes for example that ‘Depending on the video and the angle, the technology sometimes needs to simulate parts that do not appear in the original photo, such as teeth or ears’ (MyHeritage 2023b). As we have noted elsewhere (Kidd and Nieto McAvoy 2023), algorithmic remediation works best at a degree of abstraction, and tends to lead to uniformity and conformity in how an image is ‘brought to life’, as programmers tend to exclude more atypical or ‘chaotic’ data entries to ensure more predictable and persuasive outputs (Markham 2020: 10). We found a level of homogeneity in the looped videos (the tilting of the head, the movement of the eyes and a smile) that contrasts with the claimed uniqueness of the remediation. Once animated, the motion a user sees (the blinking, the smiling) is evidently not the photo subject's own, but rather, has been directed by a programme(r) on the basis of another body's (or other bodies’) gestures. While new media have historically been used to revive our loved ones – for example, Victorian photo books of the dead or the séances inspired by the phonograph (Walter 2015) – these algorithmic remediations signal a qualitative shift towards automated and co-created digital mashups that reinforce and destabilise psychological and social practices of mourning, as well as carrying ethical and philosophical implications. Through the revivification process the datafied body in the image is connected intimately with other datafied bodies; the user, other users of the platform, and the person(s) in the driver video which underpins its animation. On the release of Deep Nostalgia, El-Hadi (2021) referred to the videos it produced as ‘digital frankensteins’ which captures this fusion – as well as its ensuant uncanniness – well.⁶ In DeepStory creations the addition of a customised text-to-speech audio track intensifies this sense of bricolage, and of uncanniness. The options given to create the video narrative are however also limited in terms of voice, gender and storyline, for example, even when offering a wide range of languages.⁷ Their homogeneity ultimately highlights the fact that the mechanisms are coded in ways that are culturally situated (historically, institutionally and geopolitically), as well as narrowly defined (see also Kopelman and Frosh 2023). Despite the data mashups and the potential uniqueness of the outputs, in both processes, remediation leaves little room for what Quadflieg et al. (2022) term ‘disobedience’ in the face of algorithmic power, resulting in a high degree of conformity in outputs.

Given their illusory qualities, and current limitations in the remediation process, it is tempting to conclude that these outputs represent simply a new kind of ‘genealojunk’ (Willever-Farr and Forte 2014). Instead, however, we prefer to understand them as examples of what we call ‘deceptive genealogy’, echoing Simone Natale's recognition of deception as increasingly ‘banal’ (Natale 2021: 128), especially in the context of artificial intelligence where trickery has become domesticated, subtler and more pervasive. Natale proposes that rather than assessing only whether deception is occurring, we should instead seek to understand what specific outcomes and implications flow from those deceptive effects; what feelings, thoughts, beguilements or diversions, for example, do they produce? It could be argued that the deception of these digital reenactments is not different from the speculative nature of genealogists’ interpretations of their ancestor's personality or other attributes. Storytelling is at the heart of family history; relatedness and kinship not ‘a biological given but a social construct’ whereby genealogies ‘are the narratives we construct to actually make them our ancestors’ (Zerubavel 2011). By selectively remembering and forgetting, users ‘accommodate personal as well as collective strategic agendas of inclusion and exclusion’ (Zerubavel 2011). Not only genealogy, but digital immortality is designed ‘from the perspective of those who remain, not of those who have died’ (Sisto 2020, 64). In relation to Deep Nostalgia and DeepStory specifically, accepting that the representations they produce are similarly specious, we should ask why – or in what ways – their deceptions really matter within genealogy contexts and for individual and collective memory more broadly.

Synthetic media can raise psychosocial and philosophical questions about the provenance of memories which become difficult to resolve, particularly when considering the mix of sources and agents at play. Yet the idea that memory might be open to manipulation and falsity is not new. ‘False memories’ can be said to occur when ‘mental images of a fictional event are mistakenly recognised as the product of lived experience’ (Murphy and Flynn 2021: 480–481) and are particularly likely where a mental experience has the qualities of a real memory; rich details, imagery (including moving imagery) and sensory information such as sound (Nightingale and Wade 2022). Deepfakes – and, we contend, outputs such as Deep Nostalgia and DeepStory – have the potential then to be particularly rich stimuli for false memories, ‘engaging neural networks and controlling mental, perceptual, emotional, and cognitive states’ (Liv and Greenbaum 2020).⁸ In response to such stimuli, researchers suggest that our brains tend to ‘auto-fill’ false details, and to do so with high levels of confidence (Liv and Greenbaum 2020). ‘Digital trickery’ coupled with our imaginations, can lead to ‘remarkably high levels of belief and memory distortion’ according to Nash et al. (2009: 421), to the extent that ‘false evidence can, in effect, change the past’. Such implications would be intriguing, and of course troubling, and more research will be needed as algorithmic procedures are used to more comprehensively underscore interactions with our individual and collective pasts. But there are further philosophical ramifications that emerge from the concept of deceptive genealogy, not least in relation to representation as we go on to explore.

Algorithms are of course not able to properly remember (or forget for that matter) ‘they merely calculate’ (Esposito 2022: 71), and they must do so in limited ways. MyHeritage creations must conform, for example, to a gender (male or female), or, in the case of DeepStory, to one of a limited range of voices.⁹ This suggests further ethical considerations in relation to potential biases that might emerge from the algorithm's training data and the categories used to classify it (e.g. the limitations of the gender binary), as well as about what broader cultural assumptions might be embedded within these systems, including in relation to how we represent the deceased. Given their resultant uncanniness (often referred to by users as ‘creepiness’; Kidd and Nieto McAvoy 2023), we might ask why users would wish to reanimate ‘their’ dead in this way at all, although media archaeological insights would suggest that it is not a radical or surprising undertaking. As noted above, emerging technologies are invariably tested as mechanisms for communing with our ancestors, to the extent it might be said that ‘technology is where the dead live’ (Kasket 2019: 7). Introducing new technologies to our ancestors demonstrates our deep human need to ‘live with them in more or less proximity’ (Laqueur 2015: 4); giving us a (techno-)material sense of their intimate and continuing presence in our day to day. According to Sisto (2020: 24), through the invention of photography, radio and television ‘the dead now have their voices and even their ability to move restored’, giving them ‘a unique opportunity to ‘survive’ death through sounds, images, and movement: repeating the same scenes for eternity’. The tools we explore in this article extend those opportunities quite explicitly and remarkably, but produce ethical considerations as we have noted. The algorithmically generated images result from semi-automated data collaborations and mashups (behind the scenes and less explicitly), which – in the insertion of drivers and other information – become ever more dissociated and detached from their source materials, while giving the impression of immediacy.

Ethical entanglements relate to the extent to which creations can be considered exploitative of those who have died, or a breach of (often unspoken) social and cultural norms around death. Although there are no legal barriers preventing a user from creating an output with one of these tools, there are of course moral considerations here in relation to, for example, a person's right to be forgotten, or their wish perhaps to be remembered in particular ways. These considerations clearly need to be balanced against a user's feelings of ownership over a past and the bodies that seem to manifest it, and the agency they may feel accompanies their sense of a duty to remember. Claiming the dead as our own, and for our own purposes, is of course what has driven genealogical research since its earliest days (Laqueur 2023), but it takes on new significance within these emergent formats where ‘algorithmic afterlife’ (Lambert et al. 2018) or ‘synthetic resurrection’ (Ajder et al. 2019) work across the technological and the material, resulting in forms of ‘biomediation’ (Garde-Hansen et al. 2009) which can be particularly affective, but unsettling also. We have argued elsewhere (Kidd and Nieto McAvoy 2023) that these applications can be seen as generating and exploiting what we have termed ‘algorithmic nostalgia’; eliciting distinctly nostalgic effects and affects that we understand as persistent rather than adaptive, and quite literally perpetual in that they play on a loop. The animated human bodies produced by Deep Nostalgia and DeepStory are recognisable but flimsy, seeming to channel the past, present and (lost) future simultaneously. Such material and temporal disjunctions might best be described as ‘hauntological’ (Fisher 2012: 16).

But despite these concerns about memory, and about representation, we have observed a knowingness in the ways these tools are often used. Whereas traditionally we might have understood photographs to represent an ‘uncorrupt’ representation of the past (Stone and Zwolinski 2022), most people are now familiar with the manipulation of digital images. Indeed, MyHeritage users are actively encouraged to engage in what we have termed deceptive genealogy by initiating a remediated memory without any pretence of its being authentic or truthful (MyHeritage 2023b). The promotional text on the MyHeritage platform notes that ‘the end result is not authentic, but rather a technological simulation of how the person in your photo would have moved and looked if they were captured on video and spoke the words that you provided’ (Levy 2022). All synthetic media produced in MyHeritage has watermarks and/or symbols to signal that they have been generated or augmented by artificial intelligence, and MyHeritage asks users not to remove these. In the wake of that knowingness comes playfulness too, with many users pushing at the boundaries of (or ignoring) that guidance, creating animated versions of their own image, artworks, celebrities or historical figures. The uncanniness, the creepiness and the deception become a part of the ‘magic’ in audience engagement terms, underscoring the intense virality of these outputs beyond MyHeritage as they stray into other platforms (notably X and Facebook) wherein their networked ambitions for the company can be realised. This is fraught with ethical issues too.

In reanimating the dead, users of Deep Nostalgia and DeepStory are – whether for playful or mnemonic purposes (or both) – effectively hijacking their ancestors’ free will, making them move in predictable ways but with movements they never intended (Tackett 2021). Technology is also about power, and in this particular instance, it is not just the uncanniness of the moving image, but the control of these memories that ‘creeps us out ethically’ too (Tackett 2021).

Conclusion

In this article, we have begun unpacking the socio-technical implications of MyHeritage's tools for algorithmically generating memory. In doing so, we have documented and demonstrated an enduring individual and collective need to connect with the past(s), and to test and extend our memories and recollections of ancestors through increasingly intense and proximate new media formats. We have seen how remediated memories operate within MyHeritage's corporate interests and agreements, and have examined the distribution of agency where platforms’ promotion of unique and personalised mnemonic experiences comes into tension with the algorithm's propensity to homogenise. The MyHeritage platform emerges here as a particularly potent form of mnemotechnology: crafting new kinds of algorithmic afterlife and posthumous performance; suggesting new possibilities and connections for our own and others’ datafied bodies; facilitating the (re-) scaffolding – and even creation – of memory; building tools which reach into other platforms, operate seamlessly in relation to them, and consequently expand the company's user-base. Deep Nostalgia and DeepStory remind us again that digital memory work ‘moves back and forth, on a relational level, between self and others, the private and the public, the individual and collective, and the human and nonhuman’ (Esteve Del Valle and Smit 2021: 1814). Our concept of deceptive genealogies highlights the ambivalence of the machine habitus which makes these practices possible. That they are deceptive is never in doubt – their uncanniness is a part of their appeal – but their ambiguity is ethically consequential, as we have tried to demonstrate. In this article, we have bridged debates in Critical Algorithm Studies and Digital Memory Studies to take initial epistemological steps within this nascent field, but there will be much work to do (including on the psychological and philosophical implications) as generative AI technologies become ever more pervasive.