Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments

Abstract

This article explores how computation opens up possibilities for new musical practices to emerge through technology design. Using the notion of the cultural probe as a lens, we consider the digital musical instrument as an experimental device that yields findings across the fields of music, sociology and acoustics. As part of an artistic-research methodology, the instrumental object as a probe is offered as a means for artists to answer questions that are often formulated outside semantic language. This article considers how computation plays an important role in the authors’ personal performance practices in different ways, which reflect the changed mode-of-being of new musical instruments and our individual and collective relations with them.

1. INTRODUCTION

Computation has been central to the development of new interfaces for musical expression, and this has been followed with new musical practices. During the last half-century, we have witnessed developments in the field of digital musical instrument (DMI) design and practice. However, much of the knowledge and many of the methods that have emerged in experimental music practices have been fragmentary, often responding to individual and specific artistic and musical problems. These fragments partially revolve around academic communities such as those assembled for NIME conferences since 2002, and institutions such as STEIM, established in the Netherlands in 1959. Whilst this largely heterogeneous assemblage of practices and ideas among connected communities has addressed different musical, social and technological elements of DMI culture, how computation has changed the mode-of-being of such musical instruments more generally has been less widely realised or discussed.

A DMI has been defined as a musical instrument where the digital sound generator is separate from the control interface, the two relatable via mappings (Malloch, Birnbaum, Sinyor and Wanderley 2006: 49). Whist commercial digital keyboards can fit this definition, we focus on those DMIs which have novel interfaces and mapping strategies. These DMIs are increasingly decoupled from the established relationships we have with more traditional musical instruments. Such DMIs are no longer like traditional musical instruments, such as the saxophone, sitar or piano, for which many different compositions have been written and which have established themselves as cultural icons, references and constants. Rather than being a generic type, these DMI move towards becoming a different category of musical instrument; often embodying certain theoretical approaches and affording specific practices. Today’s technological context makes it easier for composers and performers alike to develop their own new instruments and systems, tailored to each new musical context, be that a single composition, a band or an ensemble, or an interactive installation or software-based work. In this, computation is an inherent part of the DMI, much as electricity is part of rock instrumentation, and mechanical buttons and reeds are fundamental to woodwind instruments. Computation therefore shapes our relationship with DMIs and also transforms our musical norms, habits, language and intentions; it is the DMI’s unique mode-of-being in a new performance practice. Furthermore, computation impacts widely upon the nature of musical activities; how music is performed, experienced, shared and distributed.

In this article we seek to provide a new examination of our relationships with such new musical instruments. Our primary questions are: How has computation changed the mode-of-being of musical instruments? Can we think of DMIs as having an embodied quality given their digital materiality? What do today’s instruments inspire us to do differently? Can we apply the idea of a cultural probe (Gaver, Dunne and Pacenti 1999) to enquire further into their agency as far as musical composition and performance are concerned? Through considering the computational nature of new musical instruments, we question what has changed in our relationships with musical instruments, compositions, performances and musical experiences. This perspective gives us, the current authors as practising sonic artists, a lens through which to explain how computation influences the methods and processes of our musical work(s). In the following sections, we explore these questions through a phenomenology of embodied musical performance with the conceptual tool of ‘cultural probes’. This article is based on reflections around our creative processes when working with computational devices, and are the outcome of a workshop held at Goldsmiths, University of London, in early 2019.

2. PROBING: AN INTERCONNECTED ECOSYSTEM

Bill Gaver, Tony Dunne and Elena Pacenti (Gaver et al. 1999) introduced cultural probes as a tool for Human Computer Interaction (HCI) research and development in the late 1990s. Gaver and his team had been tasked to design projects for various communities, and order to initiate a dialogue with the users, the team created toolkits that enabled people to document their lives in a creative and lively fashion. As Gaver (1999: 22) describes: ‘The cultural probes—these packages of maps, postcards, and other materials—were designed to provoke inspirational responses from elderly people in diverse communities.’ The ‘probe returns’ of postcards, maps and other artefacts were used to start a dialogue so that the designers felt they were almost ‘designing for friends’ (Gaver, Boucher, Pennington and Walker 2004: 54). The idea of a cultural probe began to resonate throughout the HCI community, and over the next few years this idea was adopted by various designers and researchers, becoming a valuable tool in co-design and related design practices.

We propose that cultural probes can also be used as a theoretical toolkit to explore how we relate with DMIs. This is not to say that DMIs are necessarily cultural probes per se, but rather that thinking through DMIs and the greater context in which DMIs are used in terms of cultural probes can be fruitful in discussing the changed mode-of-being of these musical instruments. It should be noted that cultural probes, as used by Gaver et al., refer to specific packages of materials. For our purposes here, we would like to imagine a probe and its returns to have broader media and materiality. Furthermore, the idea of cultural probes has been an interesting ‘probe’ into the HCI community itself, stimulating debates around methodology, uncertainty and the role of ‘the scientific process’ in design, something addressed by Gaver himself (Gaver et al. 2004; Hutchinson et al. 2003). We believe that it is in the spirit of cultural probes to adapt the concept and apply it to DMIs. Gaver et al (2004: 53) note that the probes demonstrated that ‘research questions could be packaged as multiple, rich, and engaging tasks that people could engage with by choice and over time’. What research questions, then, can be asked through DMIs and the contexts in which they are used?

When performing with a DMI, we likely depart to some degree from traditional musical performance practice. This can begin a dialogue about what constitutes music and performance, and open up discussions about what ideologies we want to embed in the instrument and inscribe in its computation. This might range from the extremes of being told (hopefully without too much cruelty or aggression) that what we are doing ‘isn’t music’ or that our chosen tool ‘isn’t a musical instrument’, through to more positive responses, for example that a performance or instrument is ‘expressive’. Whereas Gaver et al.’s ‘probe returns’ constituted feedback in terms of photographs, annotated maps and postcards, the returns we receive from our probes come in the form of bewildered faces, dancing bodies, applause, heckles, camera phone footage and perhaps the occasional piece of rotten fruit. For the most part, we informally digest and process this feedback, and respond to it in (future) performances.

In this way we can consider DMIs as cultural probes insofar as they are a means by which we, as musicians and researchers, seek to provoke and gather responses from audiences and listeners in a dialogue about what constitutes music and musical performance (Figure 1). To design a DMI is to probe musical history and to ask what musical ideas and ideologies we subscribe to.

Figure 1. In 4 Hands Iphone performance series, Atau Tanaka and Adam Parkinson re-appropriate the iPhone and its advanced technical capabilities to transform the consumer and iconic object into an expressive DMI.

In many instances, the end result of the design process surrounding a DMI is best understood as the design of a musical experience. In this way, the musical concert itself can be conceived of as a cultural probe. It refers not just to a single performance with a DMI, but also to the larger context within which performers, promoters and audiences are conceptualising and experiencing musical performances. This includes everything from the stage, lighting, the type of venue or soundsystem, to whether people are seated or dancing. This can be conceived of as an ecosystem within which musical experiences involving the DMI are collectively imagined and instantiated. Concerts are the means by which performers, curators and audiences interrogate a new instrument and the performance practice around it. Therefore, the way in which we choose to frame the presentation and reception of the instrument can do as much ‘probing’ as the DMI itself and can be a way of packaging a research question.

When instruments are put in the hands of performers, a further probing occurs. Musicians often use musical instruments in ways that the original designers never intended, probing for hidden affordances. An oft-cited example is the Roland TB 303, a bassline synthesiser that was designed to provide backing tracks for bass players. It turned out to be awkward to programme and essentially ‘failed’ at what the designers wanted it to do. However, a few years after its release, DJ Pierre was experimenting with one of these devices and accidentally stumbled upon the ‘squelchy’ sound that came to characterise, and effectively invented, the Acid House genre which spread across North America and the UK (Reynolds 2013). Taking another example, saxophonists such as Evan Parker and John Butcher reveal ‘hidden affordances’ (Gaver 1991) that were never deliberately designed into the instrument, using extended techniques to produce multiphonics and ‘unintended’ sounds from the instruments that have come to define the style of these musicians. There are many similar cases, all of which undermine, or at least problematise, the notion that even if an instrument can be designed according to a specified set of criteria, it can never be fully limited to those definitions.

DMIs as cultural probes have specific functions in this context; by giving an undefined musical technology, albeit from a specific cultural context, to an instrumentalist the designer is able to observe the musician’s response and how ideas emerge through the use of the instrument. This has been explored by instrument makers, such as Ulfarsson (2019), whose halldorophone instrument has been adopted by various performers around the world (Hildur Guðnadóttir plays the halldorophone on the Sunn O))) album Life Metal). The results have been interesting: the instrument shapes the musician as much as the musician shapes the music, but this is not a simple picture and innumerable parameters come into play. Through the instrument as a probe given to the world, to other users, the designers of the DMI are able to explore the character of the instrument in important aspects that are not available when using the instrument themselves, as they are too familiar with its functions. It is only when the instrument is released, or set free, that we begin to see its alternative potentials.

One criticism of DMIs is that, in many instances, they are always being updated and evolving, preventing the performer from acquiring the sort of virtuosity we associate with instruments of the past. However, if we see DMIs as probes, it makes sense that they are always in development, and that they are part of an ongoing design process, forming one element of a multi-sided conversation. The relationship between audience feedback and instrument design is not such that the DMI performer necessarily responds directly to every criticism uttered, much in the way that Gaver et al. (1999: 29) note ‘Although the probes were central to our understanding of the sites, they didn’t directly lead to our designs. They were invaluable in making us aware of the detailed texture of the sites, allowing us to shape proposals to fit them.’ Likewise, this probe becomes part of the DMI’s performative ecosystem to which we respond.

The design of DMIs is often rooted in rich, multimodal and conversational design processes that resonate with the cultural probe’s methodology, as opposed to a more ‘scientific’ process. Attempts to assess the ‘success’ or ‘failure’ of a new DMI can prove to be difficult. Knowing that a DMI enables a performer to reach a distant octave in a short time period may not be as valuable as knowing that a DMI made people dance or enjoy a concert, but the latter is likely to be harder to measure and quantify. Having considered the DMI as a probe in a larger ecosystem, next we consider how phenomenology has looked at the ways we probe our world through technology.

3. PHENOMENOLOGY IN DMI PERFORMANCE

Phenomenology is a wide-ranging philosophical movement that studies human experience. It looks at how we make sense of the world and takes an approach that rejects Cartesian dualism. At the beginning of phenomenological thought, Husserl proposes a scientific, ‘systematic enquiry into our conscious mental processes without regard to their non-mental causes and consequences’ (Inwood 1999: 2). Heidegger breaks with Husserl’s neo-Kantian view and considers the philosophical question of being, and extends the word Dasein to think about ways of ‘being-in-the-world’. He coins the terms vorhanden (present-at-hand) and zuhanden (ready-to-hand) to distinguish whether an object is the focus of attention or is transparent to the action that is carried out through it (Inwood 1999). These concepts have been taken up by HCI researchers, including Dourish (2004).

Different aspects of phenomenology have been applied to studies of sound and music, and of musical instruments. Cox (2001) takes an embodied cognition approach to studying musical meaning and proposes a mimetic hypothesis where a listener naturally internally imitates a performance they experience. De Souza (2017), in his Music at Hand combines music theory and phenomenology to study ways in which traditional musical instruments become creative prostheses, technologies that condition and involve the body in otherwise seemingly cerebral tasks such as notated composition. Instrumentalists are familiar with the experience of sensing the unique character of an instrument, as if each one is a probe into unknown musical possibilities.

Ihde (2007) carries out a phenomenological study of sound and uses the philosophical frameworks put forth by Husserl, Heidegger and Merleau-Ponty to unwrap the act of listening. By emphasising phenomenology not as philosophy, but as an activity, we might liken Idhe’s gesture to Christopher Small’s transformation of music into the act of musicking (Small 1998). If doing phenomenology becomes a way to study our experiences of the world, we can deploy DMIs as scientific instruments or probes in which we further seek to understand our human condition through active musical perception (Noë 2004). Phenomenology serves here as the theoretical foundation of the instrumental probes we have in our methodological toolbox.

Instruments can function as phenomenological probes, as the example of David Sudnow’s Ways of the Hands (1993) demonstrates. For Sudnow, learning the piano late in his life became an investigation into a phenomenological condition between human and instrument, and writing the book became a reporting mechanism, akin to the postcards and artefacts with Gaver’s subjects with which the journey and its insights are captured. By bringing together insights from De Souza (2017) on the embodied cognition view of musical instrument practice with Dourish’s (2004) phenomenological understanding of human–computer interaction, we can ask how does the computational nature of DMIs reframe the phenomenological probe? What does it mean to ‘think’ with a probe, to utilise an instrument that might change or evolve over time? Are there modes of feedback – visual, haptic or sonorous – that can strengthen the performer’s focus such that new insights emerge? (Figure 2).

Figure 2. Atau Tanaka performing the piece Myogram that explores the digital music performance with sensors, networks, mobility and performer body.

However, a phenomenology of digital musical instrument performance cannot reside solely in the performer relationship with the instrument. If the performance takes place in a public setting, the experience is not complete until it is seen and heard by an audience. In the The Meaning of the Body, Mark Johnson describes forms of intersubjectivity that are useful for us to consider how a performer’s experience is communicated (Johnson 2008). Using phenomenology to probe a DMI performance, we can study the experiences of performers and audiences. They can then be combined to form a holistic understanding of performance and reception of a concert taking place on digital musical instrument technology. The non-dualist orientation of post-Husserlian phenomenology helps us to consider visceral and mental musical experience together as an intertwined whole. Studying performer and audience together may allow us to understand the experiential space that is afforded by interactive instrument design. Beyond the instrument, it may help us to understand the ways in which the performance experience is transfused through different bodies – the performer’s body, the audience’s bodies and the social body through which a moment is shared.

4. DIFFERENT BODIES

In thinking about phenomenological probes and bodies, how do we as active participants in the ecology of sonic-making relate to this, especially in a performance setting? How do we feature as bodies in this becoming-practice? What does this ecosystem of body and computation in digital instruments have to say about the music that is being produced?

From a phenomenological perspective, Ihde speaks about the significance of ‘listening to the voiced character of the sounds of the World’ (Ihde 2007: 147), equally inner imagined sounds and auditory phenomena experienced in hearing. In performance this relates to the sounds we create, and how we imagine our audience to experience them. This probing-as-listening connects the performer’s body with the instrumental body, the body of the audience, the architectural body and, crucially, the sonic body that emerges from their interplay. The instrumental body in this configuration is an interesting one with far-reaching computational implications on the sound that is being created, and how performers and audiences might relate to them by active listening. In the twentieth century, the encounter between new technologies and new philosophies gave us new musics and new listening perspectives, such as musique concrète and reduced listening. Ihde surmised that ‘perceptions are concretely situated within an emerging metaphor, a newly oriented system’ (Ihde 2007: 233). If we hear differently it is because we exist in the world differently. At the same time, within both the creation of and the listening to the sonic body, past memories emerge and are encoded continually, both culturally (Castells 1996) and personally speaking (Demers 2010); the same applies to the design of DMIs. Regardless of how the past influences auditory perception, it always takes place in the present. Even if technology shapes our listening and making of music, these are clearly deeply embodied and intertwined processes. For DMIs to work fluidly there needs to be an immediacy of interaction between body and interface. A short response time between imagining sound, executing sound and listening to the sound made, enables the performer’s body to feel intimately connected with the sonic output of the instrument played.

In Western art music, composition and performance have become seen as separate activities: according to this model a composer encodes her ideas in musical notation, and the performer then executes this score in sound. Later, some of these separate activities began to overlap as sounds could be ‘sculpted’ according to the composers wishes (Ihde 2007: 262); for example, via editing and manipulating tapes (and later desktop computers), guided by the probing ears. Further down the digital timeline, and in regard to the augmented voice pioneer Pamela Z, the ‘body becomes redefined as both a source of data and as a bidirectional, permeable membrane of transference’ (Lewis 2007: 77). In short, any envisaged ‘instrumentation’ becomes part of the composition process, imagination activated by the body and vice versa. This ‘instrumentation’ also activates us and contributes to the music that unfolds. And so, while we make music with/through an instrument, each instrument we engage with also leads us to create certain sounds, or at least favour some (Figure 3). This applies to both haptic properties and computational parameters, and their joint ‘workflow’ (Brown, Eldridge and McCormack 2009). At the core of the composition process is a system which ties together the performer, instrument, audience, performance space and sound. However, this system is never completely fixed, and to boot many of these demarcation lines have become blurred. It requires listening as a probe to retain balance in flux.

Figure 3. An example for improvised augmented voice performance by Iris Garrelfs. Performance took place at the Barbican during the 2013 HackTheBarbican event, curated by Music Hackspace.

Thinking back to Gaver’s notion of probes, we can extend this to the probing of spaces. Considered in conjunction with Castells’s observation that spaces express society (Castells 1996), perhaps we also need to re-think the spaces that we want to inhabit as musicians and sounding artists, or how we inhabit them. For instance, by creating mobile performances or using wearable speaker systems that allow sounds to emanate from the body that produces it, by which sound is re-combined with body, re-focusing schizophonic experience. The complexity of contemporary music technology, and how it influences composition, also includes aspects of co-creation where generative processes contribute to the music and offer the performer an opportunity for response. Probing-as-listening then also connects the performer’s body with this computational body, accessed through sound.

5. MULTIDIMENSIONAL RELATIONS WITH DMIS

The above contexts represent the diverse interrelations of bodies and the materiality of sound, offering us compelling possibilities for rediscovering our relationships with DMIs. These relations are mediated by digital technologies and are centred around the different dimensions in which the changed nature of embodiment may become manifest (Ihde 2010: 41). The word ‘dimensions’ is used here to mean the range of musical activities we undertake with surrounding technologies and our interactions with them. They are the ways in which our embodied relationships with DMIs may vary and take on diversity. The features of DMIs have traditionally been developed by novel technological configurations, at times embracing design constraints. Here, we explore how different aspects of human–technology interaction alter the mode-of-being of musical instruments and change our relationships with them. In order to do this, we will look at human–technology relations through Don Ihde’s four phenomenological modes of technological mediation: embodiment relations, hermeneutic relations, alterity relations and background relations (Ihde 1990).

In the first mode, embodiment relations, technology remains perceptually transparent representing a kind of isomorphism between our bodily actions and perceptions (Ihde 1990: 82). We can think of this as being consistent with Heidegger’s zuhanden. In this perspective the user of the technology is fully engaged with interactions with technological artefacts where the artefact itself slips into the background, disappearing from conscious thought. In this mode, technology becomes part of life, not distinct from it. It becomes an extension of our body and we use it focusing on the content and not the technology. When music becomes mediated through an instrument in an embodiment relation, we do not focus on the instrument as a distinct object but rather on the music itself. Another aspect of embodiment relations is that technology brings into perception that which was not previously apparent. Examples given by Ihde are the magnifying glass or the telescope, because these scientific instruments mediate perception, illuminating that which was invisible to the human eye in earlier times (Ihde 1990: 49). By mediating perception, technology shapes and transforms our relationship with the world around us. Ihde points out that once the sight of the mountains of the moon became visible, visual technology transformed the moon’s earlier existing spiritual context in human history into a visual perception that has changed our relationship with it. The moon is not part of the heavens any more, but it is an astronomical object, a body that orbits the earth.

Digital technologies result in deep transformations in embodied musical practices, enhancing the differences in comparison to acoustic musical instruments. Computation brings new methods of generating music with our bodies and new methods of producing sound. The quantity of differences in terms of the speed or accuracy of the control that we can have evolves into qualitative differences in being able to specify at various levels how sound develops in a composition (Rowe 2008). In this way, computation reveals new possibilities for sound. This results in digital technologies transforming the musical behaviour of new musical instruments. Digital technologies magnify our listening behaviour, and result in the acquisition of new skills in understanding the music, ultimately allowing us to experience new aesthetics in music.

DMIs become the probes that bring in a new embodied player–instrument relation with computationally enhanced variations. Figure 4 illustrates the performance setup of the Notion of Participative and Enacting Sonic Interaction – PESI interactive music system (Tahiroğlu, Correia and Espada 2013). This is an example of a DMI that enables embodied musical exploration in co-located collaborative performance, encouraging reflections about space and movements (Goddard and Tahiroğlu 2013; Parkinson and Tahiroğlu 2013).

Figure 4. Koray Tahiroğlu performing the piece No More Together written for the PESI system and three-musicians. PESI provides co-located, embodied and spatial opportunities for musical exploration with on-body mobile instruments and in-space surround speakers with motion-tracking modules.

The strength of our embodied interaction with the world around us gives us the capacity to reflect upon the ‘engaged phenomena’ that we encounter in the form of technology. Any technology could challenge us in a similar way to how an artwork might change our view of the world – when we look at a great painting the world opens up as we act on it (Noë 2012: 2). There are times in embodiment relations where the technology becomes too easy, too transparent, and even prescribes actions through the abilities it bestows upon us. Designers can remove the possibility of challenging users to critically respond to a technology. By making everything ready at hand (vorhanden), ready to use and easy to facilitate, the technological world empowers itself. Contrary to such a relationship, computational technologies can give us great technical advancements when designing musical instruments; as the user can redesign it through programming or hardware hacking. Such a relationship might empower the user in their use (and misuse) of technology, allowing us to discuss the changed mode-of-being of a musical instrument through other relations we have with technology. This is what Ihde calls hermeneutic relations (Ihde 1990: 80).

In hermeneutic relations, users of a technology rely on their interpretive capabilities to ‘read’ some aspect of the world through that technology. In this type of a relation, the technology itself ‘becomes the object of perception while simultaneously referring beyond itself to what is not immediately seen’ (Ihde 1990: 82). The computational nature of the DMI brings in such hermeneutic relations, another type of digital transformation, that takes our interpretive sense of embodiment (perception) to refer to the possible ways in which we interpret sound and music. Such interpretation appears as a common method of practice in building and rebuilding DMIs through which we reflect and think about computational tools (Magnusson 2009). In this hermeneutic relation with computational tools, DMIs present themselves as probes through the ways musical sound is transformed into musical object. Ihde (1990: 96) calls this transformation ‘instrumentation’, comprising our actions through technology, something that also exists in our relationship with acoustic instruments and is therefore common to all musical instruments (Vasquez, Tahiroğlu, and Kildal 2017).

The computational features of DMIs represent hermeneutic qualities that are new to the domain of musical instruments. Figure 5 shows the ‘live-coding’ performance in practice, in which coding becomes an embodied hermeneutic technic, resulting in interpretations of music not only perceived by the performer but also by the audience. Taking Ihde’s ideas further, we could argue that certain sound-generation features in computer-generated music, such as random processes applied to audio synthesis algorithms, or generative algorithms in musical composition, further unfold into sets of relations that turn the DMI into a probe and cause it to become the other, constituting a phenomenological mode Ihde terms as alterity relations (Ihde 1990: 97).

Figure 5. Thor Magnusson live coding at a festival in Bristol’s Arnolfini. The code is written in real time and presented on the screen, which opens ups for further interpretation by the audience.

The alternative and otherness relationship (Ihde 1990: 98) can take place where there exists a unique type of otherness in which autonomous and intelligent behaviours are applied to music generation; for example, with machine learning and artificial intelligence tools (Briot and Pachet 2017; Tahiroğlu, Svedström and Wikström 2015). These advanced computational technologies have been applied to some DMIs, forming technologies that are alternative to human musicians and to commonly practised musical instruments. The computational features of automatic response behaviours potentially turn a DMI into an entity, generating relevant musical actions to be performed, monitoring the music space, performing musical events that would challenge the human musical discourse (Tahiroğlu et al. 2015). These advanced automation features on musical responses bring independent decision-making technologies into our relationship with musical instruments. One could argue over whether the otherness of the DMI is less strong than the otherness found in an alternative human musician. The new relationship with DMIs provides such alterity relations with an equal sense of interacting with something other than me as discussed by Ihde, not in the sense of a competitor but more in line with a dialogue in the co-creation sense. The otherness for alterity relations with DMIs probes the possibilities of alternatives to the relationship we have with musical instruments that are designed as tools to serve musical objectives. The DMI might come to be regarded as having its own objectives and self-purpose within itself: ‘having a life of its own’.

These relationships we have with technology affect to different degrees the ways in which music made with DMIs is experienced. Some phenomenological modes may bring certain effects to the fore more directly than other relations. Ihde introduces the term technological texturing in background relations to describe the ways that technological interference is linked to our lived experience (Ihde 1990: 109). Computational technologies texture music performance environments in such a way that an ‘absent presence’ (Ihde 1990: 112) of technology in background relations may transform music perception. For example, surround sound systems afford a particular type of musical performance and perception that opens new dimensions of human–technology relations. Concert spaces in their ‘absent presence’ role where their acoustics have a prominent effect upon the way music is experienced but it is rarely the object of focus. The environment in which the music is presented reflects our background relations with music – something that can be studied by the use of probes in real-time experiments.

6. KNOWLEDGE IN DMIS

The previous sections explored our relationships with our instruments, and how we form bonds and interpret them as extensions of our physical body, as an augmented body-image. We can also explore the qualities that bestow instruments with this potential. What is it that gives instruments their character, depth and personality? Asking such a question will quickly take us to a realisation that the materialities of our twenty-first-century instruments are heterogeneous and resisting clear organological categorisations (Magnusson 2017). We find that acoustic instruments embody certain qualities of reliable behaviour and continuity, offering standards in performance and composition. We also discover the incredible flourishing of experimentation in instrument design and musical media brought with electricity, electronic and computational technologies. Here, our instruments bring new modes of thinking, performing and understanding sound and music. Digital technologies are a revolution in this sense, as algorithmic control makes design flexible, fluid and redefinable through code. Hardware is here written, like functional poetry.

Conceiving of the development of instruments in these three distinct, yet overlapping, material substrata of the acoustic, electric and digital, we also find that the instruments incorporate and embody certain musical qualities. Tunings, scales, time signatures, tempi and other musical elements are increasingly written into our tools. The 16-step paradigms of electronic sequencers and drum-machines, or the 4/4 time signature and 120 beats-per-minute setups of most DAWs, implicitly direct the way we conceive of music-making in ways that acoustic instruments never did. The result is that music has become more homogeneous and standardised, following a technological normatisation in composition, performance and mixing-mastering techniques (Percino, Klimek and Thurner 2014).

Mark Fisher recently asked, how might it be, with the powerful music technologies we have available, that so much of today’s music has become so homogenous and similar to the music of the 1990s (Fisher 2014). We can here consider digital music technologies as consisting of two strands: the first one is a simulation of historical technologies (notation, piano rolls, mixers, tape, outboard devices) and aimed at music production. In the design of these tools, we have abstracted technologies and practices and implemented them in the digital domain as software. Through this abstraction process, processes and musical objects are amplified, transformed, reduced, rejected, or simply forgotten. Many of the techniques that were possible with a multitrack tape machine cannot be done with a DAW, and some of those were never intentionally design, or indeed part of the device’s manual. In the world of acoustic and electronic instruments, the accidental was often embraced and there was a strong quality of discoverability in these systems, a quality that is reduced in digital systems.

The other strand of digital musical design could be conceived of as one of performance (as opposed to production). Here digital materials (MIDI controllers, DIY electronics and coding) are applied as sources for unique assemblages of musical purpose. Installations, software, apps, instruments, tools and other systems are made for specific ends in mind, often uniquely shaped to the creator’s interests to the degree that the digital system becomes a piece, a composition on its own. Here, what previously could be characterised as composing work has now transformed into inventing a system (Magnusson 2019). With the plethora of diverse ingredients available (sensors, motors, controllers, code libraries, etc.) an approach to composition with these materials as compositional elements becomes more diverse and less streamlined in alignment with the tools of the commercial software houses.

Considering how these two strands of design approaches have influenced today’s music, we see how the production software strand has homogenised music (Fisher 2014), but in the performance real-time strand a new world of musical creativity is emerging. Fisher’s quest for new music and new musical practices is less likely to be rewarded through music streamed from musical subscription providers, but rather to be experienced in live settings across the world in new music events, festivals, hacklabs and art contexts. New music embraces the performative, liveness, materiality and system design and these new musical practices do not travel as well as an MP3 bitstream on the internet.

What, then, are these new qualities of digital materials? How, and to what degree, can we define them as containing an epistemic dimension: one that functions differently from the phenomenological perspective than acoustic instruments? Any designer of a new DMI will be familiar with how the process of designing the instrument inevitably involves musical decisions. These might involve deciding upon a tuning system, a scale, a metric system and other musical parameters that will affect the music performed. The violin and the Theremin do not have a scale or a fixed tuning, but in a digital system any continuous dimension could easily be reticulated into discrete steps. Furthermore, those discrete steps (e.g., a scale) could be changed by the press of a button, thus completely transforming the quality of the instrument. This can, of course, also be done with acoustic instruments. We have movable frets on a sitar and we can use alternative tunings on the guitar, but these are relatively major operations that are rarely done, at least in the case of the guitar. It is hard to change the tuning of a flute, clarinet or oboe. The digital brings a certain fluidity and sculptability to our instruments: through algorithmic definitions we can transform the function of the material object with a press of a button.

The openness and lack of definition of digital materials (a sensor can equally be used in military, sport, games and music equipment) means that anything is possible and that, at least this point in time, most approaches are characteristically unique to the author or the designer. This blank technological space offers a method of probing musical possibilities through new instruments making new music in new contexts. There is a lot of inventiveness in the field of new musical interfaces and it is not limited to instrument design. Rather, what is happening is that musical composition is moving from operating with symbolic writing of dots on staff lines or signal writing on phonographic media to new digital systems that merge the externalisation of thought in material, symbolic and signal technologies. With the democratisation of hardware and software design through open source and maker culture, we also open up new performance contexts and cultural spaces for new music to emerge in. Inevitably this design is of such high level of function that it involves new definition of musical spaces, where technology sets the constraints of the possible, but at a heightened musical dimension compared with the constraints we are accustomed to with acoustic instruments.

7. CONCLUSION

In this article we have pointed to how DMIs can be used as probes into the nature of new music and related practices. The context of music-making has been transformed with computational technologies and networked computers, and we must consider our instrument design as real-time experiments that sometimes succeed and sometimes fail in shaping the nature of future music. The text has described how we, as active practitioners in the sonic arts, reflect upon the new computational materiality of the technologies we work with and how they pose a new mode of being in the performing arts. The figures in the article demonstrate the diverse practices of the five authors, which supports the individual sections.

We have applied the notion of cultural probes in the context of instrument design, as we find that the design process is a cultural experiment where we give something out into the world, whether that be an instrument, installation, code library or the music itself, and in that process we observe how the work is received and gain invaluable feedback or return. The instrument as a cultural probe is equally a test into the potential of new musical expression, body–instrument relations, sound in space technology, performer–audience relationship and many more aspects of contemporary musicking in a network of reciprocal relationships. These methods can be formal, as often happens in NIME development at academic levels, or they can equally be implicit in the work processes only rarely emerging explicitly, yet informally, through discussion in clubs and hacklabs. These intensive methods of personalising technology through iterative design approach can be beneficial to other fields of product design, as music presents a unique and highly intensive performance form (in terms of numbers of parameters, bodily training, timeliness, embodiment, etc.). We have long-term relationships with musical instruments. To fully appreciate the role of these designed objects in performance may require durational research studies. By studying DMIs in this way, we can think of them as computational sonic entities, as the objects that are fluid, never resting and continually opening themselves up for new definitions and usage.