Editorial: Perceptual issues surrounding the electroacoustic listening experience

Organised Sound 24/1 considers the perceptual issues that surround the experience of listening to electroacoustic music. In framing this topic, the guest editor aimed to contextualise issues pertinent to electroacoustic pratice and identify not only the perceptual constraints but also opportunities. This endeavour sought to promote an interdisciplinary dialogue that explores potential synergies for all parties involved, and also the review process reflected in this aim. The response to the initial call was as thematically diverse as the practices of the electroacoustic genre, addressing manifold ways in which perception, and cognition, become relevant.

Listening, as both a term and ability, gained importance when increasingly experimental approaches to twentieth-century music required new habits or ways of engagement from their listeners. To name only a few from an eclectic range of examples, such approaches concerned proclamations such as ‘new music: new listening’ (Cage 1961: 10), a conscious move away from the ocularcentric towards acoustic ecology (Schafer 1993), elitist stances on listening acuity expected of the target audience (e.g., Babbitt 1998) or more explicit mentions as in deep listening (Oliveros 2005), technological listening (Smalley 1997) and referential, contextual or reflective listening (Norman 1996).

Nowhere has listening in relation to new musical practices been more systematically studied and discussed, however, as in Pierre Schaeffer’s theories on musique concrète, formalised in his highly influential treatise (Schaeffer 1966, 2017, French original and English translation, respectively). Ever since, Schaeffer’s quatre écoutes, four modes of listening, as well as the more widely known derivative mode écoute reduite (reduced listening), have influenced theories across the entire electroacoustic genre, as is also the case in this journal issue. The quatre écoutes concern the French terms écouter (to listen), ouïr (to perceive aurally), entendre (to hear) and comprendre (to understand), with the English translations taken from the Schaeffer (2017) translation by John Dack and Christine North. Whereas Schaefferian theory is rooted in phenomenology, even if unknowingly at first, the quatre écoutes and their definitions make explicit reference to auditory perception and how it could be involved in each mode.

As this Organised Sound issue promotes a dialogue between electroacoustic practice and psychological sciences, some terminological conventions and distinctions should be addressed. This is due to major language differences between French and English, as Christine North acknowledges, writing on behalf of the translators of Schaeffer (2017). Most psychological research is published in English, where both the terms hearing and listening are commonly employed to describe activity related to audition. Hearing is more commonly understood as a passive, pre-attentive and largely automated form of perception, as illustrated by fundamental research in auditory physiology and perception opting for it; for example, Introduction to the Psychology of Hearing (Moore 2013) and Spatial Hearing (Blauert 1997). Listening, on the other hand, acts as an active and conscious mode of perception, which likely engages cognitive facilities such as attention and memory.

Among Schaeffer’s quatre écoutes, mainly auditory activity relates to écouter, ouïr and entendre. As their individual roles should be well established among electroacoustic practitioners and researchers, we will refrain from providing detailed definitions but would still like to highlight how the two English terms established above relate to the French terms. Hearing most closely matches Schaeffer’s ouïr, which he understood as involving passive, raw perceptions. Listening may apply to either écouter, a focused perception that seeks for real-world indicators, or entendre, where guided by intentions, listeners form qualified perceptions. Importantly, there is a potential source of terminological confusion in that entendre (to hear) does not match the usual English understanding of hearing, while the derivative écoute reduite (reduced listening) relates more to entendre than the etymologically closer écouter (to listen):

I understand (je comprends) what I was aiming to listen to (mon écoute), thanks to what I chose to hear (entendre). But reciprocally, what I have already understood (j’ai compris) directs my listening (mon écoute) and informs what I hear (j’entends). (Schaeffer 2017: 74)

As apparent in the above quote, Schaeffer’s fourth listening mode, comprendre (to understand), operates at a higher level than basic auditory activity. For one, it interprets information gathered by écouter and entendre to attain higher semantic meaning. Among the quatre écoutes, it therefore relies on cognition the most, while it also informs and modulates how listeners engage écouter and entendre. These interactions among the quatre écoutes cover a wide breadth of listening scenarios that would reflect similar interaction between perception and cognition. Although psychological theory often conceptualises these two as distinct components, in experimental research, the modulation of perceptual processes by cognitive factors can seldom be ruled out. Schaeffer’s quatre écoutes could therefore serve as a real-world model of musical listening that reflects the complex relationship between perception and cognition.

Schaeffer arranged the quatre écoutes into quadrants. Along one dimension, concrete modes (écouter, ouïr) reflect the inexhaustible palette of possible perceptual cues, whereas in the abstract modes (entendre, comprendre), intention and reflection guide perception, which likely involves focusing attention on features of particular interest or comparing perceptions against memories of auditory or extra-sonic origin. With regard to how his theory relates to science, Schaeffer opposed simplistic analogies between physical signal representations and his notion of the sound object, and he also criticised psychoacoustic methods of the time as being of too limited value for his conception of the quatre écoutes. Indeed, much of psychoacoustics at the time engaged in fundamental research, determining ranges of hearing or psychometric functions, whereas Schaeffer’s discussion of ouïr, for instance, involves multifaceted scenarios among several auditory parameters for which there was no overarching perceptual theory available. More than half a century later, a contemporary researcher would likely relate Schaeffer’s inexhaustible set of cues available to aural perception (ouïr) to the aims of perceptual grouping, such as formalised in auditory scene analysis (ASA; Bregman 1990). In a nutshell, ASA explains both the simultaneous and the sequential auditory organisation and grouping of sounds. Paired with a growing knowledge on how attention affects ASA (e.g., Shamma and Micheyl, 2010), this theoretical framework could explain, and possibly even predict, the manifold ways in which sounds interact and how listeners are able to navigate through auditory scenes. Importantly, ASA does not only explain how an auditory object is formed in our perceptual system, but also characterises relationships among sounds; for example, how certain sounds can be perceived as foregrounded relative to others or sounds can be understood at varying levels of abstraction. Despite his scepticism towards psychoacoustics, Schaeffer throughout his treatise did seem more favourable of ‘gestaltist’ approaches. In this context, perhaps the quatre écoutes were ahead of auditory research at the time, because as it turned out, ASA draws on the same Gestalt principles of grouping that Schaeffer must have already been familiar with for visual perception (e.g., Köhler 1947; Wertheimer 1923).

Schaeffer’s second distinction groups the quatre écoutes into objective (écouter, comprendre) and subjective (ouïr, entendre). Here, the emphasis lies less on low- vs high-level or perceptual vs cognitive processes but on whether listening for indicators and signs allows an objective reading of the external world or whether raw and qualified perceptions reflect the subjective experience of listeners. This distinction between écouter (real-world indicators) and entendre (qualified perceptions) mirrors the complex nature of timbre perception (e.g., McAdams 2013), where timbre serves as both a vehicle for the identity of a sound source or cause (e.g., a violin playing pizzicato) and a container of a variety of sound qualities (e.g., brightness, roughness). Likewise, Schaeffer’s subsequent notions of écoute reduite or the acousmatic situation, later revisions into reduced, causal and semantic listening modes (Chion 1994), and also the notion of varying degrees of source bonding (Smalley 1997) bear on timbre perception further.

Not only can concepts from perceptual research be associated with the phenomenological reasoning underlying the quatre écoutes, but also the latter could be used to reveal methodological issues of the former. For instance, Schaeffer noted that in scientific experiments, observations result from participants employing the subjective modes, and based on this idea, researchers would in turn have to translate these subjective responses into an objective, generalisable level of knowledge. In addition, one could hypothesise that the distinction into concrete and abstract plays an important role in scientific experimentation. Whereas researchers are likely interested in the concrete level, that is, immediate, raw perceptions, most research data are in fact more often than not based on retrospective judgments, that is, obtained through the abstract modes, which rely on recent memory traces and are prone to cognitive interference. Most research methods in psychology therefore cross either of the two distinctions Schaeffer made, in each case limiting the degree to which the research objectives can be studied directly.

Also in the thematic contributions to this issue, the legacy of Schaeffer’s theories feature prominently in relating the listening experience to the perceptual issues at play. In the opening article, Nicolas Marty revisits François Delalande’s notions of taxonomic, empathic and figurative listening (Delalande 1998). Marty establishes the historical context of Delalande’s work at Groupe de Recherches Musicales (GRM), where Delalande sought to expand Schaeffer’s typomorphologie, and the underlying basic quatre écoutes, with higher-order strategies or approaches listeners employ throughout entire musical works. These conduites d’écoutes or ‘behaviours’ of listening are listener-centred; the English translation may again partially be lost in translation, as it could also signify the approaches affecting listeners’ behaviour or their attitude towards listening. These conduites may even extend beyond listening centred on the acousmatic tradition but also include sonic or extramusical cues that listeners may ‘hold onto’ (Landy 1994) for guidance. They thus seem suited as a universal basis to associate composers’ intention with listeners’ reception across the diverse range of electroacoustic practices (Landy 2006).

Marty identifies methodological issues in previous research on listening behaviours in that most survey methods did not reliably differentiate between listeners’ discourse of their experience, which is prone to inaccuracy, and a truthful account of their actual experience. He proposes and demonstrates the use of the elicitation-interview technique that is hypothesised to favour responses calling on episodic memory, a more reliable source of experiential information than semantic memory. Based on an in-depth case study and some related research, Marty shows how the technique yields three macro categories for structural, embodied and imagery-based behaviours, which, respectively, largely agree with Delalande’s behaviours named above. Importantly, however, the chosen method arrives at these macro categories by first evaluating at more differentiated levels of micro categories, notably also identifying instances where issues with attentional management have arisen that were not related to listening.

Schaefferian theory features again in the following article by Ulf Holbrook, in which Holbrook discusses the influence of spatial morphologies or aspects with respect to the concept of objet sonore or sound object. He identifies all relevant parameters that influence space, in terms of both physical space and acoustics and perceived spatial hearing and impression. He distinguishes between common scenarios found in conventional, instrumental music compared to generally more complex electroacoustic contexts, as these often crucially rely on spatialisation. To encapsulate the room-acoustical contribution that links loudspeakers and listeners’ ears, Holbrook employs a black-box model that couples onto the traditional understanding of the sound object. Based on the phenomenological bracketing or reduction, he argues that reduced listening in electroacoustic settings will fail to tease apart the source from the black-box contribution of the space, especially as different physical configurations introduce their own room-acoustical coloration and modifications. The sound object therefore seems intrinsically linked to a space in question, that may compare to or complement spatial considerations related to source bonding (Smalley 2007). Holbrook’s model prompts a rethinking of traditional views that acknowledges the diverse perspectives of audience members rather than the studio-centric perspective of composers. This has important implications on the general variability of sound objects across different spatial contexts and encourages the expanded definition of the objet sonore as listening to ‘sound as being of the space’.

In the third article, Iain Findlay-Walsh addresses perceptual scenarios in soundscape composition, in the shape of first-person field recordings. Here, perception is less concerned with physical sound properties and more with meaning and ‘feeling’ conveyed by sound. Findlay-Walsh draws on several theories that understand the nature of sound as transitory, ephemeral or lacking location, and how these properties afford opportunities that reach beyond those of ocularcentric perception. Subsuming these aspects into the overarching notion of embodied perception (Clarke 2005), they also contextualise to Gibson’s theory on ecological psychology (Gibson 1979), its reception in psychoacoustics (Gaver 1993), and, given the soundscape focus, naturally also theories on acoustic ecology (Schafer 1993). Findlay-Walsh emphasises that first-person field recording does not only document an ecological setting, but also serves as a testament of the recordist’s agency in the recorded environment, allowing listeners to partake in the recordist’s own perception of and interaction with the environment. This engagement through the ‘point of ear’ is discussed and illustrated through the discussion of several musical works, including his own practice.

Although Findlay-Walsh is aware of perceptual foundations of spatial hearing (e.g., Blauert 1997), and its crucial reliance on binaural cues, the deliberate emphasis on meaning in sound may have disregarded greater realism in favour of intimacy, by emphasis on the human agency, capturing recordists’ audible actions, rustling of clothing, accompanied by breathing. Interestingly, the pairing of stereo headphones with recordings acquired through coincident XY microphone technique adds another level of perceptual interest. Lacking natural binaural cues, sound sources tend to be localised inside the head instead of being externalised into the surroundings. This lateralisation, as opposed to localisation, is a perceptual phenomenon in its own right (e.g., Plenge 1974), which only arose as a technological artefact from incompatible stereophonic formats, and it is notable that this originally unintended byproduct may here find artistic utility in rendering greater intimacy.

The fourth contribution deals with an unusual acousmatic context when compared to usual electroacoustic practice. In their presentation of the ORA project, standing for Orgue et Réalité Augmentée (Organ and Augmented Reality), Christophe d’Alessandro and Markus Noisternig illustrate that most common listening experiences of pipe organs in church settings are acousmatic in nature. Although the sounds may be identifiable as organ sounds, they often remain beyond sight, as both the organ and the player are located behind the audience. Whereas this does not match the Schaefferian understanding of the term, it does agree with the original reference to the ancient Pythagorean school, and the authors discuss this in terms of phenomenology. The ORA project also touches on electroacoustics, as it entails an elaborate architecture of live electronics designed for pipe organs. Various timbral augmentation approaches are discussed in terms of their technical implementation as well as timbral and other perceptual consequences. The authors conduct a systematic taxonomy of the audio processing following distinctions into paradigmatic vs syntagmatic, that is, whether processing retains a temporal causality or not, and whether there is an interactive relationship between signal source and processor or whether effects mainly act as an additive component.

The design of the implemented audio-effect architecture subserves three aesthetic guiding principles. Fusion deals with perceptual blending of the room-acoustical space and the individual organ sound sources that is contextualised in terms of simultaneous auditory grouping within ASA (Bregman 1990). Microphony is a term borrowed from Stockhausen, as an auditory equivalent to microscopy. Its central aim is to augment otherwise inaudible sounds related to organ playing or mechanical action, thus adding the unusual sense of intimacy to the conventional pipe-organ listening experience. Finally, instrumentality ensures that the sonic nature still bears an identifiable link to the organ as the original sound source, in maintaining its instrumental causality. This final principle naturally counteracts Schaefferian écoute reduite and his wider understanding of acousmatic listening. Still, the electroacoustic intervention onto the organ can already now achieve some sense of disconnect between players’ actions and the sounding result, creating a situation that Smalley (1997) described as ‘live acousmatic music’.

In the following article on the alarm/will/sound project, Alexander Sigman and Nicolas Misdariis demonstrate the synergetic potential that a collaboration with industrial, scientific and artistic aims can deliver. Beginning with the initial scientific aim of sound design for car alarms, the interdisciplinary exchange of this project allowed science to inform Sigman’s own artistic practice. Across several stages, the acoustical description of a sound corpus, the sub-selection of sounds based on perceptual properties, and the time sequencing of sounds informed through perceptual findings find application in a diverse range of artistic outputs spanning mixed-media compositions, installations, performances and live electronics, which are discussed in the second half of the article. The science that informs this underlies several psychoacoustical studies pursuing the original aim of car-alarm design.

Several notions of perceptual continua or categories that may be relevant to wider electroacoustic practice are discussed: categorising sounds along a continuum of abstractness to iconicity allows excluding sounds whose source or cause may be readily identified and therefore be less suited for the intended purpose of auditory warning in a more gestural sense, which impinges on Schaefferian discussion of causal listening (Chion 1994), ecological everyday listening (Gaver 1993) and more recent findings that the likelihood for listeners to employ certain modes over others varies as a function of listening expertise (Lemaitre, Houix, Misdariis and Susini 2010). Also temporal morphologies or optimal repetition rates for sequencing sounds are discussed as to their perceptual evaluation along the cateogories repulsion, indifference and attraction. Another contribution concerns the discussion of acoustical correlates for perceptual categories, which, once established, may inform the selection of sounds, as illustrated for a two-dimensional feature space for perceptual spectral centroid and harmonic-to-noise ratio. In sum, common underlying psychoacoustical dependencies can inform sound design that may find utility in scientific, industrial or artistic applications.

The sixth article again concerns an in-depth foray into psychoacoustics, here, more explicitly related to timbre perception. Reporting results from two perceptual experiments, Felix Dobrowohl, Andrew Milne and Roger Dean study the ability of listeners to detect timbral differences. They criticise that most perceptual research on timbre has not taken timbral differences or variations into account that result from common approaches of sound synthesis or manipulation, echoing Wishart’s (1994) dismay about previously employed working definitions for timbre being of no value to sound-based composition practice. Furthermore, they correctly noted the predominance of findings on isolated sounds, as opposed to sounds being embedded in a more musical context. The authors therefore centred their investigation on several established parameters of sound manipulation, spanning variations of spectral slope, the ratio between odd and even harmonics, ‘pluck’ position or comb filtering, signal distortion, and the degree of inharmonicity. Perceptual discrimination thresholds were determined for these parameters, across a number of different pitches and for both musicians and non-musicians. In simpler terms, the human ability to detect minute timbral variations along these parameters was measured and quantified, by also considering the influence of varying pitch and level of listening expertise.

Of greatest interest and novelty, however, the experiments studied timbral discrimination not between two isolated sounds, but rather in a continuous single-sound rendering: differences could either be instantaneous or gradually unfold over either a tenth of a second or a full second. The obtained patterns for perceptual discrimination varied across the audio-manipulation parameters, but similar patterns still applied to both musicians and non-musicians. Musicians, however, detected timbral discrimination with greater acuity than less-trained listeners. For electroacoustic practice, these findings have important implications: (uni-directional) timbral evolutions, in other words, spectromorphologies (Smalley 1997), that evolve across sufficiently long durations may in fact go unnoticed. Moreover, less experienced listeners may not be able to perceive the slight timbral variations that composers still can discern. Whereas these two findings suggest clear perceptual constraints, the findings do still suggest that the patterns of timbre discriminability hold regardless of listening expertise.

In the penultimate contribution, John Drever delivers an intriguing personal essay that challenges long-established expectations and auraltypical assumptions on the listening capacity of humans, advocating instead for a growing awareness and acceptance of aural diversity. In his argumentation applied to electroacoustic practice, Drever highlights the growing awareness in the behavioural sciences on most studies drawing their conclusions on data obtained from WEIRD societies (acronym for Western, Educated, Industrialised, Rich, Democratic; Henrich, Heine and Norenzayan, 2010), whereas for a large body of electroacoustic music studies – and the thematic contributions to this issue make no exception – the pre-dominantly male representation serves as another limiting factor. Applied to auditory perception, many research findings are based on data from young listeners, often psychology students who receive course credit for their participation. These listeners may be what is considered otologically normal, but even standards on hearing, as published thresholds of hearing or equal-loudness-level contours (e.g., ISO 2003), are based on an average hearing characteristic, negotiating individual differences across the population.

In the electroacoustic context, however, the ‘primacy of the ear’ (Schaeffer 1957) or ‘primacy of aural perception’ (Smalley 1986) has been proclaimed and often seen as a necessity to engage with its music, but Drever raises the important question of ‘whose ear’ is meant here. If not the ‘average’ listener, Drever argues that composers instead expect from their intended listenership, as a prerequisite, a pair of ‘golden ears’, using Babbitt’s elitist, non-inclusive stance in ‘Who cares if you listen?’ (Babbitt 1998) as an extreme example. Drever not only stresses that the potential audience may digress from the auraltypical but also reminds us of composers suffering from impaired hearing (e.g., Beethoven, Smetana, Xenakis), and how that may have affected their creative work. Sensitised through personal experience and his creative and research practice, Drever calls for a more inclusive approach that embraces aural diversity and concludes by suggesting a number propositions to the wider electroacoustic community.

The final, off-topic article concerns the issues and challenges archivists face in preserving works for music-theatre. To this aim, Filipa Magalhães and Isabel Pires discuss case studies based on two works by the composer Constança Capdeville, namely Molly Bloom and FE...DE...RI...CO. After Capdeville’s death in 1992, many of her works now require preservation that also allows future stagings to respect the artist’s vision, which is complicated by their multi-disciplinary and multi-medial nature. The authors comprehensively identify and illustrate the multi-layered nature of Capdeville’s theatre scripts, which involve varying levels of informational and temporal detail and are meant for either actors, musicians, dancers or set designers. The multi-medial nature often requires a blend of sound sourced from magnetic tape paired with live sound. Thus, in reconstructing necessary documentation, video recordings are explored as one avenue, which in the meantime, however, may have degraded in quality and moreover may only provide an incomplete account. Another route taken is by interviewing former members of the production crew of past stagings under Capdeville’s direction, trying to overcome the lack of the director’s own input, by piecing together a larger context from individual accounts, although given that decades have passed since the stagings, interviewees may suffer from memory gaps. Overall, the contribution provides a in-depth guidance to the complexities of archiving multi-disciplinary and -mediatic productions, by identifying useful media formats and information collection approaches.

The thematic contributions to this Organised Sound issue demonstrate how the electroacoustic listening experience relates to a diverse range of perceptual issues. On a conceptual level, it equally may concern behaviours on how listeners engage with music, how the Schaefferian notion of the sound object is argued to be inseparable from the perceptual contribution of room acoustics, or how to approach rather specific aesthetic aims such as controlling the perceived degree of sonic ‘intimacy’ or ‘attraction’. On a practical level, a range of acoustical correlates are explored, in two cases involving experimental validation, to establish the psychoacoustical dependencies that assist in a range of sonic-design applications. Listening, as established earlier, already builds on a body of relevant theories, derived from phenomenology and psychology. For further interdisciplinary collaboration, these theories available to date seem appropriate to address the manifold perceptual scenarios possible. A general comprehensive framework that can explain and characterise the electroacoustic listening experience would, however, also need to consider, if not embrace, the aural diversity encountered across all listeners.

This editorial began with terminological issues in describing auditory activity related to listening across disciplines and languages. We will end by doing the same for the term perception. As became apparent in some of the contributions, perception may relate to more complex notions such as embodiment, and as these notions would often entail conscious thought and reflection, it likely also employs attention and memory, that is, cognition. From the perspective of electroacoustic practitioners or theorists, perception may therefore often equal reception, which relates more to explaining composers’ intentions than what is heard at a basic level. Indeed, some of the large-scale attempts to study relationships between intention and reception (e.g., Landy 2006) or to characterise the diverse range of electroacoustic idioms (e.g., Emmerson 1986) would rely rather heavily on cognition. Likewise, increasing reliance on the conceptual, as in non-cochlear sonic art (Kim-Cohen 2009), would centre on cognition, which governs in what way auditory perception contributes to an artwork’s reception.

As discussed before, psychologists tend to distinguish between perception and cognition, despite their complex interaction. To illustrate how this could apply to our case of listening, we return to the ASA framework (Bregman 1990) one last time: At the lowest level, perception that emerges from pre-attentive, primitive auditory grouping lies beyond the deliberate control of listeners and therefore serves as a perceptual ground truth. At the next higher level, schema-based grouping allows prior-learned patterns to influence perception, already calling on both attention and memory, thus allowing a listener to ‘hear’, in the sense of entendre, one way or another. From then on, listeners will increasingly call on cognition, as in the case of seeking higher meaning, which can even be extra-sonic in nature. General cognitive resources dedicated to semantics or other areas that interface with the senses would become increasingly important. Similarly, gestural or textural analogies (Smalley 1997) could relate to a growing body of research on cross-modal correspondences (Spence 2011), which has identified cross-modal mappings that humans understand implicitly, for instance, the continuum from low to high applying to both spatial elevation and auditory pitch. Given these cross-modal underpinnings, a comprehensive description of the electroacoustic listening experience would require considering perceptual and cognitive processes that extend beyond the auditory realm, which would notably even apply to acousmatic contexts.

Just as listening is a process that can involve multiple modes or behaviours, perception can be conceived at varying levels of complexity, with a growing role of cognition as the complexity increases. It is hoped that a greater cross-disciplinary understanding of each other’s theories and methodologies paves the way toward a continued, fruitful exchange between musical and scientific disciplines. The complexity inherent in electroacoustic listening experiences assumes a special position, in that it intersects with some aspects known from psychological research on more conventional forms of music while expanding the scope towards general auditory research, given its greater sonic diversity. Electroacoustic music is a fertile ground for interdisciplinary exploration, which bears synergies to advance knowledge across disciplines. The thematic contributions made in this Organised Sound issue offer an idea of where the vast opportunities lie in the future, and we hope that they will instill further collaborative research of similar kind.