Overview
How will life change when we can no longer ignore the effects of climate change? This key question underpins my development of Speculative Soundwalking, a framework for creative projects that ask audiences to listen to speculative histories and futures of their local environment in situ.
In this literature review, I will contextualise the creative development of SpecSound (Speculative Soundwalking) in relation to the key concepts and practices it draws on from the perspectives of creative industries and media studies, with particular reference to its keystone project, Listening to Abu Dhabi. After providing background to the development and framework of SpecSound, I will then locate it in the landscape of contemporary technology, explaining how it utilises the democratisation and decentralisation of new media to create accessible public art. Following that, I will discuss how both social and semantic representation are important factors in the development of SpecSound projects, and describe its relationship to film sound design practice and the audiovisual contract. I will then move to discuss the dynamic roles of the audience and author/creator, drawing on perspectives of autoethnography to emphasise the fluid democratic relationship between them. I will conclude by contextualising SpecSound in the wider genre of speculative fiction.
Speculative Soundwalking
Speculative Soundwalking (hence referred to as SpecSound) is a mode of headphone-based interactive public art. A Speculative Soundwalk asks participants to move through a specific public place, listening to soundscape compositions that imagine the history and possible future of the space they inhabit. As they move and listen, they are invited to imaginatively embody their future existence in that space, and consider their relationship to that place through the lens of the soundwalk itself.
I established the medium of SpecSound through Listening to Abu Dhabi (Simmons, 2018), my undergraduate capstone project at NYU Abu Dhabi, in response to a growing movement of ecological sound art and GPS-based audio work in the fields of acoustic ecology and climate activism. Listening to Abu Dhabi is a Speculative Soundwalk located on the Abu Dhabi Corniche, comprised of 13 pieces, each imagining a moment in the history, present or speculative future of the city through the lens of human settlement, climate change and urban development. The pieces consist of field recordings made in the UAE, audio from online sound libraries, and my own vocal narration.
During 2023 and onwards, I plan to develop a creative framework for SpecSound, one that can be easily shared with artists and activists globally, to provide them with the tools to create their own SpecSound projects. SpecSound projects can be realised through any audio platform, but Listening to Abu Dhabi is designed to be experienced through the Echoes app (Kopeček and Fuache, 2018), which tracks each participant’s GPS location and triggers audio files to be played when they enter a specific area within the project site, as defined by the artist. This format allows participants to use their own device and headphones, moving at their own pace through the piece rather than locked to a prescribed timeline.
Technology
In using the Echoes app, SpecSound is fundamentally based on the medium of the guided audio walk (not unlike guided audio history walks for tourists), which relies on various technologies developed through the 21st century. These include Global Positioning System (GPS), open-source digital maps, portable smart devices and headphones, streaming audio, compressed sound files, and more. Arthur (in Hartley, 2013c) posits that new technologies emerge via an evolutionary process, meaning that any new technology directly descends from its predecessors, but evolves in such a way as to respond to the needs of users. When artists work with technology, they either adapt their ideas to the technologies available to them or create their own, thus participating in the ongoing combinatorial process of technological evolution. The Echoes app is thus a strong example of the actor-network theory, demonstrating that humans and technology are equal actants in driving social and economic change and shaping the future of the creative industries.
Because of the emphasis on listening in situ, using geolocated audio, SpecSound is a creative framework well suited to projects that seek to use listening to shape emplacement (affective relationship to, and identity formation in relation to place) (Labelle, 2010; see also Feld & Basso, 1996). Levy-Landesberg argues that the emergence of digital sound maps in urban environments uniquely allows for exploration and shaping senses of place, drawing on the existing hybrid space of the city (Levy-Ladnesberg, 2022). Hybrid spaces, defined by de Souza e Silva (see de Souza e Silva, 2006), are mobile spaces that are created by the intertwining of users and their portable devices. Digital maps, particularly digital sound maps blur binaries such as “public/private, remote/contiguous, and physical/digital” (Levy-Landesberg, 2022, p. 24). SpecSound utilises emergent technology to inhabit public space with very little equipment, and no additional infrastructure, lending itself well to small-budget climate activist work, or work exploring other social movements.
By temporarily inhabiting hybrid space, SpecSound projects create what LaBelle terms “auditory scaffolding”(LaBelle, 2010, p. 131). Auditory scaffolding refers to the way that pedestrians and drivers use the technology of portable music players to create a structure to associate with particular locations, moods, desires or experiences. Pedestrians, drivers, public transport users, cyclists, and anyone else moving through public space utilise technology of many kinds (digital maps, music players, Bluetooth headphones etc.) to provide “temporal and material support” (ibid.) to structure their experience of public space. SpecSound draws on this concept of auditory scaffolding as a structural model for mapping sounds to space, providing participants with a memorable sonic and narrative structure to shape their experience of the given environment. Upon returning there in the future, they will remember their experience of the soundwalk, enriching their relationship with that place going forward.
SpecSound is a product of the “movable feast” of new media, engaging evolving technology to utilise the “democratising, empowering potentials of social media” (Hartley, 2013a, p. 136). It draws on the new model of transmedia to provide creatives and activists with an alternative mode of storytelling, empowering them to create self-contained points of entry for audiences to engage with their work. Hartley explains that culturally there has been a shift towards decentralised media production, from the ‘one-way flow’ of industrialised mass media to the contemporary model, where cultural product is increasingly generated in the household sector rather than the commercial or public (ibid.; see also Hartley 2019d).
Specsound takes advantage of, and contributes to the breakdown of the traditional dichotomies of professional/amateur, entertainment/education, and community/commercialism to further democratise creative storytelling and activism (ibid.). As a form of DIY public art, SpecSound projects exist as a digital auditory scaffold that inherently and legally evades policy, law enforcement and the need for council permission.
Representation
Social
The rise of social media and the internet as a site for self-publishing has presented increasing opportunities for the DIY and cooperative mode of self-representation. Hartley explains that although representation in creative media refers to any way that materials, signs and symbols are used to communicate meaning, colloquially the public often understands ‘representation’ through its inverse, underrepresentation (Hartley, 2013b). The general consensus is that representations of identities in media should proportionately reflect the distribution of identities in the world more broadly, and that “statistical under- or over-representation of a significant group or value is widely seen as a form of political disenfranchisement” (ibid, p. 156).
The democratising value of SpecSound lies in the ability for anyone to create their own project, engaging with the local social issues that impact their own community. Despite dealing with a popularly impacting issue, Listening to Abu Dhabi is only representative of my own research and perspective. One of the key ways that SpecSound can be a site for creative self-representation is through the creation of an accessible framework document, allowing artists and activists anywhere to create their own projects and tell their one story. This framework is a product of the “rise of interactive, participatory and digital technologies” that create opportunities for “facilitation and sharing with and by users who are learning how to represent themselves” (ibid. pp 56-57).
Semantic
Moving from social representation to a more literal understanding of representation as a system of symbols, I will discuss Chion’s approach to film sound as it relates to soundscape composition in SpecSound. Hartley introduces the concept of semiotic representation as the idea of one thing standing in for another, drawing on studies of semiotics in linguistics and media theory (Hartley, 2019f). In the context of film, this mode of representation applies to most elements of storytelling; lighting, cinematography, visual design, direction and sound design each have distinct sign systems. Over time viewers become accustomed to reading and interpreting these languages, and thus become co-creators of meaning in the text, resulting in a story vastly more complex and multivalent than the sum of its individual signs.
Although semiotic representation is well demonstrated in the layered multi-sensory experience of film, it can also apply to work that deals solely in sound media. Sound design that uses Foley is a particularly salient example: recorded sounds of unrelated objects (e.g. fruit, cans, artificial flooring) stand in for real-world objects (e.g. bodies, industrial equipment, snow). These substitutions rely on the persuasiveness of other elements (e.g. field recordings of actual spaces) to help audiences suspend disbelief; rationally they know that no actors are being grievously injured, but strong sound design (in collusion with the effective use of other sign systems) can create a convincing illusion. In his article explaining the semiotics of sound design in the context of (perception of) product quality, Jekosch discusses the importance of auditory schemas in the process of semiosis, the process of organisation and categorisation that allow us to interpret signs as such. The more similar a sound (or collection of sounds) is to our existing set of experiences and memories, the more our “expectation guides perception” (Jekosch, 2005, p. 199). SpecSound exploits this process of perceptual categorisation, straddling the line between what Piaget defines as assimilation (when we perceive a new sound that doesn’t fit our existing schema, our perception of the sound is altered to assimilate with our existing system) and accommodation (instead, our schema is altered to accommodate for the new sound) (in Jekosch, 2005, p. 201). In combining unaltered field recordings of familiar environments with creative use of Foley, SpecSound draws on audiences’ existing schema to create a convincing illusion of their environment changing.
Jekosch also highlights the multi-sensory aspect of semiosis, explaining that in product design the harmony of “auditory, visual, haptic and olfactory” cues is essential in successful perception of quality (ibid. p. 205). In Audio-vision, a seminal text on film sound, Chion demonstrates the importance of audiovisual combination, explaining that “one perception influences the other and transforms it” (Chion, 1994, XXVi). Audiences therefore rely on multi-sensory perceptual experiences in forming and drawing on existing schema to interpret signs. By asking audiences to move through a familiar physical space, SpecSound provides them with causal hooks to hang onto when listening through the narrative, utilising their existing schemas of both local environment and familiar filmic conventions. From the perspective of sound and listening studies, Chion’s major contribution was his explanation of the “three modes of listening” (see Chion, 1994, pp. 25-25):
Causal: listening to sound “in order to gather information about its cause”
Semantic: listening to interpret a code or language
and Reduced: listening to the “traits of the sound itself, independent of its cause or meaning” (ibid, p. 29), a concept borrowed from Schaeffer’s “acousmatic” listening.
In dealing with composed soundscapes that prompt audiences to recall and imagine visual settings in their mind’s eye, semantic representation in the context of SpecSound mostly applies to causal listening. Audiences move through space listening to an auditory scaffolding of artificially composed environments - sounds that point to their everyday experience of local spaces but deviate in subtle and important ways. This scaffolding shapes their multisensory experience of the speculative narrative, making them “see differently” (ibid, p. 34) and adding to their schema of that environment when they return there in the future.
Audience and Author/Ship
The above discussions of semantic representation and democratisation of media production demonstrate that consideration of the intended audience plays a very important role in developing SpecSound projects. Hartley details the position of the audience with regard to how the ‘audience’ as a construction serves the interests of media organisations, media researchers and regulatory government bodies (Hartley, 2019a). In these contexts the audience is a way to imagine an “essentially unknowable group” that shifts composition based on a number of factors (e.g. location, age, gender, race, and access to technology) (ibid., p. 14). Hartley also explains that the rise of the internet and decentralised media production have problematised this understanding of the audience as passive. The “active audience” is now an economic reality, simultaneously consuming and creating content on the “cutting edge of innovation in the creative industries” (ibid. p. 18).
The audience for a Speculative Soundwalk is not an amorphous construction, but rather a live group of participants who are essential to the creation of meaning in the work, and whose experience of local environments is essential for the creative development of SpecSound projects. Additionally, the purpose of creating a shareable, open-source creative framework for SpecSound is to engage active audience members to become creators in themselves; ideally, artists, activists or community members will experience a Speculative Soundwalk and be prompted to create their own using the framework. The audience is thus both active co-creator of narrative and potential author.
In this way, SpecSound’s approach to the role of the audience in co-creating work aligns with Hartley’s position on the relationship between the author and the audience. Hartley problematises the notion of the “individual genius”, the idea that authors create finished and complete meaning in their work, which will be passively consumed by the reader (ibid.). He argues that the meanings (of a text) “are always plural, and always exceed what the writer thought was going on, intentionally or otherwise,” and it is this “polysemic potential” (ibid. p. 21) that I hope to achieve through emphasising the role of the audience in driving and creating meaning in the narrative of any given piece.
In an effort to further de-emphasise authorial authority in practice-led research, I draw on Findlay-Walsh’s autoethnographic approach to soundscape composition, in which he borrows methods from autoethnography to conceptualise reflexive, embodied sound work (Findlay-Walsh, 2018). He complicates the dichotomy and power dynamic of audience-author by building upon contemporary understandings of listening as a “perceptual, subject-forming activity... a means of producing what, where and who we are” (ibid. p. 3). He explains that the emergence of autoethnography as a methodology is in part an answer to increasing interdisciplinary “problems relating to representation, authorship and authority... in response to post-structuralist and post-colonial critiques of power” (ibid. p. 3-4). Conventional ethnography historically places the European male ethnographer as the objective author-researcher, writing with authority on the othered subject, resulting in (at best) a problematic and highly subjective representation of the people and cultures being written about. Autoethnography attempts to reposition the researcher as the “central subject of study”, and explore their personal experiences and social entanglements directly rather than obscure them through a veil of removed objectivity (ibid. p. 4).
Importantly, Findlay-Walsh explains that autoethnographic texts often “[generate] narrative ambiguities which invite interpretation by the reader, involving them in the process of meaning-making... the researcher/researched join with the reader to create a story” (ibid.). Furthermore, he explores how some field recordists and soundscape composers engage in a kind of autoethnography through their recording and composing practices, seeking to involve future listeners in the generation of meaning by positioning themselves as a fellow listener in the work (see Westerkamp, 1989). This self-reflexive practice stands in stark contrast to conventional field recording, which (like conventional ethnography) positions both the recordist and their equipment as faceless, anonymous conduits for the pristine soundscapes they capture (see Truax, n.d.).
When the audience is made aware of the presence of the recordist and/or the composer in the piece, the subject position of the artist moves from genius author to “person engaged in the [same] sense-making processes” as the listener (Findlay-Walsh, 2018, p. 6). This idea of soundscape composition as self-narrative sits well with the framework of SpecSound from two angles:
1. SpecSound projects are developed by artists and activists local to the environmental setting and are thus positioned as an equal subject to the audience. 2. SpecSound projects are designed to engage local audiences, thereby seeking to engage them in the meaning-making process by inviting them to interrogate their own positions in the inhabited environment through listening.
Genre
Having discussed SpecSound’s engagement with technology, representation, and the audience/author relationship, elements that relate to the construction and form of Speculative Soundwalks, it is important to now cover SpecSound’s relationship to genre and storytelling. As implied in the name, Speculative Soundwalking draws on genre conventions of speculative fiction. As a genre, defining exactly what constitutes speculative fiction is somewhat contentious due to a shifting relationship with science fiction and disagreement about its aims and themes, as detailed by Oziewicz (2017). This changing and problematised definition of speculative fiction exemplifies Berkenkotter and Huckin’s argument that genres are dynamic (changing over time to suit needs), situated (related to particular contexts), include both form and content of a work and are constituted by and reproduced by the actors, signalling the norms and ideology of a particular community (2016).
For the purposes of SpecSound, I draw on the definition of speculative fiction as a broad category of non-mimetic fiction, a genre of stories that question the dominant status quo, and in doing so have the “power, even responsibility, to voice alternative views” and push for equality (Oziewicz, 2017, p. 9). I also draw on Margaret Atwood’s understanding of speculative fiction as “hing[ing] on probability, although not necessarily constructed in scientific terms,” but rather exploring “narratives that can potentially take place,” given the existing state and trajectory of the world (ibid. pp. 4-5). Importantly, SpecSound “makes use of fantastic and inventive elements to comment on, or speculate about, society, humanity, life, the cosmos, [or] reality” (Merril, in Oziewicz, 2017, p. 10) with the purpose of inviting participants to interrogate and deepen their relationship to the environment they inhabit.
More specifically, Listening to Abu Dhabi (and other future similar SpecSound projects) sits in a subcategory of speculative fiction called speculative climate fiction, which imagines the future through the lens of climate change. Raipola (2020) examines the distinction between realistic climate fiction and speculative climate fiction, explaining that in the former, “recognisable human characters ponder the effects of global warming” in a relatively familiar setting. Meanwhile, in speculative cli-fi, “the entire fictional world characteristically functions as a synecdoche for the changing climate” (ibid. p. 8).
SpecSound makes use of typical iconography, repetition of codes and conventions, and familiar plotlines of speculative (climate) fiction, and in doing so provides the audience with a set of expectations around the narrative. The narrative does not subvert these expectations (broadly, that climate change will drastically change our existence and relationship to our environments). The narrative structure of SpecSound projects is simple and chronological, but the medium of geolocated audio and focus on embodied listening draws on the poststructuralist and postmodernist emphasis on the role of the reader: the audience’s engagement with the audio deeply impacts the meaning conveyed through the narrative (Edgar and Sedgwick, 2010, p. 219). The chain, or plot, is straightforward and narrated in an uncomplicated manner, but the choice to present them directly, first-person, in an embodied listening experience, creates an increasing feeling of dread and highlights the “experience of living in a changed world” (Raipola, 2020, p. 8).
In Listening to Abu Dhabi, speculative soundscape compositions are accompanied by a narrator speaking directly to the listener in the second person, giving both instructions (where to walk and what to listen for) and story (what is different between the world you know and this one). Following these instructions, the audience walks along a public path along the Abu Dhabi Corniche beach, following a chronological history and future from the Stone Age, to the 1800s, to the present day, to the year 2100. The mode of address and narrative structure, although complicated by the medium (audio) draw on typical genre conventions. The audience members are invited to fill the role of the protagonist, moving through history from a first-person perspective, and dialogue becomes an omniscient narrator stepping them through an unfolding understanding of the speculative setting (in the tone of a choose-your-own-adventure, but the narrative is determined by data projections rather than individual freedom).
Notably, while Raipola recognises that speculative climate fiction will likely have “relatively little impact on the global consumption patterns of the human species en masse,” the value of the work in this genre lies in the “metaphoric and analogical powers of speculation,” opening pathways for imagining “alternative ways of living with the catastrophe in the coming era” (ibid. p. 9). This rhetorical offering is the foundation and purpose of Speculative Soundwalking: what will life be like in the face of this change?
Final Remarks
The power of speculative climate fiction, and the purpose of Speculative Soundwalking, is to invite audiences to imagine the future of the environments they inhabit, and in doing so prompt them to ask the question, “what will happen if we continue on this path?”. Throughout this literature review, I have provided context for the current and future development of SpecSound. Specsound utilises emerging technologies to create a digital auditory scaffolding in place, capitalising on and contributing to the decentralisation and democratisation of storytelling and activism through new media. As an open-source framework, SpecSound thus has the opportunity to provide a space for self-representation by artists and activists. Compositionally, SpecSound also draws on techniques and theories of film sound to create convincing speculative soundscape compositions that help audiences suspend disbelief and cast new light on their environments.
Following an autoethnographic methodology of soundscape composition, acknowledging the creator as subject in the work, SpecSound undermines traditional author/audience structures to emphasise the importance of co-creating meaning with the audience. SpecSound exists within and responds to the conventions of speculative fiction, particularly speculative climate fiction, using imaginative narrative to speculate about the future and voice alternative views. although Listening to Abu Dhabi imagined the future through the lens of climate change, it is important to note that SpecSound projects can explore speculative futures of any topic, e.g. through the experience of marginalised communities, or through the lens of particular policy changes, or through local matters that affect communities, and so on. The value of Speculative Soundwalking is in its ability to provide audiences with a first-person look into what their future may look like, and in doing so create the space to imagine alternative ways of being.
Note: this literature review was written during a Research and Writing unit as part of my Master's of Creative Industries at SAE. Please explore my website for more writing on Speculative Soundwalking.
Works Cited
Berkenkotter, C., & Huckin, T. N. (2016). Rethinking Genre from a Sociocognitive Perspective. In Genre Knowledge in Disciplinary Communication (0 ed., pp. 1–25). Routledge. https://doi.org/10.4324/9781315538747
Chion, M., & Gorbman, C. (1994). Audio-vision: Sound on screen. Columbia University Press. de Souza e Silva, A. (2006). From Cyber to Hybrid: Mobile Technologies as Interfaces of Hybrid Spaces. Space and Culture, 9(3), 261–278. https://doi.org/10.1177/1206331206289022
Edgar, A., & Sedgwick, P. R. (Eds.). (2010). Narrative. In Cultural theory: The key concepts (2. ed., reprint, p. 219). Routledge.
Feld, S., & Basso, K. H. (Eds.). (1996). Senses of place (1st ed). School of American Research Press ; Distributed by the University of Washington Press.
Findlay-Walsh, I. (2018). Sonic Autoethnographies: Personal listening as compositional context. Organised Sound, 23(1), 121–130. https://doi.org/10.1017/S1355771817000371
Hartley, J. (2013a). Media. In Key Concepts in Creative Industries (pp. 134–138). SAGE.
Hartley, J. (2013b). Representation. In Key Concepts in Creative Industries (pp. 153–157). SAGE.
Hartley, J. (2013c). Technology. In Key Concepts in Creative Industries (pp. 157–160). SAGE.
Hartley, J. (2019a). Audience. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 15–18). Routledge. https://doi.org/10.4324/9781315225814
Hartley, J. (2019b). Author/ship. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 18–22). Routledge. https://doi.org/10.4324/9781315225814
Hartley, J. (2019c). Genre. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 147–148). Routledge. https://doi.org/10.4324/9781315225814
Hartley, J. (2019d). Media. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 200–201). Routledge. https://doi.org/10.4324/9781315225814
Hartley, J. (2019e). Mode of Address. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 205–206). Routledge. https://doi.org/10.4324/9781315225814
Hartley, J. (2019f). Representation. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 260–261). Routledge. https://doi.org/10.4324/9781315225814
Hartley, J. (2019g). Subjectivity. In Communication, Cultural and Media Studies: The Key Concepts (5th ed., pp. 15–18). Routledge. https://doi.org/10.4324/9781315225814
Jekosch, U. (2005). Assigning Meaning to Sounds—Semiotics in the Context of Product-Sound Design. In J. Blauert (Ed.), Communication Acoustics (pp. 193–221). Springer-Verlag. https://doi.org/10.1007/3-540-27437-5_8
Kopeček, J., & Fuache, Y. (2018). Echoes [Mobile Application]. http://echoes.xyz/explore/
LaBelle, B. (2010). Acoustic territories: Sound culture and everyday life. Continuum.
Levy-Landesberg, H. (2022). Sound and the city: Rethinking spatial epistemologies with urban sound maps. Sound Studies, 8(1), 20–42. https://doi.org/10.1080/20551940.2021.1982563
Oziewicz, M. (2017). Speculative Fiction. In M. Oziewicz, Oxford Research Encyclopedia of Literature. Oxford University Press. https://doi.org/10.1093/acrefore/9780190201098.013.78
Raipola, J. (2020). What is Speculative Climate Fiction? Fafnir – Nordic Journal of Science Fiction and Fantasy Research, 6(2), 7–10.
Simmons, K. (2018). Listening to Abu Dhabi [Speculative Soundwalk]. https://www.keirasimmons.com/listening-to-abu-dhabi
Truax, B. (n.d.). World Soundscape Project. World Soundscape Project. Retrieved 23 December 2022, from https://www.sfu.ca/~truax/wsp.html
Westerkamp, H. (1989). Kits Beach Soundwalk: Vol. Transformations [CD]. Canada Council for the Arts. https://www.hildegardwesterkamp.ca/sound/comp/3/kitsbeach/