Just listen: Examining the Importance of Aurality in our Multimodal Thinking

By Tracey Bowen, Joan Vinall-Cox

The continual development of new technologies and media platforms has reinforced the opportunities to produce, publish and broadcast content in new ways. Multimodal literacies and the importance of a multimodal pedagogical approach to learning has been a topic of scholarly conversations for the past two decades. Many of these discussions have highlighted the need for educators to bridge the gap between the ways in which our students communicate outside the classroom with what we are asking them to write, compose and create within the academic institutions (Bowen & Whithaus 2012). Starting with the New London School (2000), Kress and van Leeuwen (1996), Cynthia Selfe and Gail Hawisher (2004), and many others, thinking about modes of inquiry and expression beyond privileged text compositions is becoming a new norm when thinking about “writing” in Higher Education. However, a closer examination of the literature reveals that much of the research and the attention has focused on the relationship of the visual with text, or language based forms of communication. Many such as Barbara Stafford (2004), critique the privileging of text over image-based communication which has led to the illumination of the importance of visual literacy. These discussions however have not given aurality the same level of attention as was provided to the under privileging of other modalities, visual in particular.

We often focus on the visual as we move through our information saturated world, however how often do we close our eyes and listen? We are truly not exploiting the full range of modalities if aurality is only seen in relation to the visual, and the visual is examined in relation to the textual/language-based modes. Aurality is as much another important aspect of language-based literacy as text-based is also visual. As digital technologies proliferate, we must highlight the distinct qualities of aural composition and communication, not as an extension of writing, but as another way in which students compose with all the textures that sound offers. As Maxine Greene so aptly stated, we must ask ourselves, what else is possible in the ways we express and communicate our ideas. What is unique to the emerging genres inspired by audio composing?

Audio composing has continually evolved since Thomas Edison’s 1878 phonograph, and become widely possible since the arrival of easily available sound recording and editing software, most notably the free downloadable software, Audacity . What is surprising is our apparent “blindness” to the impact of the aural. The early and continual presence of the aural has led to it being widely “overlooked”. Not only is the aural sense connected to one of humanity’s earliest communication arts, rhetoric, it is also an active sense in the womb. It is omnipresent in our experiences; we cannot close our ears as we can our eyes. The story of Odysseus and the Sirens tells us that the fascination and power of sound, and its inescapability when physically close enough to hear, was as powerful for the ancient Greeks as it is today for earphones-wearing music-lovers, with music as a dominant popular art form. As much as we live within a visually saturated environment, we are often plugged into a soundtrack that parallels the narratives of our lives. Our students are immersed in sound, and, just as they need visual literacy, they also need aural literacy.

Adsanatham et al (2012) contend that audio is an important aspect of student learning and rhetorical development as students learn to “listen to themselves and to one another, and to translate this better listening to themselves and their classmates” (296). In the context of student writing, audio essays are used to better understand written texts in relation to audience, argument and word choice by hearing the composition instead of seeing the words. While it appears that audio used in this way is still in the service of the written word, Adsanatham et al suggest that hearing arguments, the chosen words, the tonality of sounds provides students with a different way to understand and develop rhetorical presentations and to “practice the analysis and production of arguments in different forms, for different purposes and audiences” (296).  Students witness “their classmates’ reactions to what they are hearing, noting what is confusing or enjoyable, if people laugh or grow silent” (297). Listening is crucial to a multimodal pedagogy that helps students negotiate diverse discourses in contemporary digital culture and gain a sense of what else is possible when the world of inquiry meets the soundtrack of their lives outside of the classroom.

 As Ihde (1976) points out, there has been “a subtle and profound transformation of experience itself as our capacities for listening are changed by technological culture” (p. 4) “What is of special interest to the thoughtful listener is then the way instruments, particularly those of the electronic era, introduce ways of listening not previously available” (p. 5). We can explore how sound carries more complex and detailed information than text. Simply find an interview produced in two formats: transcribed, and recorded. When the transcription is read first, then the recording listened to, many aspects “invisible” in the transcript become obvious in the recording: the interviewee’s accent, two different voices asking questions, the speaker sounding hesitant at one point, and so on. Learning to consciously notice what’s happening in an audio recording is an important step for aural literacy. Individuals listening intently, and repeatedly, and shared discussions of their observations help tune up this listening skill.

With recorded sound comes the layering of sound, called the mix (Rice, 2006:273). It’s an act of learning to hear the mix and understand it as part of the aural flow. In an audio documentary or podcast the sound of the plane reinforces the sense of excitement in the voice of the traveler. In a newscast, the music guides the understanding of the meaning being conveyed. Students need to become conscious of their own reactions to their aural environments. Before they can compose aurally, they must be able to hear, to read the sound that surrounds them . Discussing and comparing what they hear is essential for the development of how others, potential members of their future audiences, hear.

Composing recorded aural experiences is somewhat like composing a written text in that you have to imagine the audience’s reactions and design your communication to get the effect you want. However, an aural composition is closer to Levi-Strauss’s “bricolage” or a visual collage than to the finely detailed edited text (Rice, 2006:273). The aural is ephemeral; you can’t trap an aural frame of sound as you can a visual frame of a video. Aural