Royal Musical Association Study Day
Ludomusicology: Game Music Research
Approaches and Aesthetics
St Catherine’s College, Oxford
16th April, 2012
9:00–10:00: Session 1 Musical Structuring in Video Games
Michiel Kamp (Cambridge University): ‘Death and diegesis: music structuring gameplay in three platform games’
What makes video games unique as an audiovisual medium is not just that they are interactive, but that this interactivity is rule-bound and goal-oriented. This means that player experience, including experience of the music, is somehow shaped or structured by these characteristics. To investigate how this works for music specifically, I will start by adopting James Gibson’s ecological psychology of perception, and his concept of affordances in particular; in a game, players perceive the environment and gameplay situations in terms of the goal-oriented actions they afford. Nondiegetic music can play a unique role in the structuring of these affordances, acting ready-to-hand in Heidegger’s terminology. I will take as my case studies three platform games, Super Mario Bros. (Nintendo 1985), Braid (Number None 2008) and Super Meat Boy (Team Meat 2010), to show how the music’s nondiegetic nature shapes the situation of the death of an avatar. Music can either “suture over” (in Claudia Gorbman’s film music terminology) death events by stretching over diegeses, or emphasise them through cadences and repetition. The strict two-plane notion of diegesis that I employ, also borrowed from film music theory, has been criticised by Kristine Jørgensen in favour of “transdiegesis,” but I argue that for my case it is sufficient, and potentially phenomenologically more appropriate.
Stephen Baysted (Chichester University): ‘Palimpsest, pragmatism and the aesthetics of genre transformation: composing the hybrid score to Electronic Arts’ Need for Speed Shift 2: Unleashed’
Electronic Arts’ Shift 2 Unleashed (2011) has a unique hybrid cinematic score that renegotiates and re-imagines ten ‘chart-topping’ anthemic rock songs from bands in the US, Canada and the UK. As a composer, audio director and sound designer of the game, this paper (illustrated with video and music) will explore the underlying aesthetic objectives of the music and its compositional processes, the commercial tensions inherent in the production of a AAA game franchise and their impact on creative musical interventions, and how the score functions as a cohesive, unifying force governing the player’s emotional responses.
The central game mechanic is to immerse the player at the heart of the ‘real’ racing experience. The aesthetic goal of the game design is to represent the ‘driver’s battle’ and Shift 2 offers the player a hitherto unprecedented level of visual, physical and sonic realism in order to achieve this. The score’s primary objective is, unusually for the genre, a narrative one seeking as it does to both describe the ‘real’ racing driver’s emotional and psychological journey and by representing and enhancing the concomitant experiences of the game player. The score operates by carefully referencing the musical and orchestrational vocabulary from three key film genres and these vocabularies inform and guide the transformation of the songs into fully-fledged cinematic musical productions that the player emotionally identifies with.
10:00–10:30: Keynote Address
11:00–12:30: Session 2 Musical Aesthetics and Dynamic Music
Tim Summers (Bristol University): ‘Epic Texturing, the First-Person Shooter and Video Game Music Aesthetics’
This paper aims to begin the work of analyzing the aesthetics of video game music through examining the diverse music of one particular game genre. The first-person shooter genre allows the investigation of a game domain that foregrounds a topic with which game music is often said to be fundamentally concerned: player immersion. Music is used to ‘texture’ the game: the score deploys strategically chosen signifiers of one type of another to furnish the gameworld by importing associations that can support signification on the visual level, or can fill in gaps left on the visual (or dialogue, or other textual) levels. This effect has the result of creating depth, implied detail and rounded context to the surface level of gameplay activity. In the first-person shooter, texturing is used to involve and immerse the player through creating a sense of the epic (a concept that will be adapted carefully from literary studies).‘Epic texturing’ provides a way of understanding one function of game music. Through a series of case studies, including Wolfenstein 3D (id, 1992) and Doom (id, 2003), the paper investigates the way in which music is used to help create a fulfilling game experience for the player. The study of the music of games reveals not only the musical strategies utilized to generate human-affective responses, but also provides a new perspective on the complex interaction between music and wider culture – what will be discussed as the ‘playful negotiation’ of music in everyday life.
Mark Sweeney (Oxford University): ‘Isaac’s Silence: Delineating the Game Music Aesthetic’
In the third-person survival horror, Dead Space(EA, 2008), the player’s character, Isaac Clarke, is a silent protagonist. Given the widespread acknowledgement of the importance of ‘immersion’ as a primary goal for the video-game medium, this narrative device is of particular interest as it permits and encourages a symbiotic relationship between player and avatar. However, the dialogue in the game is composed in such a way as to make Isaac’s own lack of communication seem odd. The alienated player’s immersion is thereby jeopardised. Isaac’s silence is both literal and figurative, as his expressionless adventure also leaves a communicative vacuity at the centre of the game’s drama.The literal acoustic void left through Isaac’s silence is filled by both music and sound effects. The blurriness of this distinction problematizes the aesthetic distance between them and goes hand-in-hand with Jason Graves’s (the game’s composer) research into avant-garde experimentation with aleatorism and indeterminacy, ideas also found in John Cage’s Silence. On the figurative plane, the particular combination of neo-romantic and modernist sound-worlds suggests a tempting ‘solution’ in which the former acts as an independent ‘narrator’ while the latter functions as Isaac’s internal voice. However, this possibility is further complicated by the implementation of a ‘dynamic music system’ which both adds to and negates the aleatory techniques employed by the composer. Through a consideration of the issues raised by Isaac’s literal and figurative silence, I will investigate the acoustic limitations of the video-game music aesthetic, and the sound-worlds it favours.
Hans-Peter Gasselseder (University of Salzburg) and Maria Kallionpää (Oxford University): ‘Re-Orchestrating Game Drama: The Experience of Dynamic Music in Videogames’
Since the very foundation of the idea of the „Gesamtkunstwerk“ the relation of music, drama and emotion has become a frequent research interest. Modern video games take us one step closer towards immersive experiences while relying heavily on the emotional congruency of the presented stimuli. By offering nonlinear storytelling environments that require corresponding musical drama, video games form a valuable tool in research on the relation of music, drama and emotion as a function of user action. Thus an adaptive music score can not only be seen as a means of describing the drama of a given narrative framework but in addition as an individual component of perception and behaviour related phenomena of the player.
Within the outlined framework this paper presents the preliminary results of a pilot study exploring the emotional arousal and valence, imaginary immersion, spatial presence, game experience and game performance in the context of dynamic and non-dynamic music in action-adventure video games while at the same time considering trait tendency for immersion, musical empathy and sensation seeking. In the first experiment 30 participants were asked to answer self-report questionnaires of experiential states each time after playing the videogame “Batman: Arkham City“ in one of three conditions accounting for (1) dynamic music, (2) non-dynamic music/low arousal and (3) non-dynamic music/high arousal, in this way manipulating affective arousal, structural-temporal aspects and emotional congruency of nondiegetic music separately. Besides subjective measures, objective data on gameplay performance was collected and is discussed with the aforementioned dimensions of game experience.
12:30–13:00: Keynote Address
14:00–15:00: Game Industry Session
Nimrod Productions (Rich Aitken & Ed Scroggie)
Having cut their teeth in the music industry, the pioneering composer-production team of Marc Canham and Rich Aitken formed Nimrod Productions in 2001. With over 100 million video game sales demonstrating Marc’s instinctive composition style and Rich’s detailed and contemporary production techniques, this complete team of forward-thinking audio minds have made a huge impact in the world of game audio; from their edgy orchestral soundtracks to more contemporary sounding scores and supervising licensed soundtracks. Working from their studio headquarters in Oxfordshire Marc and Rich have developed Nimrod and the NSO into one of the most respected orchestral teams, composers and production houses in the music business. Over the years Nimrod has pulled together a team of talented and experienced individuals who work on our projects. With an emphasis on live performance driven music and exceptionally high standards of recording and post-production, Nimrod has gained a reputation of being a top-class audio team and a great bunch to work with.Key personnel are Marc Canham (Composer), Rich Aitken (Production Director), Dr Jonathan Williams (Orchestral Director), Andy Gannon (Producer) and Ed Scroggie (Engineer).
15:00–16:30: Session 3 Analyzing Game Music
Petra van Henten (University of Utrecht): ‘Applying the ITPRA-theory to Videogame music analysis: interdisciplinary approach for videogame music analysis’
In videogame research an intermedial analysis is required for there are visual and aural stimuli in gameplay. The paper suggests the application of the ITPRA-theory of David Huron, a psychological theory of expectation that can provide a moment-by-moment analysis of the psychological effects of common musical devices such as suspension and anticipation. The main question of the paper is ‘How can the ITPRA-theory, proposed by David Huron, be applied as a practical analysis for videogame music and therefore provide for a more well-rounded analytical perspective in videogame music analysis?’ The method used for this paper was the combination of three approaches in videogame music analysis (Csikszentmihalyi’s concept of Flow, linear and non-linear music, diegetic versus non-diegetic music), the ITPRA-theory and a case study on Assassin’s Creed(Ubisoft, 2007). The paper suggests that the ITPRA-theory provides the possibility to pinpoint the musical oneset event which has the most influential emotive power; hence it could be possible to trace the tension system in the music. The tension system could enhance the immersion of a player in the game (the fictional space) and increase the flow state. The paper concludes that even though the use of the ITPRA-theory indeed can provide a more well-rounded analytical perspective in videogame music analysis, the interaction with the visual aspect of gameplay has little attention.
Samantha Blickhan (Oxford University): ‘Listening and Digital Interaction in Björk’s Biophilia’
Within the past five years, technology has brought the popular music album into a new realm of possibility. What was once a purely auditory experience is now able to stimulate other senses, such as touch and sight, as well as offer an interactive experience to the consumer. This drastic change to the format of the music album demands a response to the question: how does an interactive relationship affect the way consumers listen to music?
Tom Langhorst (Fontys University of Applied Science): ‘Syntax error: Affective Prosody and Vocalization based Musical Icons in Game Sound Design’
The paper uses Björk’s new album, Biophilia (2011), as a case study. Biophilia is an interactive multimedia project available for use on the iPad and iPhone. The paper will focus on ‘Solstice’, a song from Biophilia, and present examples of the ways in which the app interactions affect the consumer’s experience. It can be argued that a directly interactive experience with a piece of music allows listeners to invest themselves in that music through the choices that they make, a wholly different experience than simply listening to an album of music. By recognizing their own presence within a song as a collaborator, the listener engages the music on a strongly personal level. This presentation will endeavor to facilitate discussion about where the line between creation and interpretation should (or even can?) be drawn.
The study of musical meaning has a long tradition from the old Greek philosophers to music scholars like Meyer  and Bernstein . Recently researchers have studied musical meaning not only from a music theoretical or philosophical approach but also from other scientific areas such as linguistics, psychology and cognitive neuroscience. Although whether music can communicate meaning in a semantic manner is still a topic of debate [Kivy, 2002] it is obvious that musical meaning is especially important in video games where music plays an important role in the players immersion and interaction [Collins, 2008]. Designing meaningful audio faces designers with several challenges. One is the fact that the perception of musical meaning involves cultural or learned aspects [Huron, 2006; Patel, 2008]. For video games, being a cross cultural phenomenon, this appears to be a significant disadvantage. Nevertheless video game history shows many successful examples of meaningful musical icons in classic arcade games such as PacMan, Donkey Kong, Mario World and Pong. Cross cultural phenomena of speech, the affective prosody and vocalization, are known to have psychological and neurobiological relations with musical expression [Juslin & Laukka, 2003; Juslin & Timmers, 2010; Peretz, 2010]. This paper analyses a few of these best practice examples (failure icons) with audio analyses software for speech analysis (SPRAAK). The results show that some of these audio icons carry many aspects of speech without actually being speech. The paper suggests that translating audio categories from the speech domain into the music domain is an effective and cross cultural way to communicate meaning and attract the player’s attention.
17:00–18:00: Session 4 Histories and Game Music
James Barnaby (Liverpool University): ‘Play it Again and Again and Again, Sam: “Looped” Music and the Cinematic Aesthetic in Videogame Audio’
A persistent feature of videogame design has been the tendency for developers to strive for a “cinematic” aesthetic. However, unlike film, videogames have no set linear trajectory: rather, the player has varying degrees of control over the nature and duration of events in a given scene or scenario, which presents difficulties when trying to set music to accompany gameplay. Repetition of a piece of music, or “Looping”, is a long-standing and almost ubiquitous solution to the problems of applying music to game scenarios of indeterminate durations.However, as hardware capabilities have increased, so have the expectations of audiences and game designers. Game composer and writer Scott Morton summarises the common criticisms of the technique: “not only have you eliminated the emotional effectiveness of the music by generalising it…but by looping it over and over you’ve completely detached the player from registering it altogether. And what’s worse, it usually becomes annoying after a time.”Through analysis of several case-studies, this paper explores the practice of looping, how composers are attempting to adapt the technique in order to produce more dynamic sound designs, and how the practice ultimately fits in with a tendency in the industry to strive towards a “Hollywood” cinematic aesthetic in videogames.
Roger Moseley (Cornell University): ‘Ludomusicality from Mozart to Miyamoto: Music, Video Games, and Reverse Skeuomorphs’
The history of video-game music is often told from a vantage point that foregrounds the affordances and constraints of technology as prime agents of change. Supplementing these insights, I contend that the history of ludomusical media traces an epistemological trajectory that reflects shifting concepts of musical ontology, performance, and transmission from the seventeenth century to the present day.After giving a brief overview of this trajectory, I will present examples of video-game music that reveal particular aspects of their historical lineage. Rather than subjecting video-game music to the inexorable logic of history, however, I will argue that the relationship is bilateral: the playing both of video games and of the music they enact, reproduce, and prompt can retool our understanding of ludomusical activities that are stranded in the past. Drawing on digital theorist Alan Liu’s concept of the reverse skeuomorph—the notion that a given historical phenomenon can serve as “index or placeholder (rather than cause or antecedent) of the future”—I will demonstrate that video games and their music provide a vocabulary with which to recount playful musical practices, old and new, that have hitherto eluded description.