Category: Guest Contribution

Iain Hart (Sydney Conservatorium of Music, Eine Kleine Pwnmusik)

C.S. Lewis wrote, in his science fiction novel Out of the Silent Planet,

“To every man, in his acquaintance with a new art, there comes a moment when that which before was meaningless first lifts, as it were, one corner of the curtain that hides its mystery, and reveals, in a burst of delight which later and fuller understanding can hardly ever equal, one glimpse of the indefinite possibilities within.”

I had this moment when I was sitting in a film music lecture and realised that the music of the video game Left 4 Dead used similar musical devices to the music of the film Psycho, and used them for the same purposes. It was a first glimpse, not of the fact that video game music was emotive (I had felt that already), but that it was deliberately expressive and powerfully designed. I’ll never forget that moment. I can’t. It’s permanently changed the way I think about game music, so as a student of game music it’s always before me. And with it, the notion that game music and film music are kindred spirits, doing the same things in the same ways.

But as I’ve studied the parallel between film and game music, the less satisfying I have found the parallel to be, and the more I have realised that the differences are as myriad as the similarities. Yes, game music does many of the same things as film music, but it also does many more (chief among them: respond to the player/viewer). And there are things film music can do that game music can’t, or that game music is only just beginning to do. Likewise, although game music and film music sometimes have the same function (space-filling, for instance), they often achieve this in astoundingly different ways. No film I’m aware of selects background music at random while you’re watching it, but this is a standard—even simple, suboptimal, obsolescent—method for keeping game music interesting. And the language used to talk about game music is subject to the same difficulties. Film music terminology can accurately describe many aspects of game music (diegesis, motifs, etc.), but it struggles to describe the effects of something as utterly crucial to gameplay as interactivity. The film/game parallel only gets us so far.

We know so much about music in films, and a lot of that knowledge informs our understandings of the music in other media, and even of music itself. It’s only right that where a parallel exists, ludomusicologists should take advantage of it and learn what we can. But that should never prevent us looking for other parallels. For instance, I have found Markku Eskelinen and Ragnhild Tronstad’s chapter “Video Games and Configurative Performances” (from The Video Game Theory Reader, Wolf & Perron, 2003, pp. 195-220) helpful for understanding the player’s role in the game because of their comparisons between games and configurative theatre. I am not a scholar of theatre, but the comparison to a performance artwork helped me reframe the argument in light of my experience with music—and, ultimately, prompted me to start making comparisons with other non-filmic media.

In some ways, music is the most artistic thing I know anything about. In school I was far more interested in scientific and technological pursuits than art, at least from a vocational point of view. I was an advanced computer user before I was an advanced computer player (and I’m still not sure I’d call myself that). I know my way around a computer quite well, and I know a bit about how its hardware and software work, and work together. This doesn’t hinder my research of music in video games; rather, this helps me understand the fundamental elements of the medium, and helps me understand why parallels to other media can be drawn to it at all.

Always, at the back of my mind, is the understanding that a video game is a computer program. A computer program looks a bit like this:

1
2
3
4
5
6

#include <stdio.h>
 
int main(void)
{
printf("Hello world\n");
}

#include <stdio.h>

int main(void)
{
printf("Hello world\n");
}

This is a very simple program written in the C programming language. Its sole function is to output the text “Hello world” on a command line (the first steps of learning a programming language often involve displaying the text “Hello world” somewhere). It’s functionally trivial, but it can teach a student about the basic structure of a program written in the C programming language and get them to understand how the computer can be controlled by their instructions. However, for the computer to run the program it has to be compiled, after which it looks like this:

01111111 01000101 01001100 01000110 00000010 00000001 00000001 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000010 00000000 00111110 00000000 00000001 00000000 00000000 00000000 01000000 00000100 01000000 00000000 00000000 00000000 00000000 00000000 01000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 01111000 00010001 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 01000000 00000000 00111000 00000000 00001001 00000000 01000000 00000000 00011110 00000000 00011011 00000000

This is the first 64 bytes of data from the compiled “Hello world” program file (the full program file is 8.32 kilobytes). Each binary digit represents one bit of data: a “1” represents an “on” state of an electronic circuit, while a “0” represents an “off” state. Computers and video game consoles are electronic machines containing billions of transistors and complex arrays of circuits, and a program or game is a set of instructions that direct the machine to perform certain calculations or manipulations on a given set of data. For the computer to manipulate data, that data must be in binary format. Anything that is stored on or manipulated by a computer—photos and images, sound and music, documents and games—must be interpreted from a human-readable form for the computer to work with it, and then must be interpreted back into a human-readable form for us to understand it. But programs, apps and games also require interpretation from a human-readable plan or ideation into a programming language (an engineering process) before being interpreted into binary format for the computer to use. Compare this with shooting a film, where (computerized assistance aside) a scene is created and then captured on film as a series of images; or to writing a book, where (again, computerized assistance aside) the words scrawled on a sheet of paper are set into type and are never ostensibly not words. These art forms use engineering processes to capture, manipulate and preserve human-readable elements, but a computer program or video game is engineered in such a way that it is, at several stages, unreadable to most humans.

This is the nature of our medium: at its core, a video game is decidedly abstract. It is, at its most basic level, a mathematical construct, an intensely imaginative piece of engineering. Inasmuch as we interact with a narrative, view images and are affected by a soundtrack, we interact with series upon series of mathematical calculations and algorithms of which we are blissfully unaware. The ever more film-like scenes, the epic soundtracks, the carefully-planned audioscapes, the gut-wrenching stories, are all added purposefully and artfully to a foundation of electronic engineering. And they are entirely optional. The creativity behind a video game starts long before any aesthetic elements are put in place, allowing extraordinary variety in aesthetic, gameplay, narrative, musical, and sonic formulations. For example, Pong didn’t necessarily need a graphical interface to be a reaction-based game. Adventure didn’t need to be humorous to be a caving simulator. Quake didn’t need a Nine Inch Nails soundtrack to be visceral (I should know, I only ever played the music-deprived shareware version when I was growing up). Portal didn’t need the promise of cake to make us play. Left 4 Dead didn’t need to aim for a filmic aesthetic in order to be a survival horror game. But these elements were included in order to shape the game into the developers’ vision and to provide an enjoyable, relatable experience to the player. It is through these included elements that we can draw parallels to other media, whether film, theatre, books or even games of another genre; but they are all additions to the game’s electronic foundation, and are there not by necessity but by conscious choice. If we study the filmic in video games, we have to start by understanding that nothing filmic winds up in a video game until someone chooses to put it there.

Video games are not films made interactive; they are computer programs made beautiful. Their aesthetic elements are pinned to abstract constructs that, from their very invention, have been more receptive to the products of imagination than to the limitations of the physical world. This is why we can see in them an inexhaustible potential for creativity, engagement, innovation and wit; why their storytelling potential is beginning to defy the pessimistic predictions of even just a few years ago; why their music can be filmic or procedural, or both, or neither, and draw us into an imagined space for hours on end. As ludomusicologists, we study just one element of this abstract medium, so we already know to seek out the parallels between music and the other elements of a game. There is no reason why we should stop there, nor why we should be content to rest on parallels to film music (strong as they are) when developers of games have no such restrictions. Instead, we should relish the opportunity to follow this technical, artistic and novel art form wherever it leads. Moments of revelation still await us, and our studies will continue to provide glimpses of innumerable possibilities as we become more acquainted with this new art.

Kyle Edward Roderick (Texas Christian University, Editor-in-chief of Musicology Memes)

It seems to me that all of art, at its core, is a link between the imaginations of the author and the reader. With modern film making techniques, virtually any set of images and sounds, which I refer to as the “audiovisual landscape,” can be presented to the reader. The film, therefore, represents the epitome of authorial authoritarianism. The intent of the film is to invade the imagination of the viewer and immerse them in its rigid narrative.

The modern video game finds itself in a similar situation. With increasing levels of “realism” in the visuals, and functional equivalency of the sounds, the audiovisual palette available to the game designer is, for all intents and purposes, the same as that of the film director. The notion of interactivity is of course that which distinguishes games from films. At the most basic level, the player takes the reins as the protagonist, moving the narrative along at his or her own pace. At the highest level are those games where the designer hands over as much authorial power over to the player as possible.

In the purest form of these so-called “sandbox” games, the narrative is whatever the player makes it. In a sense, there is no story, but that which exists in the imagination of the player. But the audiovisual landscape remains under the control of the designer. Minecraft, for example, forces the player to see the world through “blocky” goggles. The player is not so free as to see the world as made of pyramids rather than cubes.

So if film is high narrative and high audiovision, and video games are medium-to-no narrative and high audiovision, literature is, in a sense, the opposite of video games, that is, high narrative and medium-to-no audiovision.

By high and low here I mean influence of the author. In sandbox games, there is still narrative, it is simply supplied by the player. Likewise in literature, there is still audiovision, it is simply found in the imagination of the reader. The greatest influence an author can have over the audiovisual landscape of their medium is found in picture books and in avant-garde literature such as Mark Z. Danielewski’s House of Leaves.

Some sandbox game designers have expressed their desire to loosen their grip on the audiovisual landscapes of their games. In the case of Will Wright’s Spore, the player is given vast license to design many aspects of the audiovisual landscape themselves, such as the appearance and sound of their aliens, and their civilization’s buildings, vehicles, and clothing. Ironically, in the case of Spore many players found the rigid transitions between levels of play to be incongruous with the purported sandbox nature of play. While players looked forward to a game with medium audivision and medium narrative, they actually found a game with medium audiovision (they were on the whole happy with the design aspect) but with high narrative. Every species created in Spore followed the same path, whether peaceful or warmongering, from bacteria to galactic domination.

Games with high narrative that wish to appeal to the sandbox audience often throw in these false dichotomies, moral dilemmas that hardly affect the overall narrative of the game in order to afford an illusion of control. As a friend of mine put it, in many games you have the choice of either being a “saint [or an] asshole.”

An interesting case is Linden Lab’s Second Life where the “player” (if you could call him that) can not only write their own narrative and design game mechanics, but also become the painter of the audiovisual landscape by using in-game tools and by uploading images, 3D models, and audio files. Where Second Life is lacking, is that the author is divorced from the medium, and the virtual world can hardly be called a game any longer, as there is no link between an author and a reader. From this emerges many games within a virtual world, where users of Second Life are often both designers and players of games not written by the developer of the virtual world.

As this is a Ludomusicology blog, I feel compelled to mention how this all relates specifically to music in video games. When games have low narrative, that is, narrative in the hands of the player, music becomes problematic for the same reason that many other aspects of the game become problematic, and that is because the game cannot predict what the player will do. This is, perhaps, why in many games with narrative freedom, there is also musical freedom, e.g. the Fallout and GTA series. Other attempts have been made at accommodating this issue, such as the ambient scoring of Minecraft.