Creativity in abstraction: beyond the film/game parallel

Iain Hart (Sydney Conservatorium of Music, Eine Kleine Pwnmusik)

C.S. Lewis wrote, in his science fiction novel Out of the Silent Planet,

“To every man, in his acquaintance with a new art, there comes a moment when that which before was meaningless first lifts, as it were, one corner of the curtain that hides its mystery, and reveals, in a burst of delight which later and fuller understanding can hardly ever equal, one glimpse of the indefinite possibilities within.”

I had this moment when I was sitting in a film music lecture and realised that the music of the video game Left 4 Dead used similar musical devices to the music of the film Psycho, and used them for the same purposes. It was a first glimpse, not of the fact that video game music was emotive (I had felt that already), but that it was deliberately expressive and powerfully designed. I’ll never forget that moment. I can’t. It’s permanently changed the way I think about game music, so as a student of game music it’s always before me. And with it, the notion that game music and film music are kindred spirits, doing the same things in the same ways.

But as I’ve studied the parallel between film and game music, the less satisfying I have found the parallel to be, and the more I have realised that the differences are as myriad as the similarities. Yes, game music does many of the same things as film music, but it also does many more (chief among them: respond to the player/viewer). And there are things film music can do that game music can’t, or that game music is only just beginning to do. Likewise, although game music and film music sometimes have the same function (space-filling, for instance), they often achieve this in astoundingly different ways. No film I’m aware of selects background music at random while you’re watching it, but this is a standard—even simple, suboptimal, obsolescent—method for keeping game music interesting. And the language used to talk about game music is subject to the same difficulties. Film music terminology can accurately describe many aspects of game music (diegesis, motifs, etc.), but it struggles to describe the effects of something as utterly crucial to gameplay as interactivity. The film/game parallel only gets us so far.

We know so much about music in films, and a lot of that knowledge informs our understandings of the music in other media, and even of music itself. It’s only right that where a parallel exists, ludomusicologists should take advantage of it and learn what we can. But that should never prevent us looking for other parallels. For instance, I have found Markku Eskelinen and Ragnhild Tronstad’s chapter “Video Games and Configurative Performances” (from The Video Game Theory Reader, Wolf & Perron, 2003, pp. 195-220) helpful for understanding the player’s role in the game because of their comparisons between games and configurative theatre. I am not a scholar of theatre, but the comparison to a performance artwork helped me reframe the argument in light of my experience with music—and, ultimately, prompted me to start making comparisons with other non-filmic media.

In some ways, music is the most artistic thing I know anything about. In school I was far more interested in scientific and technological pursuits than art, at least from a vocational point of view. I was an advanced computer user before I was an advanced computer player (and I’m still not sure I’d call myself that). I know my way around a computer quite well, and I know a bit about how its hardware and software work, and work together. This doesn’t hinder my research of music in video games; rather, this helps me understand the fundamental elements of the medium, and helps me understand why parallels to other media can be drawn to it at all.

Always, at the back of my mind, is the understanding that a video game is a computer program. A computer program looks a bit like this:

#include <stdio.h>
int main(void)
printf("Hello world\n");
#include <stdio.h>

int main(void)
printf("Hello world\n");

This is a very simple program written in the C programming language. Its sole function is to output the text “Hello world” on a command line (the first steps of learning a programming language often involve displaying the text “Hello world” somewhere). It’s functionally trivial, but it can teach a student about the basic structure of a program written in the C programming language and get them to understand how the computer can be controlled by their instructions. However, for the computer to run the program it has to be compiled, after which it looks like this:

01111111 01000101 01001100 01000110 00000010 00000001 00000001 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000010 00000000 00111110 00000000 00000001 00000000 00000000 00000000 01000000 00000100 01000000 00000000 00000000 00000000 00000000 00000000 01000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 01111000 00010001 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 01000000 00000000 00111000 00000000 00001001 00000000 01000000 00000000 00011110 00000000 00011011 00000000

This is the first 64 bytes of data from the compiled “Hello world” program file (the full program file is 8.32 kilobytes). Each binary digit represents one bit of data: a “1” represents an “on” state of an electronic circuit, while a “0” represents an “off” state. Computers and video game consoles are electronic machines containing billions of transistors and complex arrays of circuits, and a program or game is a set of instructions that direct the machine to perform certain calculations or manipulations on a given set of data. For the computer to manipulate data, that data must be in binary format. Anything that is stored on or manipulated by a computer—photos and images, sound and music, documents and games—must be interpreted from a human-readable form for the computer to work with it, and then must be interpreted back into a human-readable form for us to understand it. But programs, apps and games also require interpretation from a human-readable plan or ideation into a programming language (an engineering process) before being interpreted into binary format for the computer to use. Compare this with shooting a film, where (computerized assistance aside) a scene is created and then captured on film as a series of images; or to writing a book, where (again, computerized assistance aside) the words scrawled on a sheet of paper are set into type and are never ostensibly not words. These art forms use engineering processes to capture, manipulate and preserve human-readable elements, but a computer program or video game is engineered in such a way that it is, at several stages, unreadable to most humans.

This is the nature of our medium: at its core, a video game is decidedly abstract. It is, at its most basic level, a mathematical construct, an intensely imaginative piece of engineering. Inasmuch as we interact with a narrative, view images and are affected by a soundtrack, we interact with series upon series of mathematical calculations and algorithms of which we are blissfully unaware. The ever more film-like scenes, the epic soundtracks, the carefully-planned audioscapes, the gut-wrenching stories, are all added purposefully and artfully to a foundation of electronic engineering. And they are entirely optional. The creativity behind a video game starts long before any aesthetic elements are put in place, allowing extraordinary variety in aesthetic, gameplay, narrative, musical, and sonic formulations. For example, Pong didn’t necessarily need a graphical interface to be a reaction-based game. Adventure didn’t need to be humorous to be a caving simulator. Quake didn’t need a Nine Inch Nails soundtrack to be visceral (I should know, I only ever played the music-deprived shareware version when I was growing up). Portal didn’t need the promise of cake to make us play. Left 4 Dead didn’t need to aim for a filmic aesthetic in order to be a survival horror game. But these elements were included in order to shape the game into the developers’ vision and to provide an enjoyable, relatable experience to the player. It is through these included elements that we can draw parallels to other media, whether film, theatre, books or even games of another genre; but they are all additions to the game’s electronic foundation, and are there not by necessity but by conscious choice. If we study the filmic in video games, we have to start by understanding that nothing filmic winds up in a video game until someone chooses to put it there.

Video games are not films made interactive; they are computer programs made beautiful. Their aesthetic elements are pinned to abstract constructs that, from their very invention, have been more receptive to the products of imagination than to the limitations of the physical world. This is why we can see in them an inexhaustible potential for creativity, engagement, innovation and wit; why their storytelling potential is beginning to defy the pessimistic predictions of even just a few years ago; why their music can be filmic or procedural, or both, or neither, and draw us into an imagined space for hours on end. As ludomusicologists, we study just one element of this abstract medium, so we already know to seek out the parallels between music and the other elements of a game. There is no reason why we should stop there, nor why we should be content to rest on parallels to film music (strong as they are) when developers of games have no such restrictions. Instead, we should relish the opportunity to follow this technical, artistic and novel art form wherever it leads. Moments of revelation still await us, and our studies will continue to provide glimpses of innumerable possibilities as we become more acquainted with this new art.

5 comments for “Creativity in abstraction: beyond the film/game parallel

  1. December 6, 2013 at 9:28 am

    Dear Iain,
    Thank you for pointing me to all these interesting examples of self-referential media products. Your argument about the necessity of a certain matureness for this to happen is certainly valid and very convincing. I think that you are also right when you mention this “dialogue about the character of the medium through the medium itself”. I would even say that Evoland could be an example of this dialogue taking place on the level of the algorithm: the structure of the character’s interaction with its environment and other characters can surely be expressed with algorithms (in fact, this is of course how the game is programmed), and as the visual style is the only thing that seems to change in this game, it could well be that some of the underlying algorithms stay the same, thus constituting a meta-message from the game’s creators.

    • December 12, 2013 at 2:31 am

      Dear Nikita,
      I agree, a changing visual style alongside a maintained algorithmic framework would seem to indicate a meta-message, drawing attention to those algorithms. Of course, the same can be achieved by modifying the algorithms while other elements stay the same. “The Stanley Parable”, for instance, actually has examples of both methods: plenty of changing gameplay rules, but also changing visual styles (at one point at least). A very engaging dialogue about the nature of gameplay and the agency of the player.

  2. December 5, 2013 at 1:57 am

    Dear Nikita,
    Thank you for your comments! I haven’t read “Computer Spiel Welten” but I’ll see if I can get my hands on a copy, it sounds interesting.
    I definitely agree with you about the importance of narrative and emotive elements in connecting the algorithms and the player, and ultimately making play meaningful and attractive. That is, after all, what we engage with and enjoy. However, I think the mathematical constructs and algorithms underpinning video games provide a different set of possibilities for creating meaning and engagement than is found in other media (such as film). With a fuller understanding of the technical possibilities of the medium—and, importantly, an understanding of that medium’s distinctness from other media—I think we’re more likely to make accurate observations about what, why and how emotions are evoked. And I don’t think technical factors are the only factors influencing potential meanings and emotions, either; I’m certain there are many books’ worth of analysis to be done of the influence on programmer/hacker culture (and particularly its unique sense of humour and cultural commentary) on video games and their elements.
    On your point about media hiding their inner workings, yes, I can definitely see that! I wonder, though, whether a medium reaches a certain stage of maturity where it can reference, challenge or even feature its inner workings. A simple example might be the film “Hugo”, but the adventurousness of avant garde film is more what I’m thinking of. And I’d say video games many examples of this: “Evoland” charts the history of gaming (and, to some extent, gaming technology); in “Portal 2” GLaDOS makes fun of the player/character’s technological inability to speak; “FEZ” is a platformer where the player/character discovers the third dimension; and the distribution of the ‘Konami Code’ cheat in games of all kinds and from various developers, which is a sort of ludic ‘Wilhelm scream’. I take this as a sign that a medium’s creators are engaging players in a dialogue about the character of the medium through the medium itself, and concurrently to the primary messages of the texts—perhaps not on the level of mathematical algorithms, but that would certainly not be impossible. To me, this indicates strong (though not necessarily broad) cultural success. I’d be interested to hear your thoughts!

  3. December 4, 2013 at 9:53 am

    Dear Iain, thank you for drawing attention to the fundamental importance of understanding the technical workings of computer programs like the video games you mentioned. Your argument about what I would call structural parallels between adventure games and a caving simulator is also important in understanding the character of this medium. (Did you read the book “Computer Spiel Welten” by the German theorist Claus Pias which describes several of such parallels?). However, I do not think that the mathematical constructs and algorithms are what makes the game meaningful or attractive for the player. Media tend to hide their inner workings and there is a theory that the better these mechanisms are hidden, the more successful the medium gets. The narratives and anything that evokes emotion is in my opinion what really plays the pivotal role of connecting the algorithms and the player.

    Nikita Braguinski
    Humbold University Berlin, Department for Media Theory

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: