This episode was written & produced by James Introcaso.
Video games are a growing industry and every play-controlled experience is defined by its harmony of music, sound effects, and voice acting. In this episode, we reveal how these elements of a video game's soundscape are crafted and come together to tell an interactive story. The most sophisticated sound design in video games allows those without the ability to see a chance to engage with some of our greatest modern entertainment. Featuring Microsoft Sound Designer, Zachary Quarles, and ArenaNet Technical Sound Designer, Damian Kastbauer.
MUSIC IN THIS EPISODE
Yearn - Chris Coleman
Cotton Float - Luke Atencio
Seafoam29 - Tangerine
Change the Game - AJ Hochhalter
White11 - Tangerine
Clear Glass - Steven Gutheinz
Though Clouds - Steven Gutheinz
20K is hosted by Dallas Taylor and made out of the studios of Defacto Sound.
Follow the show on Twitter & Facebook. Our website is 20k.org.
Consider supporting the show at donate.20k.org.
To get your 20K referral link and earn rewards, visit 20k.org/refer.
Check out Bose at bose.com/20k.
Follow Bose on Twitter & Facebook.
View Transcript ▶︎
[video Game Montage]
You’re listening to Twenty Thousand Hertz, the stories behind the world’s most recognizable and interesting sounds. I’m Dallas Taylor. This is the story of how video game sound designers create new worlds, tell stories, and bring imaginary characters to life.
[continue video game montage]
As the first generations raised with video games, a lot of us Gen Xers and Millennials continue to play as adults. In fact, more people than ever are playing, even if they don’t consider themselves a “gamer”. With the rise of mobile and social gaming, last year the industry took in over 30 billion dollars in the United States alone. That’s more than movies and music - combined.
Games are often defined by their visuals. Back in the 80’s it was all some commercials could talk about.
[80's Nintendo commerical]
So, if games are such a visual-focused medium… why is sound important?
Damien: Sound in games really comes down to communicating a sense of place and a sense of emotion.
That’s Damien Kastbauer, a leading technical sound designer.
Damien: It is an unseen art. As someone who plays games, I know when it's not right and I understand how that can blow the whole mood. Sound is about engagement and communicating this game's intention.
In other words, sound in games is all about immersion. The right sounds are critical for the player in order to authentically slip into a game’s story and become the character they control. Consider what Mario’s cultural impact would be if instead of this [Mario intro SFX], it sounded like this *[crappy, rush job in the same 8-bit style]. *Giving sound designers and composers the appropriate time, resources, and direction can take a game from passable to iconic.
[Mario theme music]
Fast forward 30 years to where we are today. It’s unbelievable the level of engagement sound brings to games. Take, for example, Bethesda Softworks’ smash-hit Fallout 4. Even if you’re not a gamer, you can appreciate the level of detail that goes into the sound. Let’s deconstruct the soundtrack and explain these audio layers. First up, the ambience, which are sometimes called the backgrounds.
[Fallout Ambient Noise]
Ambience can be anything that ties the player to the location. These are typically sounds like rain, wind, forest rustling, room tone, or anything that establishes the environment.
Another layer is foley. These are sounds that come from the character’s body, hands, feet, and clothing movement.
[Fallout Foley]
Hard effects - like machines, sirens, explosions, and other objects not attached to the character - make up another layer.
[Fallout Weapons, explosions]
And those are all just in the Sound Effects layer. You also have the voice layer...
[Array of Fallout Dialogue]
This can be anything from narration, to character dialogue, to coms and radios.
And finally… the music layer...
[Add Fallout Music]
The music layer alone might have multiple sublayers like drums, high pitched instruments, low pitched instruments, and strings.
In this respect, the audio layers in games are a lot like the audio layers in movies, however, with games, everything is variable based off of the player, so instead of mixing all of the sounds together into a single soundtrack like what would be done in a movie, you have to program thousands, sometimes tens of thousands of individual sounds that all have to work together in real time.
Damien: In a movie, As the person stepped across the room [footsteps SFX], you would place a footstep sound at each location in time. You would know every time that that scene played back [rewind SFX] that the footstep [footsteps SFX] would always be in the same place because as a viewer you don't have control over how fast that character is walking.
Games have a infinite possibility of playback. One time you could walk over to the kitchen [walking SFX], pick up a cup of coffee [coffee cup SFX] , take a drink [drinking coffee SFX] and step out the back door [door open SFX] and hear the crickets chirping [crickets SFX] in the distance. Another time in the same game, you could instead choose to sit down in a chair [sitting SFX] and listen to the rumble of the refrigerator [refrigerator SFX].
In order to achieve these infinite possibilities in playback, every game is built on top of what is called, and engine. This engine is the software skeleton that controls the physics and mechanics of a game. Sound designers load in all of the sound effects, voice, and music into this engine, then have to fine tune the programming parameters so that all of these sounds can smoothly playback together at any time and at any place in the game.
Damien: Yeah, it's a lot of math. Every sound in a space has to reflect reality as closely as possible, where it's at, what it's next to, how loud it is, these are all things that very simply add up to communicate a sense of reality for the player.
But how do those sounds get made? What’s the process for creating sounds that are intended to sound real, like the sound of a car in Need for Speed.
[Need for Speed clip]
And imaginary, like Pikachu in Super Smash Brothers.
[Pikachu clip]
And who gets to make these sound effects?
Zach: I feel most comfortable and most productive whenever I'm creating worlds. Whether that be a realistic sound design or completely stylized, as long as it's all rooted in that game's reality.
That’s Zachary Quarles. He’s an Audio Director and Sound Designer for Microsoft Game Studios.
Zach has a ton of experience in making sounds for games. He currently works on Killer Instinct, Microsoft’s big fighting game [Killer Instinct clip], but in the past he’s made sounds for Quake 4... [Quake 4 clip]
X-Men Legends... [X-Men Legends clip]
and ReCore [ReCore clip] .
For Zach and for sound designers everywhere, adding sound effects to a game is more than simply finding the most realistic sound and plugging it in. It’s about Crafting the right effect that gets the player invested in the story and the moment. That makes them feel something… even if that something is super gross.... Take this scene from another game Zach worked on called D4.
Zach: One of the sequences in D4 had to do with two characters eating different meals together and the meals are really weird. So there's a guy that has a stack of pieces of pizza that he just kind of smashes together and just kind of... piles them into his mouth [clip]. A guy rips open a lobster shell and sucks out all of the meat [clip].
I was taking stuff like crab shells and ripping those apart [shell ripping SFX], dumping cream of mushroom soup [thick soup pouring SFX] on top of that. I had a glass, like just a drinking glass, filled with raw chicken and I was just punching [punching SFX] inside of the glass, just making slucking gross splattery sounds [splatter SFX]. It was all for these meals that these characters were eating.
[D4 clip]
The foley room when I was recording this smelled so bad. We started to have people just show up in the engineering room as I was recording all this, and they're all holding their nose and just like shaking their head and cursing my name.
There was a moment I was surrounded by crab shell that was absolutely reeking with a bucket of cream of mushroom soup with my hand in a glass covered in raw chicken and I was like, I love my job.
And if you think Zach’s job sounds like fun, wait’ll you hear what else he does.
Zach: I'm actually several voices in Killer Instinct.
Not only does Zach make sound effects, but he’s also a voice actor and director. It’s up to him to bring the characters of a game to life… sometimes with a little help from man’s best friend.
Zach: Eyedol, who is this big ogre creature who has a head that's split in half and each head is a different personality. All the voice for him is designed by me, it's my voice mixed with my dogs growls.
Eyedol’s thing is: "I serve no one!"
I would take that recording and morph it with my dogs growls and a couple of other things to give it a lot more body, a lot more presence and this is what it sounds like.
[Eyedol clip]
When it comes to voices, it’s about making every character come to life. When Zach created Eyedol’s voice, he was thinking about more than just making a monster with a split head who is scary.
Zach: Before I really get started I like to see how the player moves and how the player carries their weight so I can get a sense of scale and sense of kind of, distribution of energy throughout. Where they hold their energy and everything. So Idol was hunched over, being held together by dark magic, so he wasn't very solid in terms of vocal cord structure.
That’s how sound effects and voices come together to immerse a player in a world and give them a sense of the characters, but what about music? How do games like Skyrim, that have over 200 hours of content, keep their music exciting and fresh?
And also - can a video game’s sound be so precise that people who are blind can play them? We’ll get to that after the break.
[music out]
MIDROLL
[music in]
We’ve heard how sound effects and voice come together in a game, but what about music? Video games have come a long way since the catchy tunes of Megaman.
[Megaman music]
We now have unbelievable orchestral scores, like those found in Blizzard’s Overwatch.
[Overwatch score]
Now that games have the ability to playback the highest level of orchestral recordings, the challenge becomes, how how do we keep that music fresh? Listening to a song over and over again becomes annoying over time, so how do game composers keep their music from becoming noticeably repetitive? Here’s Damien again.
Damien: Music in modern games has gained a complexity because of how variable the player's interaction can be. Music is composed in elements or little chunks or tiny little pieces or layers… that the game then controls the sequence of or the playback of based on what's happening. You almost end up with a tiny little composer in the box who is telling the violins [music starts] to start when the hero walks through the doorway or a tiny little composer that signals the trumpet fanfare [SFX - trumpet] as soon as the dragon burst out of the cave.
[music swells with dragon reveal]
Music for movies or albums can be written with full freedom, because every time it plays back in the future, it will be exactly the same. However, music for games have to be written in little pieces, modularly… and all of these little modules have to be able to work together. The most basic modular form of writing game music would be to create a music start [music start], a loopable main theme [music loop], and an ending [music ending]. Games that are longer and more complex might have 30 or 40 modules split up by themes and instrumentation. This is essentially how a 10 minute piece of linear music can transform into a 2 hour symphony during a game-play session.
Damien: It's this tiny virtual composer then that creates this musical soundscape out of these components and pieces of music that have been created for the game and really scores the experience for the player based on what's happening at that time. As a composer, you want to make sure that the music you're writing doesn't get old, doesn't get boring. You want to appropriately score that experience for the player, no matter how long it takes them to get through the forest. Then, when they get to that dragon, signal that in a way that brings the dramatic elements to the experience.
So music creates the atmosphere, sound effects create the world, voices create the characters, and together all three bring emotion to the video game’s story. Though it’s often overlooked in favor of graphics, sound is a key component to any game and has been since they first hit the scene. And for some players, sound is the only component. Here’s Zach again.
Zach: Killer Instinct is a game that is as much as it can be a controlled environment. It's not like an open world game that you're running anywhere and everywhere. You're on a 2-D plane with two people fighting each other.
A Blind Gamer by the name of Sightless Combat sent me a message on Twitter and was like, "Hey,[tweet SFX] I just wanted to let you know that me and several of the Blind Gamers that we play with, we really love Killer Instinct. It really gives us an understanding of what's going on on screen at any given type without seeing it, but we have some feedback for you."
Zach read their feedback… and put it in Killer Instinct’s next update.
Zach: A lot of it was mix changes of, "We need to be able to hear the players after they jump when they land a lot more cleanly." I was like, "Okay. That's something that's pretty easy to fix."
[Killer instinct jump clip]
When a new release would happen, I'd send him notes of things that I fixed or tried to fix and needed some feedback from him and his crew. He would play through it and he would shoot me some feedback, shoot me some additional feedback for the new character or for anything else that they came across. What they usually do is turn off music, and they just have sound design playing, so they can tell where they are in the play-space at any given time.
[Killer Instincts clip]
Then it was stuff like, "I feel like the looping sound on this projectile gets cut off too quickly. I can't tell where it is in the play space." It's like, "Okay. I will take a look at that." [Killer Instrincts clip] So, we get very, very minute with it. It's been really, really cool being able to just pick my own work apart and strip it down to brass tacks as much as I possibly can to say, "What's really important here?"
The rest of the stuff is color and it's awesome, but it's not nearly as important as these things, so I start bucketing things into accessibility versus flavor. I always make sure that accessibility bucket is very, very laser sharp in terms of player feedback and content.
Games are still continuing to evolve. In only a few decades, sound design has become so advanced that there’s a community of sightless gamers who can share in the experience. Sounds ability to draw everyone into a playable story is only going to increase. What’s in store for the future of sound in games. How will sound be used to suspend disbelief in Virtual Reality? ...and what sounds will kids now associate with their childhood?
For me back in the 80’s, it was the sound of Zelda [Zelda clip], Metroid [Metroid clip], and that super annoying, smug dog in Duck Hunt [Duck Hunt clip]. For kids today, maybe the sound a catapulting bird in Angry Birds [Angry Birds clip] or the sound of Minecraft [Minecraft clip], will be what triggers their nostalgia. No matter what it is, we can be certain that sound will continue to tap into the deepest parts of our emotions.
Twenty Thousand Hertz is produced out of the studios of Defacto Sound, a sound design team that supports ad agencies, filmmakers, and video game developers. Checkout recent work at defacto sound dot com.
This episode was written and produced by James Introcaso… and me, Dallas Taylor. With help from Sam Schneble. It was edited, sound designed and mixed by Nick Spradlin.
Thanks to our guests Damien Kastbauer and Zachary Quarles for sharing their stories about how they use sound to build worlds. We couldn’t have crafted this episode without them.
All of the music in this episode is provided by our friends at Musicbed. You can listen to these tracks, including this one, “Through Clouds” by Steven Gutheinz on our exclusive playlist which you can find at music dot 20k dot org.
You can say hello, submit a show idea, or give general feedback through Facebook, Twitter, or over email. Our email address is hi at 20k dot org.
The most important thing you can do for us is recommend the show to a friend. Say it in person, send a text, or shout us out on social media. Word of mouth is critical for our podcast survival.
As always, thanks for listening.
