This episode originally aired on Sum of All Parts. Go subscribe!
Brant Guichard has heard "The Music" for as long as he can remember. Brant has a particular type of epilepsy where he hears what are called "musical auras" whenever he has a seizure. Brian Foo, aka the Data Driven DJ, introduces a different musical element to Brant's experience of seizure.
Twenty Thousand Hertz is produced out of the studios of Defacto Sound and hosted by Dallas Taylor.
Follow the show on Twitter & Facebook. Our website is 20k.org.
Get a 4 week trial plus a digital scale at Stamps.com. Just click on the Radio Microphone at the top of the homepage and type in "20K".
Consider supporting the show at donate.20k.org.
To get your 20K referral link and earn rewards, visit 20k.org/refer.
View Transcript ▶︎
You’re listening to Twenty Thousand Hertz, the stories behind the world’s most recognizable and interesting sounds. I’m Dallas Taylor.
[Play EEG sonification]
What you’re hearing right now are brain waves. Sort of. This is from an EEG machine. An EEG machine is used to record the electrical activity of the brain. Plug that data into a synthesizer, and you get something like what you’re hearing. It’s not exactly pretty. Actually it’s kind of spooky. But what did you expect to hear inside someone’s head?
Well, if the head we’re talking about is Brant Guichard’s, the answer would be…music.
Brant: “What I call The Music”.
Recently Joel Werner, the host of the podcast Sum of All Parts, talked to Brant to find out what exactly is going on inside his head. I’ll let Joel explain…
Joel: Brant Guichard has heard The Music for as long as he can remember...
Brant: The earliest memory I have was sitting on a bed in my room and enjoying a run of music as it went by me, waving my hands as it just run through. And I didn't recognise it for what it as at the time. I listened to it, it finished, and I asked my friend who was sitting next to me watching the television and feeling bored, since we had absolutely nothing to do, and I asked him, "Did you hear that?" And he said, "What?" I asked him, "Music?" And he said, "What music?" It made me realize I was the only one who was hearing that. Nobody else was hearing this, this was all mine.
Joel: That was Brant’s first encounter with what he calls The Music - and in the thirty years since, it’s something that he’s heard multiple times every day.
Brant: The music starts by warping the sounds and things I hear. Then it adds its own rhythm and starts becoming stronger within my head. The pattern is never the same, it is never the same. It is always unique. Every time. A collection of repetitive noises together, warping together the noises around me into a rhythm, often taking any song I've heard and putting that into the mix. Anything I reach for in my memory, that will be placed into the mix as well. All the sounds, even speaking is part of the music. It's why I become absolutely still sometimes, it's because I don't want to make noise myself. I stop any noise I'm hearing if I can and I stop moving myself because that has the best chance of slowing it down a little bit.
It's partially in my control in that I don't have any control that it's going to run, so I reach and try to control where it goes. It's like sitting in a car without having any brakes and having the accelerator tied down, but you've got the hands on the steering wheel. You can control where it goes but you can't stop it.
Then I start developing a partial seizure with having part of my body losing control. And after this I will develop into a full seizure, but I will stay fully conscious at this point, although it will not look like I am. I will be on the ground with a grand mal, as most people think epilepsy is. But after this point it will continuing developing, and past that point I will lose consciousness.
Joel: Brant has epilepsy - and it’s a particular type of epilepsy where he hears what are called musical auras.
Ingrid: So, when Brant has that music that he hears, that's actually the beginning of a seizure. It's a small seizure, as he told you, but if it doesn't progress to involve more of his brain, he remains aware and there's nothing to see. Only Brant can tell us about it.
Joel: Professor Ingrid Scheffer is Brant’s neurologist, and a world leading epilepsy researcher.
Ingrid: We don't really understand why one seizure progresses and another doesn't, except that we do know that almost everyone with epilepsy is more likely to have seizures if they're tired or stressed. And so, you might have some auras, but then they might progress if you're more tired or stressed. Or sometimes people will build up. They'll have a run of auras, which sort of heralds the fact that they're going to go into the biggest seizure.
Joel: There are a few different types of epilepsy that are related to sound. Like musicogenic epilepsy...
Ingrid: Where music may trigger a seizure.
Joel: Or reflex epilepsy..
Ingrid: Where a very loud noise may trigger a seizure. There's a young lady I look after who, if there's a loud noise as she's walking along the street, will suddenly have a tonic seizure and fall to the ground. She actually has to wear headphones all the time to try and dull down the sounds around her so she doesn't get a surprise.
Joel: But, musical auras, like Brant experiences, are unusual. Like, really unusual.
Ingrid: Gosh, I think I have seen one or two, but it's rare. I see lots and lots of people with focal epilepsy and many have auras, but hearing music is rare.
Joel: Do you remember when you first met Brant? Can you take us back to that moment?
Ingrid: Yes, I can remember when I met Brant. He was 18 years old at the time and he came along with his father, and he told me the story of his epilepsy. His epilepsy had begun quite early in life with some convulsive seizures as an infant, and these had occurred every year or so. Then from about the age of eight he developed awake seizures, and these would be preceded by an aura. Brant described an aura of music where he experienced what he called twisted sounds. These were initially pleasant, but by the time he was 11 years old a couple of years later, the sounds hurt and he was scared.
Brant: Strangely enough, when I was very young, it felt good to me. It was very enjoyable and it was something I liked a lot. I was one of those people that, at puberty, my epilepsy developed quite intensely. I started having proper fits and at this point, what I call the music, that started to become something I had perhaps five to twenty times a day and became extremely intense and started to scare me. I don't understand why but the auras, they became very strong and brought on fear to me at that point. Absolute, intense fear that left me a few years later. The fear was not there anymore. And I don't know why that fear occurred at that point.
Coming up, we’ll meet the data visualization artist who took Brant’s seizures and did something…kind of beautiful with them. That’s after the break.
[MIDROLL]
Brant Guichard has been hearing music in his head for thirty years. And, in a way, for the past ten minutes, you’ve been hearing it too…
The music you’ve heard so far in this episode.. That music is intrinsically connected to Brant’s epilepsy. In fact, it is Brant’s epilepsy.
Brian: My name is Brian Foo. I'm a data visualisation artist at the Museum of Natural History in New York City.
Joel: By day, Brian works on data visualisations for the museum, but by night he’s the Data Driven DJ.
Brian: So the data driven DJ project is kind of an experimentation on different ways of expressing data as music.
Joel: Part science, part art form - this is data sonification; the process of representing numbers in sound.
Brian: So a lot of that is thinking about what are the strengths of music. Or what are the strengths of sound, as compared to more visual media like charts or graphs. So it's kind of using the fact that music is more felt, and you can kind of perceive things like change and time more intuitively.
Joel: The clicking of a Geiger counter, where faster clicks indicate higher radiation levels - this is one of the earliest and most practical examples of data sonification. Brian’s work on the other hand, is much more song-like..
Brian: Initially I was very interested in learning how to make music. You know I had a particular skill set, which was computer science. I wanted to figure out a way in which I could learn music. I did some research into data sonification. I wasn't very satisfied with the current state of data sonification. I think a lot of times it's almost like listening to a chart. The question I always had is why make it into sound, if it's already fine as a chart. I kind of used that as the challenge for this project, to make kind of meaningful data music essentially.
Joel: Brian’s take on songwriting is a process called algorithmic composition. He comes up with a bunch of rules, or algorithms, that tell the music what to do in response to a change in the data. The guiding principle of this approach is something Brian calls “uncreative creativity”.
Brian: Yes, so when I say uncreative creativity, when you think about traditional creativity, it's that's artist who is just staring at a canvas and having a direct translation of my emotions or thoughts into the medium. But because my medium is code, when I press play, that's essentially the first time I'm hearing the music. I'm hearing the song. Usually the dataset is so complex, and the algorithm and the rules, there's so many of them, there's so many different variables that it's really hard for my brain to kind of generate that. That's kind of mostly because I didn't really have a music background, it was hard for me to imagine what the music would sound like when I kind of applied this algorithm to the data, to generate the song. It's almost like I'm just designing the rules in which this song plays out, which is not a traditional way you would think about creativity. But that is where the creative act is, is designing those rules. It's designing how you map the data to sound.
Joel: But mapping the data to sound isn’t something that happens quickly - or easily.
Brian: It's very much an iterative approach where I have to constantly tweak the algorithm, because usually the first time it just sounds like garbage. It sounds awful. I mean, think that's the struggle between kind of this creative aspect, as well as the data science aspect, because if you wanted to stay true to the data, you can't really massage what the song sounds like. If there's a particular part of the song that I don't like, I can't change that one part of the song, because that would probably mean I'd have to change one part of the data. Usually when I tweak one little thing, it completely changes the whole song. So it's really tricky. It is just through brute force of just throwing things together and constantly just changing variables until it sounds good. As long as it retains that kind of faithfulness to the data. You don't want to make the song sound good at the expense of not being faithful to the data.
Joel: And so we gave Brian some of Brant’s EEG data - the brain recordings of Brant having a seizure, an encounter with The Music. And Brian turned that data into a song.
Brian: And again, this is not a research project. This is a creative project. I wouldn't take what I'm saying as actual scientific research.
Joel: An EEG, or electroencephalogram is a measure of the brain’s electrical activity. It’s a really common research technique in neuroscience where it’s used to measure anything from a person's sleep behavior, to what’s going on in someone’s brain during a seizure.
Brant: They put you in a bed, they put little dots on your head and they say, "Feel comfortable," then they walk off on you. Then they take the drugs off you, and for most epileptics the drugs they take will make them quite drowsy so they can't sleep either. And I'm sitting in there with wax electrodes on my head and I'm thinking, "Yep, they're waiting for the fit, so I'm going to be stuck here." And I was.
Joel: Brant’s fit eventually came, and it’s the data recorded by those electrodes on his scalp that Brian transformed into the music you’ve been hearing in this episode.
The way Brian composed this song - or the algorithm he wrote composed this song - draws on three elements of the EEG data; the amplitude, or the height of the brain waves, which is a measure of how active the brain is. The frequency, or the number of brain waves that occur in a given amount of time - this is a measure of how alert the person is. And the synchrony, or the relative activity of different parts of the brain. And then? He maps changes in these three variables to changes in sound.
Brian: Amplitude, very conveniently, evokes this idea of is the music louder or softer. Obviously, higher amplitude, the louder the instrumentation. Frequency also has a good corollary to music. High frequency, the instruments are playing at a higher pitch. Lower frequency, at a lower pitch. Synchrony I use to control the percussion in the music. High synchrony, the more drums are playing at a synchronous pattern.
Joel: As well as mapping to loudness, Amplitude controls the vocals in the song as well.
Brian: The higher the amplitude, the more vocals are playing. The different parts of the brain have different kind of vocals associated with it. If all parts of the brain are firing very loudly, there's gonna be many vocals singing very loudly at the same time.
Joel: Vocals are a key part of this composition - they’re the dominant feature of the song. To generate them, Brian sampled the Imogen Heap song, “Hide and Seek”.
Brian: Yeah, I very deliberately used the Imogen Heap song for a few reasons. One, it's completely vocal. Part of the way in which I try to compose these songs is, think about what the listener should be experiencing in relation to this dataset. You know this dataset represents a human being, another individual. Also, very intimate dataset. It's their actual brain activity. Is it possible to produce empathy between a listener and the subject? I wanted to use a vocal element of the song, because it is a human subject. Another little trick that I did, or another concept that I tried to leverage was in psychology, or I don't know what field of study this is, but there's something called phantom words, where you kind of stitch a bunch random syllables together. People will hear words, regardless of whether you're giving them those actual words. I kind of chopped up that song, into syllables, and the algorithm kind of stitches it together in various different ways, but it plays a little mental trick on people where people would be hearing different words, and different people would probably hear different words. Again, it's trying to create an experience that's very personal, and kind of unique to the individual. It even lets the listener's brain do some work. Again, it's trying to connect the listener's brain to the subject's brain.
Joel: By sampling a well known song, Brian also plays into Brant’s experience of the musical aura, where songs he hears, or even thinks about during the seizure, are warped, twisted, and incorporated into The Music.
In addition to the vocals, Brian sampled strings from the Philharmonia sound sample library, and percussion from the American experimental rock band Swans. And then? He just let the algorithm do its thing.
Brian: I’ll play the song in full at the end of this episode, but first - what does Brant think of the song composed using his seizure data?
Joel: That's it. So what do you think? First impressions.
Brant: It reminds me of the graph actually. I've seen plenty of them. They always show it to me after they make them. Well at least I can say to other people, I sound interesting.
Brain: I try to think about the dataset, not as this series of zeros and ones. It is a representation of an actual human being, with real experiences. I think the medium of music is very unique, in the sense that it will evoke a visceral response. My goal with this particular project is to think about what should that response be, as it relates to this particular dataset. I think that's where a lot of my creative energy goes, is thinking about how to match what I believe this dataset is about, and how the listener should kind of experience it, in this very visceral way, because music you feel something. That's what I really like about music, as compared to say a chart, or a graph, which I don't remember the last time I was moved by looking at a line chart. But it's a good match. I think it's the right medium for human data, a medium that has this very primal, visceral quality to it.
Twenty Thousand Hertz is produced out of the studios of Defacto Sound, a sound design team dedicated to making television, film, and games sound incredible. Watch and listen to the latest work at defacto sound dot com. While you’re there, be sure to reach out.
Joel: Sum of All Parts is produced by me, Joel Werner. Sophie Townsend is Story Editor. Jonathan Webb is Science Editor. Sound design by me and Mark Don.
I highly recommend subscribing to Sum of All Parts. It’s a podcast that tells extraordinary stories from the world of numbers. To hear more stories like this, search Sum Of All Parts in your favorite podcast player.
Additional help in this episode comes from Mike Nagel and Sam Schneble. It was mixed by Nick Spradlin. Also, thanks to Luciana Haill for letting us play the sonification data at the beginning of the episode. You can find more of her work at lucianahaill dot co dot uk/. That’s Luciana Haill dot co dot uk.
And learn more about our show, Twenty Thousand Hertz, at our website two zero k dot org. There you can find links to the things we’ve talked about in each episode, you can stream our archives, send us story tips, donate to the show, and even…buy stuff! The stickers are sticky and the t-shirts are soft. Also, follow us on twitter or Facebook by using our handle two zero k org. Or by searching for the name of our show, Twenty Thousand Hertz, all spelled out. Okay. I think that’s everything.
Thanks for listening.
Brant: Did you notice I had a fit during that interview?
Joel: I totally had no idea. What happened?
Brant: It was generally a three to four second lapse.
Joel: And what's it like for you? Like what's the experience?
Brant: That one wasn't a heavy aura, it was just a white noise one. Those ones generally don't have enough time to give me a strong rhythm. So they're there, I notice they're there, and then they're gone.
And now, in full, the data sonification of Brant’s seizure data by the Data Driven DJ Brian Foo.
[Seizure Song]
