We need a MIDI engine I

There was something impalpable in the way game music and sounds were played through synthesizers and samples back in the 80s and 90s. Something that was telling you that the sound, melodies and beats were coming from the heart of the machine, something REAL was happening.

Now computer game music just feels detached or forced, especially with non AAA games. It doesn’t feel “natural”.

Let’s take the ubiquitous Mario on the NES. Los doggies (amazing music blog btw) deconstructed the sound of the game, music and sound effects. Here’s the example of the famous coin sound –2 notes-:

The B acts as an ornament to the E. Together, they form an interval of a Perfect Fourth. In relation to C Major, the tonal center of Mario, they are a Major 7th and a Major 3rd respectively. What kind of world has Major Thirds erupting out of reality? Oh yeah, our world has that. Major Thirds are found in car horns, bells, telephones, door bells, convenience stores, pop music, and every other kind of music. And now coins.

Not that Koji thought about it this way but he certainly made sure that sound fxs were matching the music tune, then iterated to hear in which interval it was sounding best. Yes, you can do that with non-MIDI game audio. It’s just so much more tricky, annoying and tedious in the work flow and production process to do so. Plus we need multi-platform support business wise so it’s even more complicated. I did it for SideFlip but it was painful and I could have done so much more with a MIDI engine.

It is frustrating because visual designers have such a wide array of rendering solutions and graphic output with great control over them. We don’t have shit. The analogy with graphics is that MIDI is vector-based, sampling is vector and bitmap-based and streams are full bitmaps. We only have access to the last one, which is not the best in every case, far from that actually.

When you think about sound, music and fxs as tones that you can control through an engine and manipulate with gameplay, MIDI is just the way to go. We don’t need to reinvent the wheel every time we need this functionality, it’s stable it has everything we need. We need an audio suite with an editor to create extremely interactive audio and awesome stuff.

Basically

Game audio designers feed the engine with MIDI, feed the sampler with uh, sample banks and/or adjust synths and filters through a few scripts and the final audio stream happens in RAM, in the game. The important thing is this:

This is where the fucking magic happens

We can control everything here. We can cut the bass line if the player is low in energy or add a clap on the snare when the player is doing great. We can accelerate music and sound fxs as the player moves faster, we can filter and sweep out a jump fx made with a synth to match the exact length of an analog action, we can send a note off if the combo is not perfect, we can make musical transitions, cuts and breaks to match the action… Things we can’t do or are so tedious with audio streams and one shot samples. Loop. Fade out/fade in. Cut the ambiance or music. Repeat. Streams are so limited.

This is how professional audio software like Reason are working, MIDI to computation to RAM. People associate synths and samples with bad quality because for some reason (ha ha) they think about it as the NES sound, the 8bit sound, the lame 90s Internet music (powered by 2 or 4Mb built-in soundbanks) and nothing else. They don’t know that a nice 64 Mb soundbank and a few synths can be used to make great music and soundscapes like Nintendo is doing since ages (people often can’t even tell the difference with streams since the Gamecube/Wii). They don’t know that computers are so powerful today that they can emulate with extreme accuracy a dozen if not dozens of complex synthesizers on the fly with real time input under 2ms without sweating. I mean, in ten years it is really ridiculous how much power we gain in audio rendering and computation (cheap ass netbooks ship with 1Gb of RAM and 24bits audio output).

This interactivity opportunity needs to be deployed into game development. Not in a back to the old school way, but more as a rebirth. After ten years making audio and music for computer games, I haven’t been convinced by the stream road we took, at all. In a lot of cases, especially for small and medium-sized games, a MIDI engine would stimulate the gameplay and make it more alive than a 2 channel soundtrack looping in the background with stock sound fxs. But yes it means you really need a musician/designer/scripter on that part of your game!

Next time I will talk about the tools we have now, the tools we need and the weird situation with programmers.