We need a MIDI engine I

There was something impalpable in the way game music and sounds were played through synthesizers and samples back in the 80s and 90s. Something that was telling you that the sound, melodies and beats were coming from the heart of the machine, something REAL was happening.

Now computer game music just feels detached or forced, especially with non AAA games. It doesn’t feel “natural”.

Let’s take the ubiquitous Mario on the NES. Los doggies (amazing music blog btw) deconstructed the sound of the game, music and sound effects. Here’s the example of the famous coin sound –2 notes-:

The B acts as an ornament to the E. Together, they form an interval of a Perfect Fourth. In relation to C Major, the tonal center of Mario, they are a Major 7th and a Major 3rd respectively. What kind of world has Major Thirds erupting out of reality? Oh yeah, our world has that. Major Thirds are found in car horns, bells, telephones, door bells, convenience stores, pop music, and every other kind of music. And now coins.

Not that Koji thought about it this way but he certainly made sure that sound fxs were matching the music tune, then iterated to hear in which interval it was sounding best. Yes, you can do that with non-MIDI game audio. It’s just so much more tricky, annoying and tedious in the work flow and production process to do so. Plus we need multi-platform support business wise so it’s even more complicated. I did it for SideFlip but it was painful and I could have done so much more with a MIDI engine.

It is frustrating because visual designers have such a wide array of rendering solutions and graphic output with great control over them. We don’t have shit. The analogy with graphics is that MIDI is vector-based, sampling is vector and bitmap-based and streams are full bitmaps. We only have access to the last one, which is not the best in every case, far from that actually.

When you think about sound, music and fxs as tones that you can control through an engine and manipulate with gameplay, MIDI is just the way to go. We don’t need to reinvent the wheel every time we need this functionality, it’s stable it has everything we need. We need an audio suite with an editor to create extremely interactive audio and awesome stuff.

MIDI engine
Basically

Game audio designers feed the engine with MIDI, feed the sampler with uh, sample banks and/or adjust synths and filters through a few scripts and the final audio stream happens in RAM, in the game. The important thing is this:

midi engine interactivity
This is where the fucking magic happens

We can control everything here. We can cut the bass line if the player is low in energy or add a clap on the snare when the player is doing great. We can accelerate music and sound fxs as the player moves faster, we can filter and sweep out a jump fx made with a synth to match the exact length of an analog action, we can send a note off if the combo is not perfect, we can make musical transitions, cuts and breaks to match the action… Things we can’t do or are so tedious with audio streams and one shot samples. Loop. Fade out/fade in. Cut the ambiance or music. Repeat. Streams are so limited.

This is how professional audio software like Reason are working, MIDI to computation to RAM. People associate synths and samples with bad quality because for some reason (ha ha) they think about it as the NES sound, the 8bit sound, the lame 90s Internet music (powered by 2 or 4Mb built-in soundbanks) and nothing else. They don’t know that a nice 64 Mb soundbank and a few synths can be used to make great music and soundscapes like Nintendo is doing since ages (people often can’t even tell the difference with streams since the Gamecube/Wii). They don’t know that computers are so powerful today that they can emulate with extreme accuracy a dozen if not dozens of complex synthesizers on the fly with real time input under 2ms without sweating. I mean, in ten years it is really ridiculous how much power we gain in audio rendering and computation (cheap ass netbooks ship with 1Gb of RAM and 24bits audio output).

This interactivity opportunity needs to be deployed into game development. Not in a back to the old school way, but more as a rebirth. After ten years making audio and music for computer games, I haven’t been convinced by the stream road we took, at all. In a lot of cases, especially for small and medium-sized games, a MIDI engine would stimulate the gameplay and make it more alive than a 2 channel soundtrack looping in the background with stock sound fxs. But yes it means you really need a musician/designer/scripter on that part of your game!

Next time I will talk about the tools we have now, the tools we need and the weird situation with programmers.

6 Responses to “We need a MIDI engine I”

  1. zeduckmaster Says:

    The same story happens to 3d.
    First highly customizable, all done by cpu, slow but with high creative potential (video games were not here though these times).
    Then first hardware came out, all games looked the same.
    Then shaders came and opened 3d (not so much but as least more, it is still too much rigid imo).

    The same should happen for music because i agree, midi music are dawn so much more dynamic!
    My latest experience was the last pokemon. Some examples:
    – in one city, you find a piano and a drum player, after speaking to them, the piano and the drum are added to the main theme of the city music.
    – at the end of the game, i found the battle theme strange, the same but not the same. I just realized after few seconds they modified it on some part (don’t know exactly what they did) to make it more epic. The same listening but not the same feeling.

    In anyway, i agree with you, the future of music in vdeogame is midi.

  2. zeduckmaster Says:

    by the way i’m curious (no a trap question though), what do you think about wii music? especially in comparison of guitar hero?

  3. harold Says:

    I didn’t play the Wii enough to say :-) I never thought that it was bad or too “cheap” or anything. I though, found countless PS360 games with annoying and more than forgettable music themes.

    Did you play Rhythm Tengoku? It’s so hilarious, using sound feedback and synchronizing everything… I recommend Everyday Shooter, it’s really great for that. Once you forget about graphics and focus more on sound feedback you notice something is really different all of sudden (sound effects done with a guitar, accordingly to the music).

    I’ll go into details more with examples of great stuff and history of MIDI game audio next time ;-)

  4. h. Play» Blog Archive » We need a MIDI engine III Says:

    […] We need a MIDI engine I […]

  5. h. Play» Blog Archive » Polytron and me Says:

    […] Bedard did a fantastic job at coding everything, even a complex audio system (we really need a real MIDI engine instead of reinventing the wheel each time). Chapeau […]

  6. h. Play» Blog Archive » Like me some gamasutra Says:

    […] article on Pugs Luv Beats’ Theory of Fun and generative music. Basically Pure Data is my MIDI engine, the only difference is that I would less go on the playground’s road and aim more at a middle […]

Leave a Reply