We need a MIDI engine II

The goal would be to provide a toolset a game audio designer could use to:

-Make him able to be free and creative by using his/her standard sequencer software and export midi/soundbanks/scripts in a convenient and efficient way to the game.

-Minimize the programmer’s work who would spend NO time on things he/she shouldn’t do like fixing sound fxs volume issues (remember, I am not talking about triple A business) and spend more time on better things like polishing.

What we have

The tools are a mess. Some tools do some extent of the job, some did it perfectly but are now dead, some survive but are dying… We’re almost there. And yet so far!

First there’s the MOD file (.s3m, .xm, .it). Yes, it’s MIDI data + samples/synths. It’s not crazily supported but Irrklang does as well as well as the ubiquitous Fmod.

Where it’s so weird is that it’s only for playback purpose: no way to interact at the lib level with channels and parameters of MIDI data! Why would you absolutely just play a mod file instead of any pcm stream, I don’t know.

To add to the mess there is the tracker, the most annoying and unnatural way to create music. I don’t want to argue about the fact that it can do great music, it does. But we shouldn’t have to deal with the tracker if we want something with MIDI + samples/synths. Like it or not, we don’t have to enter music on computer keyboard in hexadecimal anymore.

So because the MOD file while supported is definitely depreciated, because the tools are awful, it’s pretty much useless for game development.

OpenAL? No MIDI, bye. OpenSL? Just started for mobile, there’s MIDI, there’s a mapping of its data –I like to read SLMIDIMuteSoloIt– but there’s no designer tools (and what the fuck these profiles mean? 3D Audio for games? On mobile? Please). SDL? No MIDI.

There’s the BASS audio lib that is supporting the playback and manipulation of MIDI files and real-time events through soundfonts, it’s cross-platform but I’ve never been able to see a game using it. People use BASS to build their own little tracker or MIDI player. Soundfonts are pretty much dead too.

The Miles Sound System seems to do everything right according to this page… But it’s 4000$ so I never tried it.

That’s it. No widely available tool to help you create sound interactivity with MIDI. Microsoft had this amazing thing called DirectMusic Producer but they killed it when they shut down the MIDI part of Windows with Vista’s brand new audio stack. They did that to simplify APIs I guess and also because nobody wanted to use MIDI (I’ll get to that in part 3) but man, that was annoying to witness.

Since then, it’s all about 3D. I never really got this obsession with 3D audio. There’s way too much tools for it compared to the number of people or games actually using this feature (FPS and hardcore gamers, and? Who is playing with a nice 5.1 setup? People spent like crazy for their big ass TV and thought “fuck it” for the audio gear, for what I saw).

Main tools and audio engines like Fmod and Wwise are providing designer tools aiming for “volume” features, things helping to manage huge amount of streams and sound fxs, localization files etc. Which is fine but it’s now a very small part of the game development market! Teams, projects are shrinking heavily (for the best) these days.

What we need


My handwriting sucks, I know

I tried to get more specific about file formats.

The sequencer of your choice export data in .mid (ubiquitous format), the soundbank for the sampler is provided either with dls (industry standard and there are tools to edit them) or sfz (Cakewalk’s take on soundbanks), the advantage of sfz being that it’s highly flexible and completely free. Plus, you just need a text editor to change things, I know programmers would be really happy with that :-) I use this format all the time and it’s really nice.

Then, .mid/.dls/.sfz are mixed together and controlled by scripts. I’d go for Python in terms of readability but all of them would probably work perfectly. The output would be a Fmod-like pair of files separating data/content. It could almost be only done ala command line style!

I added a door in the synth/filter part of the engine so that if someone wants to throw in their own filters or synths they have already coded, they can (VST format).

Also overall, the GPU could be used to take care of heavy processing synthesis systems or shit like that.

I see many advantages with this MIDI game audio engine:

-A strong personality and interactivity: the game audio designer set up a “band” and makes it play –in every way- with the game.

-High scalability: the .mid can be reused with minimum efforts on any platform, only the “band” needs re-design. It’s the fun part, it’s like changing textures in an environment. You can have samples in 24bits 192KHz if you want or minimize to 8bits 16KHz for a small game on a phone, no problem.

-Efficiency: .mid are a few Kb, .dll are a few dozens of Kb and soundbanks are the size you want. Between the compression settings and what your “band” is, I’m sure an uncompressed 100 Mb soundbank –in RAM- would provide more than enough for a majority of games. Graphic cards have 1 Gb dedicated c

-Stability: editing scripts in a text editor or a free IDE is stable. Editing soundbanks in a text editor is stable. Exporting midi is trivial. Maintenance would be kept at the minimum, only the audio engine would need it. That’s huge!

Anyway, it’s just some thoughts.

Next time I’ll talk about the social-economics of the game audio engine.

2 Responses to “We need a MIDI engine II”

  1. Deaq Says:

    I had a similar idea, when I was composing music for an Iphone game, but there were no simple tools, I had to get Renoise tracker. I had already done the music piece in another program the traditinoal way, but needed to “convert” into mod file. But the process was such a hassle that I gave up.

    I think your idea is great idea. However there is a flaw with it. Most composers use multisampled libaries. 1 Piano sound can be up to 1 gb, and I doubt most game companies want their game to use 1 sample instrument which sounds very unrealistic. So in the end the total size of the game would be much huger then what is done now. Since you would have to include the sample libaries and the synthesizers.

    I as you however love oldskool midi music too, it could be taken to the next level if filesize wasant such an issue.

  2. harold Says:

    Hey,

    I hear you about sample sizes. There’s a balance that we can obtain with soft synths: need a very realistic piano? Go for a soft synth and CPU hit instead of samples and very high memory print. Samples play nice with compression too so that the size of the game wouldn’t get too crazy, it would make loading longer! We have quite a lot of options fitting a wide range of possibilities.

    The bottom line is, high quality samples or synths don’t matter so much compared to what is the game audio itself and how it interacts with the rest. A bit like so many pop songs were made on cheap keyboards or guitars but conquered millions of hearts.

Leave a Reply