The goal would be to provide a toolset a game audio designer could use to:

-Make him able to be free and creative by using his/her standard sequencer software and export midi/soundbanks/scripts in a convenient and efficient way to the game.

-Minimize the programmer’s work who would spend NO time on things he/she shouldn’t do like fixing sound fxs volume issues (remember, I am not talking about triple A business) and spend more time on better things like polishing.

What we have

The tools are a mess. Some tools do some extent of the job, some did it perfectly but are now dead, some survive but are dying… We’re almost there. And yet so far!

First there’s the MOD file (.s3m, .xm, .it). Yes, it’s MIDI data + samples/synths. It’s not crazily supported but Irrklang does as well as well as the ubiquitous Fmod.

Where it’s so weird is that it’s only for playback purpose: no way to interact at the lib level with channels and parameters of MIDI data! Why would you absolutely just play a mod file instead of any pcm stream, I don’t know.

To add to the mess there is the tracker, the most annoying and unnatural way to create music. I don’t want to argue about the fact that it can do great music, it does. But we shouldn’t have to deal with the tracker if we want something with MIDI + samples/synths. Like it or not, we don’t have to enter music on computer keyboard in hexadecimal anymore.

So because the MOD file while supported is definitely depreciated, because the tools are awful, it’s pretty much useless for game development.

OpenAL? No MIDI, bye. OpenSL? Just started for mobile, there’s MIDI, there’s a mapping of its data –I like to read SLMIDIMuteSoloIt– but there’s no designer tools (and what the fuck these profiles mean? 3D Audio for games? On mobile? Please). SDL? No MIDI.

There’s the BASS audio lib that is supporting the playback and manipulation of MIDI files and real-time events through soundfonts, it’s cross-platform but I’ve never been able to see a game using it. People use BASS to build their own little tracker or MIDI player. Soundfonts are pretty much dead too.

The Miles Sound System seems to do everything right according to this page… But it’s 4000$ so I never tried it.

That’s it. No widely available tool to help you create sound interactivity with MIDI. Microsoft had this amazing thing called DirectMusic Producer but they killed it when they shut down the MIDI part of Windows with Vista’s brand new audio stack. They did that to simplify APIs I guess and also because nobody wanted to use MIDI (I’ll get to that in part 3) but man, that was annoying to witness.

Since then, it’s all about 3D. I never really got this obsession with 3D audio. There’s way too much tools for it compared to the number of people or games actually using this feature (FPS and hardcore gamers, and? Who is playing with a nice 5.1 setup? People spent like crazy for their big ass TV and thought “fuck it” for the audio gear, for what I saw).

Main tools and audio engines like Fmod and Wwise are providing designer tools aiming for “volume” features, things helping to manage huge amount of streams and sound fxs, localization files etc. Which is fine but it’s now a very small part of the game development market! Teams, projects are shrinking heavily (for the best) these days.

What we need

My handwriting sucks, I know

I tried to get more specific about file formats.

The sequencer of your choice export data in .mid (ubiquitous format), the soundbank for the sampler is provided either with dls (industry standard and there are tools to edit them) or sfz (Cakewalk’s take on soundbanks), the advantage of sfz being that it’s highly flexible and completely free. Plus, you just need a text editor to change things, I know programmers would be really happy with that :-) I use this format all the time and it’s really nice.

Then, .mid/.dls/.sfz are mixed together and controlled by scripts. I’d go for Python in terms of readability but all of them would probably work perfectly. The output would be a Fmod-like pair of files separating data/content. It could almost be only done ala command line style!

I added a door in the synth/filter part of the engine so that if someone wants to throw in their own filters or synths they have already coded, they can (VST format).

Also overall, the GPU could be used to take care of heavy processing synthesis systems or shit like that.

I see many advantages with this MIDI game audio engine:

-A strong personality and interactivity: the game audio designer set up a “band” and makes it play –in every way- with the game.

-High scalability: the .mid can be reused with minimum efforts on any platform, only the “band” needs re-design. It’s the fun part, it’s like changing textures in an environment. You can have samples in 24bits 192KHz if you want or minimize to 8bits 16KHz for a small game on a phone, no problem.

-Efficiency: .mid are a few Kb, .dll are a few dozens of Kb and soundbanks are the size you want. Between the compression settings and what your “band” is, I’m sure an uncompressed 100 Mb soundbank –in RAM- would provide more than enough for a majority of games. Graphic cards have 1 Gb dedicated c

-Stability: editing scripts in a text editor or a free IDE is stable. Editing soundbanks in a text editor is stable. Exporting midi is trivial. Maintenance would be kept at the minimum, only the audio engine would need it. That’s huge!

Anyway, it’s just some thoughts.

Next time I’ll talk about the social-economics of the game audio engine.