The last post was written late, and I forgot to mention the biggest issue I had with the soundtrack project. Because it has to generate a separate sound clip for every note, it’s constrained to a musical universe of three octaves, one instrument, and one duration. I’d love to mix it up with some whole notes to generate harmony.

Hell, I’d love to avoid the base64 hack altogether. HTML5 audio is in a shocking state, as far as cross-platform solutions go. That base64 hack doesn’t even work on IE. The Webkit audio API rocks, but it’s not available on FF or Opera. Mozilla’s Audio Data API is fine for raw audio processing, but has been deprecated in favor of a future API to be developed by the Audio Working Group. (Someday.)

On that subject, I tend to prefer Webkit’s patch-and-plug architecture to Mozilla’s web worker model. Calling back into JS to process low-level audio works for demos, but I think it’s going to have issues when integrated with other realtime libraries like WebGL. Even with just-in-time compilation, JS is still basically glue-code, and not really suited to processing large blocks of audio on a tight CPU budget. The patch-and-plug model gets this, and though it provides the means to process low-level audio it doesn’t depend on it.