Working with Audio

Audio processing

• Cycling'74 Max/MSP - Visual programming environment for visuals and digital sound processing. Can also be scripted with Javascript ($)
• SoX Command-line utility capable of a variety of audio processing

Surround

• Google Resonance Audio - Tools for working with surround sound, particularly in VR/AR environments. Unity, Web, etc.

MIDI

MIDI is the protocol used for sending commands to most software and hardware synths, drum machines, effects and so on. It's pretty versatile, and depending on device capabilities, can be routed over cables, Bluetooth, Wifi and USB.

There is a huge range of MIDI controllers which send MIDI but don't have any sound creation or manipulation capabilities. Since they are often cheap, easily connected, programmable and have interesting input capabilities, they are quite useful for prototyping. Moreover, many allow you to send MIDI data back to them in order to turn LEDs on and off, or even move motorised faders.

Even though MIDI is designed for audio applications, you can easily use it as general input protocol. Chrome supports the WebMIDI evolving standard, allowing you to connect web apps directly to MIDI devices. Shields for Arduino exist to pipe data over standard MIDI DIN ports, you can use a Teensy device to send MIDI over USB, and many microcontrollers with Bluetooth support MIDI-over-Bluetooth.

'OSC' is an alternative, newer protocol which is sometimes used for interconnecting devices and software. It's not as widely supported however.

• WebMIDI example
• USB Host Board