Real-time tonal analysis -> MIDI control signals for iPad

ZeroStride's picture

I've been developing an iPad application which uses the Constant-Q transform to do much more in-depth and specific audio analysis and send the results of user-defined filter areas to other software using MIDI.

I'm looking for beta testers for this application, specifically performers who can provide feedback on the workflow for both live tasks, as well as prep work. If anyone is interested, you can sign up here: https://testflightapp.com/join/dd9b574e74e7392f36c1ff6b5b55f5a5-MTA2MTg3/

PreviewAttachmentSize
new_ui.png
new_ui.png182.32 KB

gtoledo3's picture
Re: Real-time tonal analysis -> MIDI control signals for iPad

This looks like a neat idea. I've had trouble being able to detect the notes in polyphonic signals in high decibel scenarios, let alone with built in mics, so it will be super if this gets that down. Monophonic, not so much a problem.

A desktop version might be a cool idea to eventually make it easier to use external mics/preamps/compressor/gates, without having them hooked to an iPad.

ZeroStride's picture
Re: Real-time tonal analysis -> MIDI control signals for iPad

Yep, I agree. It actually started as a desktop app and then I got a wild hair and decided to try it on iPad. Primary advantage to iPad is offloading the processing, and usability. I still don't recommend using the built-in mic but it does work (and is kind of amusing to turn on in a bar and watch what the sounds around you look like).

I've written the core in 100% ANSI C and it makes heavy use of the Accelerate framework and NEON instructions. Processing of an audio frame is very fast (6ms mono, 12ms stereo), gathering enough samples to feed the CQT is what takes the time. There is also a "tone isolation" mode which runs over each octave and does its best to find the most relevant bins of signal, then applies an exponential function to the octave to make those peaks more pronounced.

In my testing it does a much better job than the VDMX Audio Analysis plugin, at the expense of some latency (due to gathering more samples) and it has the advantage of sending to any MIDI receiver, which is helpful especially if you are using multiple bits of software to do your performance (linked with Syphon etc).

Make sure you register a device with TestFlight, otherwise you won't be able to run the builds when they start.