Neon Audio Processor - OpenCL iTunes visualizer

cybero's picture

A screen grab of some work in progress, an OpenCL graphics kernel based iTunes visualizer.

The construct is essentially one whereby the audio spectrum and peak data are separated into peak,low, mid and high , these values then being the initial values for the computation of the resulting audio reactive graphic.

The bands are high, middle and low from top to bottom.

Looking at setting up some presets, as the slightest change in certain of the input variables results in drastic changes to the graphic rendered.

photonal's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

Love the music!

Who's it by?

ooops - just saw the source on vimeo....

cybero's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

General Fuzz , one of the most generous musicians on the face of the planet, made the music :-).

[Published under Creative Commons licence].

dust's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

thats pretty trippy have you tried maybe cutting out some frequency bands of the audio track or used a drum track, at points it looked like the top two bands where kind of reacting to the same frequency. i have been finding that i am more pleased with reactive visuals when i separate the audio tracks. is kind of hard to do if your not making the music. although its possible to isolate bands and save three different files and use them to drive the visuals but still play the master mix just not use it to drive the visuals.

what i mean is taking lets say the kick drum and baseline and putting them in one audio track for the low end visualizations. then taking the lead and snare for mids, and violins and hihats etc for the hi end of the spectrum. then running the spectrum through off those files but playback the master mix ?

i don't know if that makes any sense to anyone. unfortunately its not possible to do that with an itunes visualizer or as a vj but it makes pretty good results if your using qc to make other type of reactive art or something ? i don't know where i'm going with this.

i like the movement. i wish i could get CL down like you. i have been trying to extract xy out your optical flow CL patch but running into troubles. i have been able to convert UV to xy from apple optical flow though.

photonal's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

using separate tracks definitely gives better results but I find using midi note + control data is even better because it provides more scope for arranging the visuals - especially when e.g. having separate tracks in say Ableton Live for outputting the 'visual' data into QC.

dust's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

i'm actually the opposite i find i like the spectrum stream better.

i mean there is an obvious 1 to 1 mapping correlation between hitting a midi note and turning something on and off in qc, or using the note value to change a color but i find use the peak amplitude and a few spectrum index way more than i bother mapping midi out in qc.

using the midi note values booleans and velocity info in concatenation with ableton live pushing out various tracks eq'd to a particular band through soundflower(16) that you are in turn parsing out in qc to drive the visuals ultimately would be the best i guess a combination of both.

cybero's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

I think you mean like a FLAC track - separate tracks for guitar, drums and so on?

[Just re-read after the day's first cup of tea and see where you're getting your lows, mids and high tracks from - good idea for presentations, installations and offline renders].

Would produce a more reliable, faithful and configurable result.

Not always effective depending upon the initial audio track type.

MIDI to separate .mp3s - definitely feasible to FLAC style.

Thanks for the advice.

BTW - it's an iTunes visualizer - far more restrictive on what can be used ;-)]

cybero's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

Separating tracks however , although possible when doing offline renderers is not possible, in quite the same way for obvious reasons, when running this as an iTunes visualizer. Your advice and that from others regarding MIDI controlling and delivering dedicated instrument or low, middle and high tracks is well received in regards of their pertinent application.

For an iTunes visualizer, especially if one is wanting to make one that will run on on any Mac running 10.6.x, I have to ensure that it will run utilising the input instruments natively available. The screen grab was taken from an actual iTunes session and is not in fact an offline renderer.

Perhaps the bucket aggregator script in the Audio Processor patch which provides low, middle and high information from the audio spectrum could be improved upon further. I have been refining this particular visualizer and shall be uploading an updated version of the video pretty soon.

I love the way it goes to black when there's no sound and that it picks up now on all sounds very well, although that does now mean that at any time one or more bands will be struck out, although that only is true if the track is predominantly low, mid or high band. Thinking about an entirely different way of treating rendering to different spaces in response to predominantly low, mid or high, leaving less screen estate going black at any one time.

dust's picture
Re: Neon Audio Processor - OpenCL iTunes visualizer

Honestly cybero flac is one of those audio formats I have yet to explore. if it encapsulated multi-track formatting it would be perfect for what Im talking about. theta is only one spot that briefly it seemed the bands where meging in this predominately there is clear difference. it's actually interesting to see how they are stacked and to see where one band leaves the other starts similarly then there are times when there is a huge difference where like the mid band is just doing something totally different than the others. I have just been isolating frequencies and killing every thing else lately. Mainly so I know What index or indexes are best to use from the spectrum. l