Reverse Visualisation or Sonification Leads

(..where's the edit button? :)

For example I'd like to feed some video (could be live source too) into a composition and perhaps select sections of the viewport as data sources; multiple ones would be great. Would it be asking too much to be able to measure hue, saturation and brightness and convert to e.g. midi or OSC data?

You could look at the Image to Sound plugin from _1024 or do some vertex displacement of image to mesh.

yes thanks I looked up the _1024_Image_to_Sound QCplugin, but it appears todefault to the "first 1024 pixels of the image" ("upper left corner") so it doesn't seem to be possible to determine which 1024 pixels you'd like to analyse, i.e. make the 1024 pixel buffer 'look' at some other part of the input image source.

or as Henry Ford would put it; you can sonify any part of an image as long it's the upper left corner! ;-)

But if you placed a process before that samples distinct 1024 portions of an image, and send that to Image to Sound that would begin to fill that bill.

Doing an Image to Mesh usually involves the whole image, but thereafter the mesh is potentially accessible.

ah great idea, that's what you meant by "some vertex displacement of image to mesh" ! :)

thanks for the pointers

you can actually crop your wanted portion of the image, then eventually resize it to 32 by 32 pix. (== 1024) so that it fills the audio buffer. see attached example.

Sorry if I'm missing the obvious here but I don't see how I can use the analysis data from _1024_Image_to_Sound plugin as it does not seem to have published outputs, so I can't use that data to drive my sounds.

What's missing from the thumbnail sketch is a Kineme Audio Input that can pick up the generated sounds from the interpreted image and feed back the bucket aggregated lo mid hi and also send back peak, waveform and also frequency data in a variety of formats. Using Noise Industries free Random Crop allows one to have admittedly random points in the image interpreted to sound.

If you also have a built in process to audio react or affect some other part of the graphics produced prior to going again to the Image to Sound, then you have the facility to get some audio visual feedback loops .

I'm beginning to think that something along the lines of David Langford's Weighted Grid Renderer might prove useful.

And looking briefly into that, it looks like good for something else entirely :-)

you can look at the dev. sample patches "color to midi" . as far as i recall, it will convert a color to midi note, or something like that. Modifying it to accept any other kind of data should be trivial.

I hacked this together as a starting basis (from a Chart comp and the 32x32 _1024 comp), but now having trouble getting midi out of the kineme midi patch - how do you set the midi port to receive the midi data?

I tried using the system 'MIDI Controllers Sender' patch but that keeps causing QC to crash.

Any clues welcome.