Kinect

here is a proof of concept I'm working on that utilizes my midi arpeggiator plugin.

this video demo's the use of the floor as an interactive instrument. the project target is to be a children's discovery museum exhibit.

while building this instrument it became apparent that some sort of interpretive and or choreographed dance / theatrical routine could be performed with multiple kinects and or triad chord articulations in concatenation with a second person. like setting one foot to be a chord and the other to lead instruments starts making much more complex instrumentations.

the implicit intention is to teach children cognitive and muscle memory through a cross modal auditory and visual sensory feedback system based on the body and motion. in other words a giant musical simon says type of game, which ironically isn't as fun to watch.

the idea isn't entirely novel as i have seen a few interactive floor projections and multiple floor pianos. you can even buy these types of things. most people ask why there isn't a chromatic scale and my answer is that it would sound like a cat running across your piano keys at night time. as it is now its fun for most ages as it requires no musical talent at all.

I'm thinking of some to incorporate rhythm, like possible maybe looping play head people jump or something to create rhythm sort of like a jumping rope idea i guess. i want to make it fun for kids but don't want to make the cliche hopscotch piano. any thoughts on this subject. has anybody done anything similar.

I have been messing around with using a Kinect, and a Projector (Quartz/Tryplex/Synapse) to make any wall or surface interactive.

Here is what I have come up with so far. I was able to get the kinect world coordinates into -1 to 1 space, which you would then use in a render to image that was being mapped to any surface.

At this point it can recognize two hands, and swiping gestures.

http://www.youtube.com/watch?v=wVqeRMshT9E

I simply took one of the compositions that came with the Synapse OSCeleton downloads, put in vade's particle warfare and added that to the gestures that casey and jamey made...

When you touch your hands together, the active particle hand changes so it appears as if you can move the particle from one hand to the other. I'm sure there is a better way to do this than my clunky math, and really the ideal would be if the particle system behaved sort of like a separate object that could be tossed around... but I digress.

What I was trying to do and got stuck is: make the inactive hand control the colors of the particles. I wanted to have it change color as the off hand moves up and down the torso. For example off-hand at stomache would make particles yellow, off-hand at neck=blue, etc. But I couldn't figure out the first step of identifying the off hand.

Any help is appreciated, regardless, hope some of you can enjoy this as is...

Obviously this requires the kinect and synapse. Here's the one that Sebastian wrapped up with OSCeleton and Tryplex which has it all: http://code.google.com/p/tryplex/wiki/Installation