Kinect/TuioClient Integration?

kcfotog's picture

I was curious if there was any way to parse Kinect structure data to a TuioClient patch? I've been looking at the various ways to get x/y and velocity data from the kinect into my Quartz comps and I'm really hitting some dead ends.

I do have a trial of Osculator and can see the data structure moving around when I hook up the Kinect. I assume their is massive translation that has to be done to get that data to a format that OC can recognize? Perhaps a massive array of structure index and key members?

Thank you for any possible advice you can send my way. Cheers!

SteveElbows's picture
Re: Kinect/TuioClient Integration?

OSCeleton now has a mode which sends OSC messages in a format that QC can handle. See the OSCeleton readme for details.

https://github.com/Sensebloom/OSCeleton

kcfotog's picture
Re: Kinect/TuioClient Integration?

Steve, thanks for the heads up on the OSCeleton. I remember seeing this posted a few weeks ago but was immediately scared away because of all the extra libraries and such that had to be installed via the terminal. I'll man up and actually take a shot at installing this over the weekend. Much obliged:)

dust's picture
Re: Kinect/TuioClient Integration?

use the tuioKinect app and the tuio client for quartz composer on tuio.org

kcfotog's picture
Re: Kinect/TuioClient Integration?

Dust, thanks for the suggestion. I actually tried that route and couldn't parse any x pos, ypos, xaccel, y accel data like I'm able to do with a just a simple web cam and the tuioClient patch. Is there an extra step like opening up Osculator to bridge the osc data from the TuioKinect app to the TuioClient patch? Or perhaps a device ID isn't matching up? I'll give it another shot, today.
By the way, I have learned much from your examples and project breakdowns. Thanks to you and the rest of the QC community for your helpful contributions. Cheers:)

dust's picture
Re: Kinect/TuioClient Integration?

you need to set your threshold right. in the kinectTuio app i usually push the "<" key till its at about 100 then you should see a blob and id when you wave your hand.

kcfotog's picture
Re: Kinect/TuioClient Integration?

Thanks, Dust. I'll give that a try. I've adjusted those parameters before in previous attempts and I could actually see blob id via the TuioKinect app. It even translated position and accel data to Osculator. However, getting it into the TuioClient is where the process fails. I have the client on UDP port 3333...not too many other parameters I can think of to adjust. After checking the Tuio.org forum, it seems this is a reoccurring issue. Hmmmm...the troubleshooting continues. Thanks again for your help.

dust's picture
Re: Kinect/TuioClient Integration?

don't use osculator to send the tuio events to qc. osculator re-formats the tuio events so you would not be able to forward them via osc. you may also be running osculator in which case port 3333 is bound all ready and the qc client will not work. try closing your osculator processes.

kcfotog's picture
Re: Kinect/TuioClient Integration?

Good point with the use of Osculator and Tuio ganging up on the UDP port. Currently, I am running the TuioKinnect app with Quartz only, and use the attached iterator to pull the Key info from the Tuio structure. Right now, I'm using the Tuio key names for the x,y data: "x pos" and "y pos". Perhaps they changed those particular keys for the TuioKinnect app?

Thanks for your help on all of this. I'm learning a bunch in the process so I really appreciate all the informed input and feedback. Cheers.

PreviewAttachmentSize
TuioKinect.qtz3.33 KB

tuscland's picture
Re: Kinect/TuioClient Integration?

dust wrote:
don't use osculator to send the tuio events to qc. osculator re-formats the tuio events so you would not be able to forward them via osc.

Hey Dust,

This is not true. OSCulator offers the option to reformat the messages for easy use, or keep them intact. In the Parameters Window, you can choose between Interpreted (reformat) mode, Raw mode or both.

Best, Cam

dust's picture
Re: Kinect/TuioClient Integration?

awesome i use osculator all the time, didn't know you change the formatting around. thats sweet, hope it didn't sound like i was putting down osculator by saying not to use it. just trying to trouble shoot the issue.

must say osculator is one of the most useful apps i have ever bought, there are a ton of new features, every time i open it there is a new update ready for download.

was just building a tuio controller the other day and using the osculator namespace to get my tuio messaging right but what i didn't know is the tuio messages where not working because the namespaces where different in osculator. thanks cam usually my problems stem from user error.

kcfotog's picture
Re: Kinect/TuioClient Integration?

Hey Dust , if you ever get anywhere with your Tuio/Osculator input project, I would love to know your workflow. That sounds like the solution I've been looking for. Thanks!

dust's picture
Re: Kinect/TuioClient Integration?

well I have a bunch of projects using oscillator. I made a simple tuio controller for the iPhone in unity to be used with xtuio. I posted it over at at xtuio if your interested. xtuio parses tuio into unity. seeing that it doesn't send tuiommessages in thought it was a good idea to send tuio messages from unity as well. the interesting thing is that you can use your local loop back 127.0.01 and send the messages to yourself on an iPad or making it possible to use xtuio with a mobile. go to xtuio forum and have a download if it interests you.

kcfotog's picture
Re: Kinect/TuioClient Integration?

Xtuio looks pretty promising for the mobile and multitouch apps. Lots of exciting possibilites, for sure.

Were you able to check out the TuioClient setup I sent in my previous post? I'm curious if my iterator used to parse structure data is calling for the correct key for the x and y data coming from my TuioKinect app. I'm able to get the x,y data using the same Tuio patch coming from a web cam and my CCV tuio app...still no luck with the Kinect.

Thanks for the help. Cheers.

dust's picture
Re: Kinect/TuioClient Integration?

im picking up my machine that has the tuio kinect on it today, had some issues with it. so when i get home i will try and run your file with my kinect and see if it works then fix it or upload. martin from tuio made the tuiokinect app and not someone else so i would say it has the same tuio functionality as normal. your saying that your getting tuio events from ccv but not from the kinect app ? i have been using the binary tuio kinect without any issues.

(edit) i looked at your file. so a few things not sure about the "x pos" or the "xpos" key and which one is correct. i'm going with "xpos" you can hover over the cursor output of the tuio plug it will give you a tooltip showing what the keys are. secondly you index member is connected to the "key" of ypos and not the structure input of key member ypos. that is why ypos wouldn't work. another thing is that using an output from the iterator will only give the last known position. you must connect a circle inside the iterator to visualize more than one hand. with tuio kinect there is no need for an iterator as your only going to get 2 hand blobs. (edit)

see if any of that helps. will check back in a few hours or so after i pick up my laptop

kcfotog's picture
Re: Kinect/TuioClient Integration?

Dust, you are a kind soul, Sir. Thanks for notes on my structure rig. I will make some revisions per your initial notes and see if that helps. I really appreciate your time and look forward to see what you think once you get your laptop with Tuio back.
Best!

gabemott's picture
Re: Kinect/TuioClient Integration?

Did you ever get this one to work? TuioKinect.qtz Thanks...