|
|
CIKernel Color Tracking so i have been messing with color tracking today. a few years back i made an ambient light theremin in max utilizing the 3m or average rgb mean of a video input, entirely simple. so i decided to cobble together a ci color tracker from the source at apple. i have found some things on google, http://lists.apple.com/archives/Quartz-dev/2008/Mar/msg00137.html http://petterniklas.wordpress.com/ i can't really find anything on kineme on the subject. i planed on making a plugin to do this but found it was just as simple to copy the kernal into a ci patch in quartz. so with my limited cocoa and ci skillz i have been able to patch together a few kernels to track a color. similar to the apple example. i want to know if this topic has been discussed as i want to learn more about ci as well as maybe get some help to infer a better z depth. i found some people wanting color tracking in a open cv thread. i know GT posted a link to the ci tracker before and was wondering if anybody has got this working 60fps. as it is this is working and tracking the color orange you can give it a try if you like. track what ever color you want. i am still trying to grasp some concepts, looks like i need to buy the gpu gems book.
|
This is something I was almost obsessed with for awhile. I finally got it working... I ended up with a qtz with a feather that moves around almost like optical flow via CI, but which tracks to color. I felt like the Z inference was laughable, but that also has to do with the lack of res on the built in i-Sight and non "pro" lighting conditions. I felt it was really finicky to do it in CI and get anything meaningful.
It was a lot more cool in theory than in application. The fps was definitely lacking... I just did a spotlight search for the qtz, but I suspect it's on an external HD that I don't have by me right now.
There mayyy be a decent example hidden somewhere at quartz compositions. It almost seems like after I spent weeks messing around with it on the side, that I found the exact same implementation posted somewhere (not on the GPU Gems page either).
I remember messing with this like 6 months ago when I found that Xcode example. I hadn't read that post by Niklas previously and it was pretty funny to see that I did exactly the same detective work as he did and looked at the AppController.m to see how to wire up the QC patches.
I found some old example I did and it runs pretty steadily at 60 fps both with a video file and the PS3Eye camera. Slows down a little with the iSight, but it's pretty stupid to do any tracking stuff with the iSight anyway. You really don't need a resolution of 1280x1024.
Here is the Image Unit I compiled if anyone needs it: http://mansteri.com/download/software/imageunits/TrackingImageUnit.plugin/ Put it in your Library/Graphics/Image Units/ folder
And here is my example: http://mansteri.com/download/examples/qc/ColorTracking.qtz Just the basic setup.
I think I tried it with VDMX, but ran into some really weird rendering error. Can't remember what it was, but something strange was going on. I'll have to try it again some day.
i must say you are right on with the lighting situation. last night this was tracking pretty good with my isight at 50 frames a second but this morning its just to bright in my office. these are the things im thinking that would help this out because maybe like you im obsessed to get this working very well. one is to threshold the color mask to the tracking color then take the mask cheap blur it out so its a black and white blob then feed it into another color mask and track the white color. this seems to improve the tracking a bit. im thinking your idea of optical flow could be interesting either use it to smooth out xyz structure or use it and average the flow xyz with pixel average maybe get a rgb optical flow thing going on. i will experiment some more maybe search for that comp you are talking about. i think if i am able to track two objects then with some fancy math i might be able to infer rotation as well and use the center of the tracked objects as my xyzpos, xyzrot, and maybe size which can also cheaply infer z depth or at least thats how i cheat with facial tracking. otherwise i need to know all kinds of info like the fov of the isight then subtract 1 to be center zero to offset the screen this will give me a proper z depth then i will be able to do some trig to get rotation at least thats the idea. im taking a multi touch class this semester and i am comfortable with qc so i am thinking we are using nuigroup software that translates to tuio but would be cool to be able to do some real blob tracking in QC.
awesome your works better i think. i really tried to replicate the x code project like you did but for some reason i think the numbers where a little off due the original being an application and not a qc app. plus i negated to use the overlay because im more interested in xyz but i think if i put these two together i will be able to do what it is i'm trying to do. eventually make musical instrument ;) thanks for the help, this set up seems to work very good with the isight actually. i didn't use the image units i copied the kernels by themselves maybe thats why your set up is more accurate. ??