|
|
OpenCV Blob Tracking (built for Kinect) Hi everyone, I'm pretty new here, but this website/community has helped me a lot during my last project so I figured it's time to donate back :). During this last project I was building an installation that used Kinect to detect your silhouette and hands, with which you could then control it. Like so: http://vimeo.com/31154077 The reason we used blob tracking is because Synapse was of no use to us due to the calibration every user has to go through. Please note that despite that, Synapse is more accurate and less heavy on your CPU :) Long story short, I've built this plugin I wanted to share with you. It uses OpenCV and CvBlob to detect the blobs and outputs their position and size as a Structure. The minimum blob size and "movement threshold" (to smooth out glitchyness from the depth image) can be configured in QC. However, I'm not very experienced with 'cmake', 'make' and install-name, so I've got some problems with including the openCV lib in the plugin. What still needs to be done is:
I've been fiddling around with it, but no luck so far. If anyone could give me some pointers or help out that would be swell. For now this means the plugin wont work without some extra work: placing the openCV and cvblob libraries in /usr/local/lib I've included the libs for 64bit Snow Leopard (in the zipfile in /usr-local/lib), simply copying them and the plugin to their designated place should do the trick. Obviously, building the libraries on your system will also work and you'll know for sure the libs are compatible. Let me know what you think.
|
Great ! I didn't look at your plugin, since I'm reading your post on my mobile phone. If you want I can help on making a 'buildable' plugin. I managed to compile opencv 2.3 as static libraries.
I'll look at it when I'll have a wifi Internet access !
Thanks for sharing !
Ben
Great stuff!
Ben, would you mind zipping the static libs up with the headers to share? I am in a similar situation and use version 2.2 but like to go to 2.3. For some reasons all I am getting are compiler errors. :(
Peter
OK, I'll do that this evening (it's 6 pm here in France). Do you know if it's ok with the OpenCV license ? Or I can send it by email.
BTW Peter, what are your compiler errors ? I had a lot. Seems that the Intel Compiler isn't OpenCV friendly :-/ Try with LLVM instead.
Same here... or could you zip it w/ static libs for me too? I'd like to run through the code, and check any "gotchas", help tighten it up (if it needs it... not trying to get too much ahead of myself). gtoledo3@gmail.com
Thanks for the awesome work. (Haven't tried it yet though!)
Updating OpenCV with install_name_tool was a pain in the ass. I had to use this script to get the OpenCV.dylibs I manually compiled as 32/64 bit to load from the Plugin Bundles Content/Framework folder:
Once you do that, you can run a Move Files build phase after to copy the Dylibs into Frameworks folder.
Now the plugin is self contained and be distributed so it links to its copies.
Beware, however. Other plugins loading in QC that use OpenCV, may have name space collisions, and you cannot guarantee which versions of the Dylibs are used in that case, unless you do some insane magic to swizzle all the function names, and thus break every-last-fucking-thing in OpenCV, so you get to go back in and re-name all the function calls. I think thats the only way to avoid those name space issues. Bleh.
Edit:
Additionally, that brings up the issue that there really ought to be one go to 3rd party OpenCV plugin setup. I really dislike how the Kineme OpenCV plugin works, to be honest. I have some stuff working, but its not as complete as others.
Thanks, I will give it another try. Bottom line, compiling OpenCV 2.3 or now 2.4 seems to make you loose sanity. ;)
If you like, please send me the static libs and headers to pbugaiski@gmail.com. I would appreciate it.
I feel you Vade, I gave up building dylibs because of all this hassle and switched to use static libs only.
Building the Dylibs with cmake was not so hard. It was more of a waiting game.
Its the integration into a project and allowing it to run from wherever that was a pain, but the install_name_tool tool script above should work. I used it with 2.2 and 2.3, no issues. Have not played with 2.4 yet.
What is it you don't like about it? It doesn't use the newer CV. It hasn't implemented some objects. The inherent structure of it isn't bad (to me).
Also, what does anyone think about CVGPU or GPUCV or whatever the heck it's called? Anyone looked into it?
Also also.
Blob tracking cool.
But does every darn interactive gag now have to run through a friggin 100 piece of plastic.
I say this as someone that really values what it can do, and will work with it with great joy... but ... I like the idea of blob tracking for QC being predicated on just plain ol blob tracking. (srry, only skimmed the thread, I'm sure I misread something).
CVGPU (CUDA support) runs only on NVidia cards. Unfortunately not an option for me. It would be very interesting though to hear some benchmark experience from others on the Mac.
Ohhhh , cuda. Eww, makes it totally useless then.
Didn't find in the license if I can share the libraries or not. Tell me if I can't...
Otherwise, you can download from the Box on my (very new) blog : http://www.benoitlahoz.net
So impatient to look at this plugin now, as it could change how I'll make my shadow tracking in the installation we'll open at the end of this week !
[edit : now, can't manage to build static libs of cvBlob. I can't find the right option (and don't know how to chaneg it in the cmake txt file) in cMake]
I've actually added the dylibs in the zip, so you can try the plugin by placing those in /usr/local/lib. It isn't pretty that way, but it should work on 64bit snow leopard. Or don't those work for you Ben?
Additionally you can simply download the openCV (and cvblob) source and build it on the machine you intend to use for the installation. The plugin should also work that way.
I'll try the install-name-tool script from Vade in a bit (thanks for supplying it!), see if that helps.
Any tips on how to build OpenCV/cvBlob for 32/64 bit? Or on using static libs instead of dylibs?
Vade- thanks for that.
this is how i compile open cv to work with plugins or apps.
ok some important things to mention. at this point you need to make a decision as to what architectures you want to build for etc... to make things simple lets assume you are on lion.
ok some more important info. you now need the include headers to be built. there are various ways to do this. one is to make a new cmake and select unix executable instead of Xcode and build them via the terminal. or you can use the Xcode project and select the build and install target.
this will try to build and put libs in your /usr/local/ directory. you will get errors probably due to permissions. just read the console log in Xcode it will tell you whats wrong. things like /usr/local/include directory not found or /usr/local/lib/bin/release not found etc... just make the needed directories and build-install again. repeat till you have all your include header files. copy them from /usr/local/include to a folder on the desktop call it include. inside it put the header files in their folder structure.
well you more than likely made this far your self.... now you need to make a new Xcode project that will be your app or plugin.
#import <opencv/cv.h> or #include "opencv/cv.h" //above objc define #ifdef __OBJC__ #import <Foundation/Foundation.h> #endif [code] 23. now you should be able to build your project app or plugin with open cv included in the package. just remember to make sure that choose the same deployment and architecture types as you built open cv with. meaning make sure to build your plugin as 64bit making sure to select the correct scheme. in the example herein using release scheme for the open cv libs now make your plugin project setting also the same. 24. viola i have not checked the portability of gpu-cuda or open-ni builds. i assume the process would be relatively the same as above mentioned. making sure to include the cuda and kinect libs in your app as well. hope this helps.wow, thanks for this very extensive post dust! I'm currently a bit busy to work on this unfortunately, but i'll definitely give it a go soon.
keep you posted
Hi Dust,
What version of openCV are you using and what compiler?
Cheers James
Hi!
Just tried your plugin today, and I noticed that it might have a memory leak. Memory usage went from 0-8 gigabyte in less then 10 seconds. It ate up less memory when adjusting the size divider, but still there.
Thanks for testing the plugin! It appears there was indeed a (rather large) memory leak.... Pretty sloppy, but i've fixed the leak and it should be much more stable now. Check out the new version in the attachment.
Unfortunately I didnt have the time to look at the including of OpenCV/cvBlob libs, and probably wont any time soon. If anyone is willing and able to fix that; please do and share it with the community. The source is (obviously) free for all.
Can someone re-up this as a plugin like the original? We don't have the libraries here to build it ourselves locally :/
Second this, has anyone got the complied plugin? Missing the OpenCV Frameworks here :(
Isn't the OpenCV framework available from http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.2/ ?