Blob Tracking Plugin (Carasuelo OpenCV 0.02a) (Composition by benoitlahoz)

Author: benoitlahoz
License: Creative Commons Attribution
Date: 2012.01.23
Compatibility: 10.6, 10.7
Required plugins:
Kineme GL


A new plugin in the Carasuelo OpenCV bundle : Blob Tracking. I implemented the cvblob library and externalized a lot of parameters to prepare the image, and to set the tracking accuracy. It's supposed to work with 10.6 and 10.7 in 32/64.

It outputs a structure of blobs with coordinates, angle, and tracking datas (has been active for x frames, has been inactive for x frames, exists since...).

An image output is available, with the possibility to output the intermediate image. It runs quite fast on my MBP 13" even when the frame isn't resized (640x480).

The 3 example comps (Blob Tracking, Contours and Presence) are documented. I have to work again on these 2 plugins to make them fit a CVImage class I wrote to simplify OpenCV use.

There's a big issue on the image dimensions, as I have a trouble with "special" width on input images, but in most case it should be ok for everyone. I'm working on it, but if someone has an idea...

Please tell me if it works for you !

I would be glad to see what you do with it.

cybero's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Sad to report this, but on Lion, in either /Library/Graphics or ~/Library/Graphics Patches or Plugins folder the new Alpha version fails to load.

Computer says

 Patch with name "QCPlugInPatch:BlobTrackingPlugIn" is missing

and it says that consistently whether running QC in 64 or 32 bit mode.

3lab_VJ's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

sorry... also sad to report this, but on MBP 15" / SL 10.6.8, the same problem. by the way, thanks for developing such great plugins.

benoitlahoz's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Sorry for this new mistake in the linking.

Thanks to this vade post : I was able to link the dynamic cvblob lib (which I was unable to compile as static) to the plugin contents resource folder.

Here is the new build (updated in the first post), tested on another user (which doesn't find Kineme patches in the Patches folder BTW)...

It has to be copied in the Plug-Ins folder (not the Patches one).

:-) And thanks Cybero to always be so quick to test !

cybero's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

On first boot up of QC 64 bit, with your new bundle installed, I found that the Blob Tracking Plugin needed to be copied into /Library/Graphics/Quartz Composer Plug-Ins to load correctly , which it did not initially do from ~/Library/Graphics/Quartz Composer Plug-Ins. However, having obtained success from the main Library folder installation, I was subsequently able to load the example compositions successfully whether the plugin was installed to either Quartz Composer Plug-Ins folder.

One other weeny snag that could cause the composition to bounce out is that the path used for the Movie Importer in the Blobs example composition is ~/Documents/QTZ/PROJECTS/OPENCV/blobs_img_01.mp4 whereas /blobs_img_01.mp4 works fine.

Unfortunately in Lion, due to an issue I've been advised is pretty specific to my iMac's build version, the Movie Importer patch doesn't work in either 64 bit or 32 bit and although I can work around that by using the Kineme Video Player in QC 64 bit, I can't obtain the same convenience in 32 bit.

I rather think that is not a 32 bit Lion issue with your plugin, but it still might be worthwhile looking into.

Suffice to say that in Lion QC 64 bit, your plugin works , thus far, very well indeed.

lookitscook's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Blob demo works great on i5 iMac and air. Contour demo pegs cpu on the i5 and crashes, just crashes on air. Also, sorry to be naive.. but what is the presence demo testing for?

benoitlahoz's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Thanks @cybero for your returns and comments. Terrible habits I have with absolute paths in my comps :-/

Here it's running quite well in 32bits (Lion 10.7.2, I guess you're on the 10.7.3 rc, aren't you ?). It has to, because VDMX doesn't recognize 64 bits plugins and we use it a lot for our works.

My Kinect is away so I didn't test it. It would be of great help to have some returns on this point, with pics eventually.

@lookitscook : thanks for your returns too. The presencce plugin is testing the number of contours (blobs) without performing them, to gain some CPU. I'll make a preview output for this one too. We're using it in an installation to test if someone is entering (against the light, so we have a blob) in it. You might want to perform a background substraction on your input image. I'm working on a frame differencing one, but that's really not ready.

I'll test once more leaks in the contour one.

monobrau's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

I'm getting a steady 60fps and stable blob tracking with a kinect on a top-down setup. On a 2.6 c2d macbook pro and SL. Very happy bunny so far!

cybero's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Just to confirm benoitlahoz, yes I am testing your plugin on the rc distro.

I'd still say that the Movie Importer | Video Input patch problems I referred to are probably iMac build version specific.

FPS numbers on the Blobs composition are [weighted average] between 75 to 140 fps, Presence gives 9 to 13 fps whilst Contours2Structures gives 12 to 15 fps [using built in iSight].

benoitlahoz's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Wow, good news for the 75-140 fps ! But I definitely have to work again on the 2 others. I think this version of Contours is not too bad for the moment, but Presence was made too quickly, because we needed it "now".
The fact is that it depends how your input image is prepared (threshold, smooth, approx), then how precise you want the contours. The more precise they are, the more computation you need. A very slight smooth is performed before thresholding in the plugin, but I may externalize the param.

What I should do too (after reading this vade post) is to give an option to output in Pixels or in Units, it might be useful.

I would really appreciate if someone pointed me on a downloadable video of the Kinect Depth image to make my tests, as mine is far away from my place. :-)

jersmi's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Hi. Exciting work here.

Unfortunately I cannot get the plugin to load. I'm on a MBP, Lion 10.7.2, tried QC in 32-bit and 64-bit, tried ~/Library/Graphics/Quartz Composer Plug-ins and /Library/Graphics/Quartz Composer Plug-ins.

(null) : Patch with name "QCPlugInPatch:BlobTrackingPlugIn" is missing

cybero's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

I found that I got that message, searched my folders and found that I had double installations of the plugin - one old version and one a new version. I uninstalled both, reinstalled the newer version and all was good to use. See attached proof of pudding pictures.

Might be worth a try.

An aside -

For me, such patches as tend to use either Movie Importer or Video Input still cause freezes and / or crashes in Quartz Composer 4.5.

Wish that would get sorted out, but that's definitely worth a different thread and not either kineme's nor this plugin author's fault, thus worth a thread on a different forum [such a thread has already been posted along with bug reports, BTW].

OpenCV_Blobs.png85.78 KB
OpenCV_Presence.png330.18 KB
OpenCV_Contours2Structure.png355.38 KB

tarrabass's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Blobs and presence works fine, but contour structure crash on 10.6.8. to sad...

franz's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

tiens, tu te mets a quartz maintenant .... hahaha

tarrabass's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

Ouuh, toi tu m'énerves ;-p

benoitlahoz's picture
Re: Blob Tracking Plugin (Carasuelo OpenCV 0.02a) ...

@tarabass J'ai à nouveau un peu de temps... Je vais regarder ça... Pourrais-tu m'envoyer ce que te dit la console, ou bien détailler le crash ? Merci !