OpenCV Contours & Convex Hull 2 Structure plugin [DEPRECATED] (Composition by benoitlahoz)

Author: benoitlahoz
License: Public Domain
Date: 2011.10.19
Compatibility: 10.6, 10.7
Categories:
Required plugins:
Kineme GL, Carasuelo OpenCV

[EDIT 20120124 - UPDATED - please see this post now : http://kineme.net/composition/benoitlahoz/BlobTrackingPluginCarasueloOpenCV002a]

My first OpenCV plugin : it gets the contours and convex hulls present in the input image and output a structure of lines structure.

Very-alpha-version : it doesn't support for the moment a difference between the image size and the renderer size, but I'm working on it.

(BTW : i'm looking for a way to get the renderer size in pixels inside of my program... if someone knows how to do this :-/)

I didn't use the Canny filter which slows a lot the comp. I would be interested to know the performance on others machine than mine.

TODO :

  • Track blobs [EDIT : done in the new plugin, see below]
  • Delaunay Triangulation
  • Output Image (arghhh)
  • ...

Thanks to Mirek OpenCV plugin that helped me to understand how to convert the input image to OpenCV. Thanks to Mehmet for the threshold CIKernel.

Works in 32/64 bits

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

And the plugin itself...

PreviewAttachmentSize
CarasueloContourConvexHull.zip38.54 KB

gtoledo3's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Wow, this is excellent. Would you mind sharing the source? This is one I've wanted to do for a long time! Kudos. (haven't looked at the attachment yet). I think that it's probably possible to get canny working fast, and also, you might want to look at that threshold filter from the apple "combed trail.qtz" :-) thanks for sharing!

jrs's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

No Go, I get the following (in 32 and 64 bit mode under lion)

: Patch with name "QCPlugInPatch:ContourPlugIn" is missing

And I tried putting the plugin in both

"/Library/Graphics/Quartz Composer Plug-Ins"

and

"/Library/Graphics/Quartz Composer Patches"

gtoledo3's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Now that I look, it doesn't restore for me either (I'm looking in Lion)...

I think there's supposed to be a lib file in there, no? I'd expect to see something for the cv stuff, but maybe not.

monobrau's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Awesome work, that can be very useful! It doesn't load here though, "QCPlugInPatch:ContourPlugIn" is missing. Tried 32/64 mode, patches/plugins folder. I'm on 10.6.8.

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Oooopsss... In fact I didn't noticed that the file size was 90Ko.

In the new attachment, I included ALL libs, so it makes a 21Mo plugin file... Sorry for this. I may select which libs I'm using, but for the moment it might work...

@gtoledo3 : thanks for the "threshold" advice. I have my own GLSL Shader for thresholding, but I can't share it (licence problems) so I was looking for a good patch for the example comp. I'll look at the patch you're talking about ! Thx !

[edit : actually the "comb trailed" threshold is VERY good ! Thanks again !]

PreviewAttachmentSize
Carasuelo OpenCV_allLibs_included.zip7.34 MB
hand.jpg
hand.jpg109.79 KB

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Whereabouts is combed trail.qtz to be found?

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Same result here @jrs in Lion.

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

@cybero: even with the new post ? I'm on Lion too (10.7.2)

I found combed trailed.qtz there : http://www.f2dd.info/Developer/Examples/Quartz%20Composer/Compositions/F...

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Sadly, even with the newly posted version, whether in either /Library/Graphics/Quartz Composer Plug-Ins or /Library/Graphics/Quartz Composer Patches or in ~/Library/Graphics/Quartz Composer Plug-Ins or ~/Library/Graphics/Quartz Composer Patches in Lion, precisely the same result, the patch does not restore successfully. Looks interesting from what I can tell of what other postees have stated as to their immediate experience with this patch.

Thanks for the pointer to the Developer example mentioned by gtoledo & thanks to gtoledo for bringing that to our collective attention.

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

The plugin is aimed to go in the "Quartz Composer Plug-Ins" folder. I can't figure out why it is not working... Libs are bundled in the plugin...

[edit : here's a new version with the "install name" and "run path" configured... Sorry for flooding... Hope this will work]

PreviewAttachmentSize
Carasuelo OpenCV.plugin.zip7.34 MB

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Sorry to report that the plugin doesn't load in Lion.

I did think that the Plug-Ins rather than the patches folder was the place to put you plugin, but had tried the other folders just in case they might work.

Is there a dependency that needs to be satisfied client-side?

monobrau's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Still no luck in 10.6.8

this is what the console say's

20-10-11 12:51:20 Quartz Composer[61883] *** QCPlugIn: Cannot preflight plug-in at path "/Library/Graphics/Quartz Composer Plug-Ins/Carasuelo OpenCV.plugin" (Error Domain=NSCocoaErrorDomain Code=3587 UserInfo=0x117818df0 "The bundle “Carasuelo OpenCV” couldn’t be loaded because it is damaged or missing necessary resources." (dlopen_preflight(/Library/Graphics/Quartz Composer Plug-Ins/Carasuelo OpenCV.plugin/Contents/MacOS/Carasuelo OpenCV): Library not loaded: /Users/benoitlahoz/documents/ObjectiveC/cvblob/lib/Release/libcvblob.dylib Referenced from: /Library/Graphics/Quartz Composer Plug-Ins/Carasuelo OpenCV.plugin/Contents/MacOS/Carasuelo OpenCV Reason: image not found))

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Thank you for the Console message ! Actually I WAS using cvbloblib. Didn't notice that a reference was kept in the config.

Does it work now ?

PreviewAttachmentSize
Carasuelo OpenCV.plugin.zip7.34 MB

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Sadly, no. Thanks for trying out a new build though :-).

32 or 64 bit QC 4.5 [10.7.2] the following is obtained with the plugin in either Plug-Ins directories.

> (null)
<QCNodeManager | namespace = "com.apple.QuartzComposer" | 918 nodes>: Patch with name "QCPlugInPatch:ContourPlugIn" is missing
 
> Macro Patch
Cannot create node of class "QCPlugInPatch" and identifier "ContourPlugIn"
 
> Macro Patch
Cannot create connection from ["outputHull" @ "PlugInPatch_ContourPlugIn_1"] to ["Structure" @ "Iterator_2"]
 
> Macro Patch
Cannot create connection from ["outputHull" @ "PlugInPatch_ContourPlugIn_1"] to ["inputStructure" @ "StructureCount_3"]
 
> Macro Patch
Cannot create connection from ["output" @ "Splitter_5"] to ["inputMinThresholdValue" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["outputPixelsHigh" @ "ScreenInfo_1"] to ["inputHeight" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["output" @ "Splitter_9"] to ["inputThreshold" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["output" @ "Multiplexer_1"] to ["inputImage" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["output" @ "Splitter_8"] to ["inputMaxArea" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["outputPixelsWide" @ "ScreenInfo_1"] to ["inputWidth" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["output" @ "Splitter_7"] to ["inputMinArea" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["outputContour" @ "PlugInPatch_ContourPlugIn_1"] to ["Structure" @ "Iterator_1"]
 
> Macro Patch
Cannot create connection from ["output" @ "Splitter_6"] to ["inputMaxThresholdValue" @ "PlugInPatch_ContourPlugIn_1"]
 
> Macro Patch
Cannot create connection from ["outputContour" @ "PlugInPatch_ContourPlugIn_1"] to ["inputStructure" @ "StructureCount_1"]
 
> (null)
State restoration failed on <QCPatch = 0x6d11ec20 "(null)">

ditto on the Console message posted earlier.

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Aarrghhh ! :-)

I can't understand why it doesn't work for you... It works very well for me. Now, the plugin bundles all the libraries...

Unfortunately I only have one Mac. Could it be that QC considers it unsafe ?

[edit : new build (may be cleaner) attached. Thanks a lot for taking time to test !!]

[edit2 : I've understood. QC is searching the OpenCV libs in another place... I have to find why...]

PreviewAttachmentSize
test.jpg
test.jpg606.36 KB

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

LOL :-). Thanks for the bold type caveat. Glad to hear you are making good progress though in understanding the root cause of the malaise.

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Ouch... In fact I was trying to link dynamic libs and the plugin was searching the path on my computer.

Here's a new version with static libs I had to recompile. It might work (I hope !!!) Please tell me, so I can sleep next night :-)

The plugin is 10Mo... Oooops...

Please, consider that - as I didn't manage to have an image output - I couldn't test the CV Threshold so if you want a precise result, you may want to use a "pre-threshold" for your image.

Next step : meanShift for tracking...

PreviewAttachmentSize
Carasuelo OpenCV.plugin.zip3.36 MB

monobrau's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Works!! on 10.6.8 and in 32/64 bit mode. This is the plugin I was hoping for a long time, thanks for developing and sharing!

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Success at long last. May you enjoy your well earnt slumbers Benoît :-). The result I get is well and truly spooky with the way the contours of one's face appear above or to the side of one's head.

Anyhow, a picture speaks a thousand words ... please see attached

[10.7.2 - 64 bit QC 4.5]

PreviewAttachmentSize
CarasueloOpenCV_ContoursConvexHull.png
CarasueloOpenCV_ContoursConvexHull.png44.45 KB

Peter's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Thanks, this is an awesome plugin! :)

I think you might don't need to implement mean shift, I just implemented the camshift function in my small OpenCV project, which I will upload very soon. (I didn't have the chance to create good examples yet).

It comes with a full blown camshift implementation and kalman filters to stabilize and predict the sensor data.

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

@cybero : Alleluïa !!!! ;-)

@Peter : great ! I'm impatient to try your plugin !

[edit : @cybero I'll have to manage with the difference between renderer size and image size. Do you have a tip to get the renderer pixels bounds in XCode ?]

[edit 2: the new version renders at any dimensions, as it scales the image input, please download (once more :-/)]

PreviewAttachmentSize
Carasuelo OpenCV.plugin 2.zip3.36 MB

cybero's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Now that Benoit has his plugin working on both Snow & Lion I was wondering @jrs, if you found , as I have done, that the use of video input on Lion in QC is a trifle like use once but don't stop and restart, as the composition [any composition, BTW, not only any based upon or incorporating this brilliant new plugin of Benoit's] will hang, even crash upon restart?

benoitlahoz's picture
OpenCV Contours/ConvexHull/Presence unleaked

Hi,

Here's a link to download the new unleaked version of my OpenCV Contours/Convex Hull plugin. I added a "Presence" plugin that gets only the number of contours in the image without outputting a structure. It result in very fast "presence detection".

I had huge leaks in the previous plugin. This one is far much cleaner !

http://www.benoitlahoz.net/?p=15

gtoledo3's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Do you have a qtz setup for this one? I can't get it to not crash... I'm just feeding a 640x480 image, w/ 640x480 specified, and outputting the structure a gl line structure patch (first crash), and then the enable on a Billboard (2nd crash).

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Hi,

I can't understand why but it seems that the plugin doesn't take still images as input. Only movies. I don't know if this is your setup.

Could you post a console log if you have one, or your qtz ? It works really well for me, so I'm interested by feedbacks.

Merry XMas !

Ben

gtoledo3's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

This was with the video player. Just hook a video player to the patch, or the kinect. Then, hook the output of your patch to a billboard (so that you can try to monitor what it outputs). I don't have a composition handy, didn't save it. If I try it again, I'll try to send you a log.

Do you have a setup that is known to work (re:, question above)?

franz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

crashing here. too bad b/c the plugin seems quite fast. 60FPS when downsampling to 320*240, but then crash....

keep up the effort ! merryxmassss

gtoledo3's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

I'm just seeing this... glad you found that threshold in the combed trail example to be nice, it's one of my favorites :-)

benoitlahoz's picture
Re: OpenCV Contours & Convex Hull 2 Structure plugin ...

Hi,

Here's is a sample comp and the plugin itself, as I don't know which version I posted here... Thanks for your feedback.

FYI I'm working on 10.7.2 and the plugin is 10.6 compliant.

I have to work on the OpenCV threshold as it could be much better. Next version will provide an adaptive threshold as a test (and the Delaunay constrained triangulation, which is now working but crashes sometimes without any apparent reason).

I would be glad to see what you can do with it !

PreviewAttachmentSize
OpenCV&comp.zip3.51 MB