Recent comments
Recent topics
Recent wiki edits
 |
OpenCV PatchesHi, I know this is a hard one... OpenCV stands for Computer Vision. It is an open source framework (check it out on sourceforge: http://sourceforge.net/projects/opencvlibrary or here: http://www.cs.cmu.edu/~cil/vision.html ) that allows for all the kewl stuff you might like: - Face detection (using Haar cascade algorythm) - Color Tracking - Feature tracking - Motion detection and so on.... Currently, this framework has been adapted for other graphical programming interfaces, such as: - Max Msp Jitter with CV.jitter lib (here : http://www.iamas.ac.jp/~jovan02/cv/ ) - VVVV with specific nodes (mostly freeframes). I could use these two apps to get the features, but: VVVV runs on windows (and even with boot camp, is NOT as fast as QC in terms of video) Jitter is damn slow, even with the new UB. Right now, apart from ReacTV (TUIO) and its fiducial trackers ( http://mtg.upf.edu/reactable/?software), no lib exist fro QC to allow for some image interpretation, so i think porting openCV to QC would be really interesting for a lot of ppl.
|
This would be so incomprehensibly cool :) but also incomprehensibly difficult... hmmm... this looks like fun.
After playing with the samples for a bit tonight, it doesn't seem quite as incomprehensibly difficult (though still pretty close :)
I think it'll be patterned after the Texture Patch. That is, it'll have patches for the various functions, so you're responsible for chaining together useful stuff.
Are there any hot functions that are really useful to focus on? (There are a few hundred functions, so I don't think it would be wise to make a patch out of all of them.... many also appear to overlap with CoreImage filters, so those probably aren't as immediately necessary either.
Thoughts?
HaarCascade
I think you are insanely cool. The HaarCascade / aka FaceRecognition is fairly the most impressive function avaiable in openCV. Any pattern recognition/learning is awesome. (Automatic) feature tracking is also uber-great. StraightLine finding/Highlighting is nice. OpticalFlow to RealtimeImageStabilization seems more custom. CameraMatching is overkill (do they bundle this one ?)
I've not played with HaarCascade yet, but when I saw feature tracking working in real time, in crappy lighting, with an iSight, I almost fell out of my chair (and the tracking points followed :)
With face recog, you need to train it I think (still scouring the docs), so that might be impractical at first (it'd need to save state to be useful on the go). Line finding is cool and pretty simple.
The camera orientation stuff looks pretty cool (give it reference points, and it calculates the camera's orientation), when combined with ... other 3d stuff ... ;)
Face Recog
Training files for FrontalFace and ProfileFace Recognition are already included in the download bundle. You only need to train it more if you want custom object recog.
basically, training files are just XML files.... Look for HaarCascadeFrontalFace.xml
I'm glad that you see my point about camera orientation... and the earlier discussion we had about the need for cameras...
shortly after my previous post, I discovered the xml file loading aspect, and the default recognition stuff (there's even a demo that does it :)
Do you say that it can do 3D tracking in real time ?...:) yanomano.
Some of the demos I've seen can take a known object (like a chessboard), and determine how the camera is oriented based on how the chessboard appears in the image. Since a chessboard has many corners, it's probably a much simpler case than normal objects, but it's a pretty cool trick nonetheless :)
2D tracking with dozens of points is fully real-time, which was enough to impress me :) Makes the wii-mote IR stuff look rather weak :(
yes.....
http://openvidia.sourceforge.net/skeltable8.jpg it is not really 3d tracking, as PFtrack-like sotfwares (14000+$) would do (though one can dream.... it will be possible one day, sooner than expected), it is called 3D registration. Have a look at openVidia, which is an OpenCV port on the GPU, this will blow your mind http://openvidia.sourceforge.net (didn't i post these links already ? It seems i'm nearly the only one being totally wet about openCV)
or more generally, there: http://www.cs.cmu.edu/~cil/v-source.html
and you'll see that openCV, more than being opensource and FREE, is also used in amazingly specific tasks, such as fimage-based skin cancer finding.
Remember EyeToy on the PS2 ? .... openCV I've lost the link - i'm a messy guy- but openCV has already been embeded onto ArmProcessor (system on a chip) to make experimental vision enhanced mobilephones.
This is more personnal, but here's what i did using openCV (and vvvv) : http://www.exyzt.net/tiki-index.php?page=SPPEGX3T (sorry but the blah-blah is in french...)
note for cod3rs: have a look at the very bottom of the page, in case you missed it: GPUcv, which is actually ported on window$ only. Don't you think it might be a good thing to already from the start consider using GPU only libs (to my experience, the openCV routines i used on PC where wrapped around Freeframe DLL's, so they were running exclusively on the CPU, and it was quite slow - around 50% framerate drop)
Without any working OS X GPU libs, I think I'll stick with the CPU-bound ones. The cool part about libraries is that you can swap them out for new ones, and the code that uses them doesn't need to change (much, usually).
Porting Cg/HLSL to GLSL (and fitting within the QC environment and getting reasonable performance) is a lot of work, so I'll probably not get into that in the near future... :) Definitely a cool goal/dream though.
understood
got it.
one day...
so i have been waiting for the new revision of openCV. hopefully soon. in the mean time if you want to do haar cascade head tracking, have a look at this patch. its using a java, open cv, osc, and kineme 3d. nothing special just a simple head tracker using the standard xml.
thought some one might want to mess around with it until the revision comes out. its running thirty + frames a second and the poly count is something ridiculous like 2 million, which is larger than it ever really needs to be unless your making pirates of the Caribbean or something. just open the face_detect2osc file and the headcv file and mess around. like i said you need kineme 3d for this patch to work but, you could just add a osc receive and use the name space /x and /y on port 8000 and do what ever you want with the floats. they go from 0-256, so you don't have to use them as head tracking coordinates. have a look at the pic if you want.
merry christmas, oh sorry about the models size, remember less is more....
OpenCV is actually already on my (personal) to-do list and I hadn't gotten around to adding it on kineme.net yet.
Yes. Let's do this.
Feature tracking on the GPU... agressive speed
Did you know about this one:
GPU-based Implementation of the Kanade-Lucas-Tomasi Feature Tracker using OpenGL. http://cs.unc.edu/~ssinha/Research/GPU_KLT/
And this one: http://openvidia.sourceforge.net/
Seems promising...
more on reactable's toolkit
beyond reactable itself there are are simple but brilliant things like "everybody can be wireless artist". [a meta-moment: i just searched for some links to it, and found myself in the video, playing with it at sónar earlier this year]. the video doesn't quite do it justice, but such is life: its a live performance thing.
http://www.youtube.com/watch?v=k6XkP7VggxE
to my understanding, they downloaded the binary, edited its xml file, and the rest was done in ableton live. which ain't bad for "enabling technology".
this basic approach of searching for known symbols and outputting corresponding key'd trigger is a powerful one that i think would be good as a starting point in qc's search for computer vision.
reacTV alreary for QC
hey toby, reacTV as already been compiled as a patch for QC see here: http://www.tudra.net/wp/wp-content/uploads/2007/01/tuioclient_qc_0_2.zip... and the performance is not that bad...
But OpenCV is really better (i think) because its libraries are wider. Algos inside are rock solid and developped by intel. It has all the neat features one would like for, say, control a robot-like installation with QC (=Phidget Patch+ WiimotePatch+ OpenCV Patch...hehe)
ye gads. how did i miss
ye gads. how did i miss that. trés bien, merci.
vs GPU
u welcome... btw... all of these are pretty slow compared what can be done on the GPU these days.... so bad the GLSL scene on the Mac is that weak (and still emerging) compared to the DirectX scene (and Cg).... i'll be more than happy if someone has a few links about that subject...
feature tracker
it seems that sam kass is developping a feature tracker (as seen on QC dev list): http://www.samkass.com/blog/page2/page2.html plus, have a look at the optical flow leopard sample plugin... let's hope things will be moving fast the next months
reacTVision on leopard
fyi i tried using reacTVision on leopard but couldn't get it to talk to qc properly, despite in theory having the right osc paths etc. i had a look at recompiling that plug-in for leopard but thats well above me. so we'll see where i get...
toby
(disclaimer: I know absolutely nothing about OSC)
From the discussion on the mailinglist, there may be some problems (?) with the current OSC patch when paths are complex. I don't know enough to know the details, but there have been some interesting test cases in the past week that might help you out?
which reminds me, my mailing
which reminds me, my mailing list subscription seems to have stopped while i was away, and that is definitely something that needs sorting out!
QCdev vol32 issue 35
it seems that QC osc doesn't support hierachical keys. (at leat that's a question, I couldn't find what was the answer when browsing my archives)
Face detection would be wonderfully fun, especially for iChat filters. Reactivision is great for what it is, but is by design limited to position and rotation, motion is generally lost.
one very interesting link
https://picoforge.int-evry.fr/cgi-bin/twiki/view/Gpucv/Web/WebHome
open CV on the GPU ... only for window$ right now, but still interesting
I was just checking this out, and noticed that they're using OpenGL/GLSL (not HLSL like every other windows CV project I've come across). So porting this should be way easier than crafting GLSL from HLSL/Cg and all that jazz. I'll definitely be looking more into this. Thanks for the link.
source
Yeah, i didn't want to be too "pushy". They also provide the source, that seems to be platform independant and very understandable .
Ok, after 2 days of poking around, I've got feature tracking working, along with a couple other minor features (camera input, opencv image -> qc image, opencv rgb -> opencv grayscale for optical flow, etc).
Working on a few more functions, and some performance improvements... OpenCV, being very image-based, has its own tightly integrated image types. I'm not sure if I should spend time making wrappers to turn them into QC Images, or if I should keep them CV images for better performance (the screenshot show 12fps, which is a bit lower than the normal 25-33 for opticalflow + cv-camera to QCImage display). Right now I'm planning on keeping them all CV images for performance, but it's not as intuitive if you don't know that they're not QC Images.
A quick example of what people have done with 'feature tracking' type effects. It might not look like much from a still image, but everything here is actually tracking a video.
It starts off being very hard to work out what the video is, but gets more and more obvious. Stuff like this should be possible in QC with the CV tools. Keep up the good work!
curiosity
just curious: why would you need to turn CVimages into QCimages ?
As Cvimages could be used to get the data from, while the QCimage coming from VideoInput could be used into an independant CIchain to be graphically processed. Are you using a CV function to display the features on a CV image, or are you feeding a sprite iterator with data coming from a CV structure ?
psonice: this demo is awesome, so stylish !!! Please, keep these links coming !
Right now, in the screenshot above, the camera data comes from OpenCV's camera capture functionality (not the Video Input patch). So the whole processing chain uses OpenCVImages rather than QCImages, except where I needed to convert to a QCImage for the billboard in the background.
Converting between the two (QCImage, CVImage) appears to be unnecessarily expensive (I probably don't understand QC's ImageProvider class, which may provide a cheaper solution to this in the future), so I only perform that one conversion.
If they get to be transparently compatible, CI Filters could be used in place of CV filters for some powerful image filtering paths.
CVdriven GLsplines
first tests.... (testing the performance actually)
and some more...
here:
chris (+franz), heroes amongst men
sooooooo excited to see this coming to fruition.
i concur... can't wait to mess with it
Just released the first beta yesterday: http://kineme.net/OpenCVPatch20080207beta
Access Denied
Hi, why do I have an "access denied" message to that URL even if I'm logged in? Isn't it possible to access the patch anymore? Thanks, Andrea
The OpenCV patch is still in beta, so you need to check the "I want to be a Beta Tester" box in your user profile.
Thanks a lot. Sorry I'm new
Thanks a lot. Sorry I'm new to Kineme. Andrea
Not a problem :) The bounce page ("Access Denied") isn't very helpful, I know. it confuses lots of people... stupid drupal...
I just added a comment about betas to the Access Denied page.
i have tried to access beta openCV ??? i have checked beta test etc... ????
You'll have to check the "I want to test new and upcoming patches" checkbox in your account profile to opt-in for Beta test access.
i would really like to have some openCV patch plugins for quartz composer. willing to run some haar training examples for you ?
would like to test this...i know it is out there i have seen it...have haar training experience will trade ?
This is starting to look good (which I guess means it's not far off being ready :)
What is the performance like, and how does it look realtime? Things like this usually look pretty random if you turn off the background and just keep the lines, until you see it running and the pattern becomes clear.
video
it is around 8/10 fps in realtine / 200 features being tracked at max. (MBP 2Ghz / X1600)
http://www.exyzt.net/x_video/lab/DataSpace_InProgress.mov
Looks a bit different from how I imagined - the feature tracking is much more accurate than I expected. Is it possible to pick points along the edge of a shape (so that the splines would follow the pattern of the video)? Something like the picture I attached would be amazing - obviously that's way beyond what's possible with this kind of software, but I think it should be possible to do a similar effect.
And how about performance? Is it possible to boost it by using less features, or by optimising the code more? Although I think 8-10fps is probably acceptable at 2ghz - it should be fluid enough, and you can give the impression that it's running faster by overlaying effects running at full frame rate.
I'm very interested in putting this to some use btw, any idea of when a release will be available? :)
it took a performance hit when I switched from the 1.0 release to the CVS version of OpenCV (not what I was expecting). It's a chore to get it to build for both ppc and intel, so I'll probably not switch back for a bit. I've also spotted a few tweaks that might help a bit (redundant colorspace conversions in the Mac code), but they'll make it non-standard compared to normal CV. It also won't make an earth-shattering performance change either :( cv's remarkably slow.
I think the most promising route for 30+ fps would be to change it to GPUCV, though that's still a bit tricky (and having an intel graphics card will make it difficult to see actual performance increases :)
Franz, could you post the filter chain you're using, along with the parameters. As psonice noticed, it's really accurate, so you may have some of the tracking settings set to "Slow But Good" mode, if you know what I mean :) (there are lots of input params to control how it works)
settings
CVgood points: 200 quality: 0.01 minimum distance: 20
window size 20 iterations 18 epsilon 0.03
At 100 features and 15 iterations, i'm almost @12fps, which is quite nice. I'll also try resize the CVimage to 240 or 160 pix ...
Having the CVsilhouette node would help locating good features only on outlines.
www.exyzt.net/x_video/lab/DataSpace_InProgress-01.mov www.exyzt.net/x_video/lab/DataSpace_InProgress-03.mov www.exyzt.net/x_video/lab/DataSpace_InProgress-04.mov
The tracking capacity is really impressive... and the precision is awesome... Your composition too frantz ...:) yanomano.
careful with resizing (as you've probably noticed, it doesn't like image size changes yet .. heh. unplug, change size, plug back in). Did you notice any loss in tracking capacity with a lower resolution?
I've found that removing/disabling the CV->QC image patch will raise me from 9 fps to 12ish fps.
Oh yeah: HoughLines! (generally a bit faster than feature tracking, but this screenshot says otherwise)
resize = crash
really... i just don't manage to get resize working...
Can HoughLines be used to help define good tracking points ? (should be)
I would think so.. If you feed it an unfiltered image (no canny filter with really high thresholds) it goes horribly slow (1.7fps... meh), so it's difficult to get several good points out of it, but the ones it gives are probably very good quality ones (image corners that are likely to be backed by real corners in 3-space)
[EDIT: totally sorry about the resize stuff. unplug whatever the resizer will plug into before connecting it. a disconnected port forces all the patches to flush their temporary memory, but they're not smart enough to do this when the input size changes (resize was a last-minute addition, and before that resizing was Not Possible... starting to sound like Pierre now :) I'll address that before the next alpha]
Thanks to some brief profiling by psonice, we noticed that simply taking CV Camera input and putting it on a billboard was unspeakably expensive and slow (thus limiting all other patches). I've implemented a QCImage -> CVImage patch that lets us use the built-in Video Input patch (which uses a much faster, multithreaded video capture method).
This has improved performance for me by maybe 20% (your mileage may vary), and allows us to use CIFilters instead of porting every OpenCV filter (a half-step towards using GPUCV). Alpha testers can expect another version sometime tomorrow (cleaning up a few other pieces). Also good to note: while the framerate may not be radically higher, the latency is dramatically reduced -- much less lag, which makes it feel way more fluid.
...for a couple of reasons (apart from the obvious performance increase :)
It'll be possible now to tune the input image with the standard QC filters so that the desired features are more easily tracked. Especially good if you want to track say an outline but not features within it, or features of a certain colour.
Also, this means that it's not necessary to use video as the source - you could use a QC composition instead.
Big thumb up! =)
On a side note, how do you capture a composition to save it out? Is there a tool on the mac, or do you just dump frames from within QC somehow?
capture
use Kineme GL read pixels / or render to texture, then use the Movie Exporter plugin.
Match shape outline
Awesome topic.. bwt im currently working matching by shape outline(regardless of size) to images using opencv.. sadly, the guide for haar training, which comes with opencv package, aint specific enough... i'll share the project when its done :)
Cheers Juz
open CV's latest
http://www.youtube.com/watch?v=NNVaU2eOcbs head tracking...
points coordinates
Anyone nows if/how i can get the points coordinates from the OpenCV Calc Optical Flow Pyr LK patch, i´am trying to make something move in the screen with my head :).
the output is a structure of X/Y points, so just use Structure Index Member to extract points, and Structure Key Member to extract X and Y data from those points.
Thanks
ahh... I have try do did the first part but not the second :).
Thanks.
camera correction ... ?
as far as i remember, camera lens correction is part of the OpenCV suite.(?) I recently ran into such problems, aka lens correcting a camera input to have horizontals and verticals straight lines. I currently use a pinch filter (built-in) to undistort camera image, but it is far from being precise. Are these options still part of the openCV framework, and if so, would you ever consider porting them to QC ?
It's still in there -- I've considered adding camera correction patches, but the patches take a matrix input, and I'm not sure how to obtain a correction matrix, or how the QC interface should be structured to accept one (do people build them by hand?)
matrix ?
strange... commercial packages (like PFtrack for instance) usually have a "Barrel" / "pin cushion" param. that you tweak till verticals appear so. Or you know your camera and just enter the Lens type / filmback values.
Hi! Does this "OpenCV haar detect objects" work? I could not find any example. I tried it... using a haarcascade_eye.xml as a cascade. But I could not return nothing!
Can you tell me about that? Is the output "rectangle" a simple structure?
thanks Marco (Italy)
In this beta, it's broken, and you're lucky to get any [useful] output. In the next version, it works. There'll be an example composition to demonstrate how to use it as well. Its output is a structure that contains all the matching rectangles.
nice
... I just wanted to ask the same. I get an output, and I've achieved somehow to correlate the apparently random values to more or less the actual coordinates. However, without great succes. Excited about this next version ;)
Regards,
Rob
Hi, Open CV patches seems to be one of the best project for Kineme's members. 3D Object Loader will be release near Christmas if i understand ? Would you plan something for OpenCV ? I understand you've got a lot of work, i just want to know if there will be an update or a public release in the next future. I'm working on two project with your OpenCV patches (Beta), and it's really the most impressive patches that we could add to an "interactive" software like QC... I'm waiting for this update...
Thanx for all.
If you checked off the beta tester box in your preferences, when you log in, you should see the beta testing tab in your upper left corner. So, there you will find the OpenCV patch... soooo it is definitely available if you are on the beta testing program. Just fyi... :o) ....
EDIT: Whoops ....you are asking about final release, in which case, excuse me for butting in !!!
Hi qtoledo, Yes i'm working with the beta, i'm on projects with those promising patches. I just answer if CWright plan a new release coming soon because i'm deathly addicted to CV stuffs... Maybe a new beta release with more patches, or increased performance... Maybe the first public release (stable and documented)... I expect a lot for this project (like the 3D importer and Speech recogntion...) I just need some news...
The next beta of CV (probably early January?) will feature a lot of performance improvements, and some bug fixes (haar detection, primarily). There are a lot of potential patches, but no one seems to want to tell me what they'd like (I've got a request for camera correction, but that's very complex and requires the user constructing transformation matrices, which is complex for the user as well). If you drop some idea, I can see what I can do. If you say "blob tracking!" you had 1) better know what you're talking about and 2) know how to explain it, so that you can teach me enough to get it working (I've never seen any working blob tracking demos with CV, so I don't have any reference points to know what I'm trying to accomplish)
Alternatively, there are a lot of other CV-like libraries out there as well. Some are mentioned on this site. It would be nice to integrate those as well (eye tracking, occluded face detection, etc). Please post URLs, licenses, and what features you'd like from those libraries -- makes my job easier :)
Please Please Please don't say "openFrameworks!" ;) (I Hate oF, after poking at it some... what a nightmare) [you can request features from it, but make sure it's implemented somewhere else too, so there's a coherent code base to work with]
blob tracking !
Blob tracking please ! jk
For an overview of OpenCV's functions, i would suggest having a look at: http://www.iamas.ac.jp/~jovan02/cv/
it is the implementation of oCV for max/jitter, coded by fellow frenchman Jean-Marc Pelletier. Then, i would also suggest to install Max5 demo, then install the openCV external and have a look at the examples. They are very well documented, plus you have sample patches showing what the functions actually DO. However, this port is also very unstable.
More generally, openCV functions are of 2 types (to what i understood): - image manipulation (thresholding and such) : these can be reproduced with QC native patches. - image analysis : returning an image (like optical flow) or returning values (like blob tracking: position, size and number of blobs, center of mass, pixel area...)
Image analysis are the only functions i'm personnally interested in. So far, the openCV KnM port is practically unusable (you'll say "of course since its a beta", and you'll be right) due to: - poor speed - hickups in specific plugins like:
Tracking / unability to toss out points / unability to sort points (never managed to , at least) Face Recog. / buggish and very jittery Hough lines / unability to correct camera, so finally, even if the plugin is working correctly, there aren't any straight lines in a cam feed.
However, i'm pretty sure a 1.0 release is close and doable, since the plugin you developped is already working well in some specific situations. OpenCV is a great addition to QC, and i'll never be thankfull enough.
Thanks for the directed feedback (some of which you've already mentioned elsewhere) :) And for the link -- opensource, very handy, and it lists a bunch of features (so I know what's useful/what isn't)
are you basing this off the the public beta, or the unreleased version (not sure if I sent that one to you or not...) -- should be faster, and have working face detection. (and I keep saying I'll release it "soon", and never get around to it... :/) point-tossing might be in there too...
How do you envision point sorting working? (That's not a feature of cv, but it's trivial to do ourselves -- I just have no idea what's needed)
As you noted, camera correction is a pretty big problem, and I have no idea how to realistically solve it. The built-in demo correction stuff produces garbage for me (wildly disfigured images, or nothing at all), and the matrices used to perform it aren't "user friendly" -- Not sure how to gather specific matrices for every camera out there, to make it a simple drop-down.
Good point about dividing things between manipulation and analysis, I think that's very accurate, and can probably help me focus on the analysis side, rather than the manipulation side. [that said, camera correction is technically a manipulation that can be done with CoreImage filters...]
I understand that undocumented wishes aren't very useful for you, but i'm not really efficient in CV Stuff to talk about technics... I'm very sorry of that because adapting OpenCV in QC is my favorite thread in Kineme's projects ! I'm working with the OpenCV beta for a while, and i think it's the most valuable addition to QC (with increasing 3D, Audio and Video Patches). I put some OpenCV in my MediaCenter Project, and, for shure, like many of Kineme's users i'm secretly planing to make my own Multitouch project ;)
So here is my poor contribution:
When i'm working with the actual Beta: It could be very useful to be able to reset points, identify them (using boxes, squares...) by their size, velocity, or other values. When a point is loosing is target, it could be great to be able to kill or reset it or to be able to ask for re-pointing the same characteristics. When i'm working on a finger-tracking screen/surface, i use to downscale my video input to a 40X30 Pixels image, to increase the tracking response and velocity. The Quartz2CV, CV2Quartz video Convertion seems to use a lot of processing resources, and as we have to manage with many Native QC's Filters to improve tracking, it's very difficult to build a sophisticated composition in addition... If the Video Filters (to improve tracking) have to be include in the OpenCV Video "language" it could become very complicated in the future. I use to Reduce image size, to increase Luminance, to manage with a kind of "Difference Key" technics, to change Contrast/Gamma/exposure... It could be very useful to have a kind of "Good Point to track" Thresold, based on the quatity or velocity of moving pixels. It could be also useful to have an Image correction Filter, including Threshold, Gamma, contrast, Color to Black & White options (like a colors mixer to define the B&W resulting image), a good Median Blur could be very useful to kill video noise too. It could be great to be able to define what is pertinent to track and to be able to save this "definition" with the composition, in the patch settings. As OpenCV Patches does not seems to works with colors, I'm figuring that the luminance is the only variation in pixel aspect that could be analysed. So we need tools to manage with Luminance property in our video input. Minimum & Maximum quantity of moving Luminance to be considered as a "Good point to track", the luminance consistency should pass through a king of tolerance option.
I'm very interested in Face/eyes/hands/objects recognition too ! It looks very uneasy to input a simple picture (photos/scans) and to have a basic recognition ! It could be great to ad a Colision Patch in the next feature package ! I know that a kind of Zone colision is easy to create with QC but as we have to manage with a lot of this patches a complete Colision Patch could be very useful (also in GL Tools, if it understand Z positions..). Could you tell us what kind of analysis tools are listed into OpenCV project? I'm shure that you give us a link but i couldn't find it...
-Franz: if you know about a French documented site about OpenCV, let me know !
Thanks for your input on this.
Some dev notes: Yes, the QC2CV and CV2QC image transfers are very slow -- this is a QC limitation, not something we can really work around. However, there's not much need to convert CVImages back to QC (just use the original QCImage), so that can save some time.
"Good Track Points" are points with high contrast from their surroundings, so they're easy to track as they move. points that match the background are harder to track accurately, because it can't find their edges as well as high-contrast points.
CV doesn't give size information about track points, so it's impossible to report how large a point is.
Most of your filter ideas are perfect for CoreImage (and would be hardware accelerated) -- don't use CV's resize/image processing patches unless absolutely necessary, since they're likely much slower than CI. (thresholding, luma, gamma, contrast, B&W, median blur, etc are all possible with CI without a lot of effort, and very fast.)
40x30 seems a bit small for tracking... 240x180 shouldn't be much slower even with a larger number of points.
Handling lost-point behaviour would be nice -- I plan on handling that better in the next released beta.
As for listing the CV analysis tools, there are a bunch, and the documentation is terrible for almost all of them. Shape detection (circles, squares, others?) is possible, some kind of blob stuff (not really documented at all), optical flow (point tracking), haar detection (face detection), some kalman filters (not possible in CI, as far as I know?), and maybe some other things I'm forgetting...
Collision is an interesting idea -- ties in with Kineme3D, ParticleTools, GLTools, and physics stuff. Not sure exactly what aspect of collisions you're looking for (possibly just simple hit tests?)
Training haar cascades from images is tricky, and requires lots of input to get it working. A single image probably isn't enough to train it to recognize it later. I don't know much about image recognition though, so maybe it is possible?
I have v0.1
I have v0.1
About point sorting, i just went through this document : http://www.opencv.org.cn/images/d/d1/Opencv_introduction_2007June9.pdf
and realized i should use canny pruning before auto-track in order to obtain contour-sorted tracked points ! (btw, the pdf explains really well the whole blob-tracking stuff... with a spy-cam example ;)
Camera correction is typically doable via Corefilters. It is basically a pinch/barrel distortion filter. Since CI is boring to me, I tryied to apply the video on 2 pre-ditorted 3D meshes, then blended via KnM 3D morpher and placed in a render in image patch, but i didn't achieve good results yet.
oh, and here's a french forum about openCV: http://www.developpez.net/forums/f739/c-cpp/bibliotheques/opencv/
Merci ! I will check this French Forum...
I am really laughing out load here... After Memo's suggestion, I have been looking at openFrameworks and it just pisses me off. I mean, for some reason, it isn't too stable on my system, and I've had stuff crash just by clicking on the render window with my mouse. (oh, I am editing this statement to make clear that this is totally on the front end of checking it out, and I've seen awesome things done with it or I wouldn't be checking it out in the first place).
I'll say EXACTLY what I want :o) I want to be able to setup something similar to the ar toolkit markers... but using 4 corners and color recognition of those corners, or of some swatch of color that is on the piece of paper or whatever.
Or something along the lines of being able to hold up an item that is red, and hook coordinates to a 3d transform that tracks x/y/z ...
Setting up stuff like face/ hand/ eye recognition would just be icing on the cake.
I would also like a "reset" points.
Would it be possible to do something like your hough line example, but instead of generating lines, generating 3d positioned polygons that could actually be used with GLSL shading?... I'm thinking something along the lines of hough line, gl height field, rutt etra... mixed with openCV and kineme3D. It seems like some kind of depth map could be generated and transferred to a plane? I am clueless :o)