|
Processing Data to QC via OSCHi! I was trying to use the blob detection in processing (with the OpenCV and OscP5 library) and then send the blob points (blobs[i].points) and the blobs' rectangular vertex data (blobs[i].rectangle) to QC. Since I've just started processing, My knowledge is very small about it's work flow. Thought maybe one of you guys can help me with this: 1- does QC accept structures via OSC, for example can I send the whole "blobs" object to QC and then use Key member Patch to take the data I want in QC. if I can do this please let me know what should I write in Processing. 2- if not, What is the most efficient way to send the data to QC ( I can make multiple Arrays and copy the "blobs" data to them one by one but I think it slows down everything) Cheers for any suggestions!
|
set up oscp5 address like normal and send your message like this to qc.
use the name space "/blob/data" as type floats in qc osc receiver as it is the only type that will parse a structure. or you can pack all your data into a string and then do string explode in qc as well. i like floats. make sure to cast your data to float, or int qc will parse ints as a float as well.
a single osc message can look very different but the message you want to send for qc structure of floats would be a string like this.
"/name/space float float float float...."
Cheers dust for the quick reply. It worked and it's all thanks to you!
Could you give a rundown of what the blob detector in OpenCV/OscP5 library should be putting out when it's achieving what you wish ...I know you have your problem solved, but this might be helpful for getting blob tracking going in QC itself at some point.
Is there any pre-built little processing app/sketch that puts out OSC info for blobs... or rather, could you point to what you're using as your starting point? I used to use processing quite a bit, but never looked at openCV blob tracking for it.
Sorry George for the late reply,
Blob detection in Processing is available via the excellent OpenCV Library. The library accesses the blobs through the opencv.blobs object and the Blob[] class. so one can activate the blobs with a command like this:
Blob[] blobs = opencv.blobs( 100, w*h/3, 20, true );
The library gives access to some important properties of the blobs. for instance if the blob is inside another blob or the area of the blob etc.
What I do here is simply use the OscP5 Library to send 2 things to QC:
1- The rectangular dimensions of the blob via a command like this:
myMessage.add(new float[] {blobs[i].rectangle.x,blobs[i].rectangle.y,blobs[i].rectangle.width,blobs[i].rectangle.height});
2- The Points with which you make draw the blob itself in QC via the "Point" class and the "points[j].x" and "points[j].y" objects.
Since you can only send one-dimension float arrays (as far as I know) via OSC, you can separate the blobs' data by using a separator variable like 666666 for instance, so when you reach points.x =666666 and points.y=666666 in the sent array you'll know that you have moved to the next blob.
I can post the whole processing code and the QC patch if you'd like.
i for one would love to see your setup, idlefon!
count me in as well!
OK everyone,
I attached a .qtz comp wich draws the blob and the rect
the processing code is just like the example in the OpenCV library, just remember to adjust the ip address in the code to your system's ip.
I couldn't attach the .pde file (it's not allowed apparently), so I'm gonna copy the code in the next post. by the way I adjusted the code so that it only sends the biggest blob (the biggest area) to QC. you can configure it easily to send all the blobs (use the method I suggested in my previous post)
Have fun!!
Hey, that's cool! (haven't looked yet)...
@smokris, et al.: is there a way to use processing.js to have sketches embed in the site? If it's really trivial (I suspect it may be), it might be a cool feature... but I'm not sure how much people would use it (one use case so far!).
@idlefon: You could attach the pde (or any other non-supported file) by zipping it. (not a prob to just post the source like you did either, thanks!)
BTW, one "blob" tracking, as you mention this attachment being, is doable using QC, with CI, GLSL or CL. Multi-blob tracking is also doable...but no one has released anything. I guess I'll have to check out with the code to send multiples; I'm interested to see that part working robustly, so that I can tune my QC multi-blob tracking to be at least equivalent.
It seems like this could be exposed in the CVTools library in way similar to the HAAR object; image goes to blobber filter-> to tracker that has similar settings to the HAAR tracker.
I went ahead and attached it as a PDE... still haven't played w/ it :)
What version of Processing did you use, and do I need to add any other libraries from the standard dl? I'm getting an unexpected token result right where the "myMessagePoints.X..." clause starts.
I haven't used Processing for a bit, but my old comps are working, so I think my install is ok. I'm probably using the first or second official release (as opposed to the prerelease stuff that was around forever and a day).
I used Processing 1.2.1 Goerge and yes you'll need two extra libraries:
The OpenCV Library: http://ubaa.net/shared/processing/opencv/
The OSCP5 Library: http://www.sojamo.de/libraries/oscP5/
Sorry, You're right, I should have zipped it (Done!)
I really didn't know one could do blob tracking (with a fast FPS) with CI, GLSL or OpenCL in QC . I for one would kill to know how. I thought it's a CV thing and since it was not in the Kineme OpenCV, i assumed it was not doable in QC.
Thanks, I appreciate it. Hopefully I'll get a chance to give it a run tonight, and report back.
Checkout smokris and jstrecker's composition Centroid Calc (I think that's the name), in the composition repository. On strecker's machine, it's sorta slow (I can't remember the fps, but I think he's using the nVidia 8600), but on my machine, I get an easy 60fps, using the nVidia 9400 or 9600 (mbpro laptop). I don't know what the dif with that qtz is performance wise across a wide variety of machines.
That composition returns 1 centroid/blob.
Blob tracking is basically just a generic image routine that doesn't necessarily have to be linked to any library... just a library that supports the steps needed to derive the routine.
What's less intuitive to do in QC is to derive the region of interest, and then have the blob bounds track to just that area. It's super easy to derive one blob.
I've been been deriving multiple with a loop, and have taken the CI image routine, and converted it step by step to CL. I'm happy with fps, and it seems ok, I've just been looking for a decent example of "blob tracking" happening in some other environment, so that I can compare, and make sure I'm not unleashing anything dumb.
I've also been thinking that maybe I can be using local memory and add some index pixel offsets in a certain way (OpenCL), that would make stuff be even better... also, also, I've been noting certain kernels run more effectively when converting an RGB structure to a single lane structure, then converting back at the end. I want to play around w/ all that before I'm satisfied with a multi-blob QC routine.
However, as you say, this is sorta written into the CV library, so my hope would be that it would just get piped into that at some point, and be reasonably performant, rather than rewrite the whole routine in a C derivative (OpenCL or GLSL) and run it in QC...groan. (I may end up just rewriting it for speed/performance/"fun" though...).
I still think it hasn't fully occurred to people the breadth of things that OpenCL can do, because of sort of lacking QC examples.
@gtoledo3, we're discussing adding a processing.js filter to allow inline compositions on here. Meanwhile I've added
pde
as an allowed attachment file type.Thank you for posting this. (And thank you, GT, for the nice overview on blob tracking re: OpenCL, et al. Good luck with your pursuits in OpenCL. Exciting stuff!)
Cool... this is obviously a QC-centric forum, but I've always really loved the way that processing.js can embed in a webpage. I would love to see more people of other disciplines enter the mix.
I got a basic blob tracker going with open cl. I'm using it as a depth tracker with kinect. Right now I'm getting decent results. I have added a variable resolution so you can get a faster track if you want, it seems the higher the grid count the better more accurate it is for reporting blob size. a really small grid like screen size / 40 tracks and runs much faster. When I get online I will post.
Wow, sound cool! can't wait!
You can now embed Processing.js sketches in kineme.net forum posts, using markup like
...and you'll get a source and rendered view:
Thanks, @mradcliffe!
Marvelous!
Thanks Smokris and mradcliffe!
Javascript is loading on init now instead of trying to do it in the filter itself. A bit disappointing.
Edit: found another bug when posting this :|
It's kind awesome to see that pop up on here though...
This is a tweak on a sketch "navie" from openprocessing.org, from the recent posts.
What sort of fps do you get on that sketch?
I guess it's a good example of how nice WebKit is. It renders at 6fps (Safari) v 1fps (Firefox 3.6). And that's cross-platform Firefox too.
It's a choker for sure... that's sort of why I posted it; I wanted to see how stable everything would be with a harder one to deal with. I get 8fps in Safari.
wow this is cool you can embed processing into kineme. i want to try.
(edit) damn this doesn't render must be the lib ?