Kinect Known Issues....

In preview here the strip will show up as black, not grey...

it seems you get the same fringe artifacts as well. i really first noticed it in ofx on the rgb image but its there in qc as well. it doesn't seem to bother the tracking.

tracking seems to be working in qc. here i'm overlaying the depth blob image with my isight which are at two different depths.

My Kinect should be coming today. I was just wondering if there is any idea when the new driver with better framerate might make its way into kinect tools? Cheers.

Sure- it's definitely close enough of a match for virtual object placement; it's using the depth map to create a mesh then texturing the result that gets a little awkward... but doable.

I'm sort of curious about the "strip" on the depth map on the right side.

I'm curious about that as well. Also, I'm sort of curious about what looks like the queueing of the frames. What I mean by that, is that if I have frames rendering, and do something that significantly changes the graph, or causes the Editor to bog down (like transmutating a macro), one sees all of the frames that the kinect was grabbing while the editor was hosed suddenly render quickly.

It looks kind of funny/cool... it's the only time I can remember ever seeing anything like that happen with QC.

Wahey my Kinect arrived :)

Saw that there was sourcecode for the plugin so I thought Id have a go at patching it myself but got horribly stuck, cant build in xcode, not sure if its because SkankySDK.framework is red.

I can say the strip on the right side doesn't seem to be there in ofx. I'm thinking it just has something to do with the depth camera being offset and that may be just the inter oculars disparity between rgb and d images. kineme makes the alpha channel clear where there is no depth reading. you got to work around that. it seems color blob tracking tracking works but when you track with open cv the alpha becomes a problem. yes I did notice using the mesh depth field mapping to be little off. although gt your depth field test images look very good. I have noticed if you correct the offset for the right side the left side is off and vise versa. so thinking there must be away to double up the depth image and offset for both sides ? either way nice test images. are those coming from you cl kernel

The framerate improvements described in https://github.com/OpenKinect/libfreenect/issues/issue/22 aren't working for me --- if I make those changes, libfreenect never receives a valid frame from my Kinect.

Still investigating.

The fact that the strip is there isn't hard to workaround; I wanted to make sure it was coming through that way in QC for others, to make sure my unit wasn't out of alignment.

I find myself shifting to the right, and oversizing the RGB slightly to make it match. I haven't found any magic number and I don't think there really is one, given that the RGB channel is to the side of the depth sensor. It becomes a bit like trying to match a left and right eye image.

Yes, that is essentially the OpenCL kernel that I posted with R/G/B inverted but not alpha, and some other tweaks. I've had to add steps to offset and resize the color image and keep origin 0,0.

Though I've been visualizing the output mainly with OpenCL in my tests, I have to throw in that this is a very good scenario to use the Kineme3D heightfield for. The K3D heightfield's resolution isn't locked to pixel w/h, and can go higher than the GLSL grid in vertex count. There are already separate image inputs for the heightfield and the texture images. You can use the built in K3D deformers on the result, and stuff flies along. However, I like that I can get the vertex and color info with the OpenCL kernel and do_stuff.

Thanks.

Maybe I'm wrong, but it seems like part of the problem is that it isn't dropping frames. I may be using the wrong lingo.

Whenever something happens that normally bogs QC down a bit (taking the editor window and vigorously shaking it back and forth, moving a patch back and forth quickly on the editor, doing a screen cap, taking focus on or off of QC, etc.) I can see latency increase, and I'm suddenly still seeing the frames for X frames ago, instead of the signal just being "now" + normal latency.

However, if I just let stuff render, don't do a bunch of editor events, etc., I see very minimal latency, and performance seems to stay at a really quick rate in general. It's when what looks like queuing of the video stream starts happening, that stuff goes awry. If too much of that happens (err, queuing of the video stream, or storing in memory, or whatever is going on exactly...not sure of the right technical term), then QC will actually crash.

Again, thanks for whipping this up.

The new release is slick.

It's way quicker. It's so fast on my laptop that I'm able to make it look like my room and I can dance in realtime. This thing is actually pretty fun, especially now that the performance is getting sweet.

Thanks so much for the updated version, its great :)

George Id just like to thank you very much for the various OpenCL compositions you posted in the past that work nicely with the Kinect. I had fun playing around with this stuff on Saturday evening. Ive yet to create anything very dissimilar to the videos youve already posted, but if I do I will post video.

Perhaps the most simplegeek fun I had was when I realised I could remove the background by simply placing a large qc cube at the required z-pos.

Ha! Nice. Great track selection.

check out this vade and coge webkit kinect fun.

That song is pretty awesome. I feel like Timbaland's production on that was great. It was a real interesting bridge between what was sonically trendy at the time, classic stuff, and then things that have come to be associated with his sound since.

Ginuwine does some move in the video, and then they're riding an electric bull... the pace my room was moving somehow reminded me of that movement from the vid.

You can use a sprite behind the main mesh renderer too... you can link the color of the sprite to your clear color and make width and height really large.

That's cool. Please do!

If you change the mesh renderer to point mesh, and make the points kind of chunky, it will look like a smooth mesh with gaps on the sides - if it's not using GL Lighting.

I've had a few functions I'm trying to get into the kernel. I've come close adding an alpha threshold function to the color output. Last night I got it working to change all of the color outputs by a given r/g/b/a input value, so now I just need to add the "if vector length is less than rgba, then 0,0,0,0" aspect to it successfully.

I was trying to add a certain smoothstep function, but when I write "smoothstep" it doesn't highlight in the OpenCL editor as a function. Maybe I'm using it wrong(?), but it looks like it's part of the CL 1.0 spec.

I want to actually figure out how to pass only vertices that exceed a certain floor value of the color vector length, but my syntax seems to not work, and the CL editor doesn't highlight any lines, it just tells me there was a compiler error. It becomes a little trying! I've never output "less that what I put in" with any kernel.

I guess on the plus side, I've learned a ton more about OpenCL.

Point meshes are what I have been playing with so far, as Id done a bit of that stuff in the past with other CL things. But Ive not had much time and havent achieved anything I like yet, will be trying again tonight or tomorrow. Cheers.

I feel like every time I try to work on a kernel, I'm relearning things. I finally got my syntax correct on at least a couple things I was attempting, so with some polish, it will be quite cool.

Man, I wish I could just type pseudocode sentences of what I want to happen... it's always a pain when I just forgot a semi-colon or parenthesis somewhere. The compiler doesn't always catch that. Also, it seems like things that should work sometimes just don't, and it gives a message like "compiler error", with no useful lines highlighted.

I was trying to do a smoothstep function last night, and it didn't highlight like it's able to be done, but I'm not sure if I'm using it correctly - it should highlight either way, it seems like. Anyway, things like that sometimes make it a pain to get kernels going. "It is me compiler, or is it you?"

the smoothstep is a normalized hermite interpolation.

it works like this. your input value gets stepped in a normalized 0->1 if start vale is less than the input value and input value is less than end value otherwise the result is 0 or 1

so this is sort of pseudo code i guess or well the kernel set up needs to be done but this is basically the smooth step function.

float startedge = 0;
float endedge = 1;
float inputvalue = .5;
float stepvalue = 0;
 
float step = clamp(((inputvalue - startedge) / (endedge - startedge), 0, 1);
 
stepvalue[tid] = step * step * (3 - 2 * step ); 

Sure, but I mean that smoothstep is a supported function in opencl 1.0 but it seems not to be in QC. I wasn't meaning that someone couldn't write something equivalent. I was using smoothstep as an example.... I could have picked any of a number of other functions or operations ( I'm not sure if I'm using the right semantics) that seem like they should be supported but may not be.

thats actually the open cl smooth step function i converted from pseudo the smooth step will not return anything but one or zero unless the input value is between 0 and 1. so using it to blur or gradient is possible but you cant lets say input your mouse coordinates and normalize them from 0 to 1 because qc unit is -1 to 1 but you can convert units to pixels then subtract width or height depending on which axis x or y and then you can use them with smooth step.