motion detection

DanieleCiabba's picture

this is the idea

"Development - motion detection / computer vision I have an idea (maybe wrong) that a static matrix like this could be used as the starting point for a motion detection / camera vision thing. Imagine if you processed the video (desaturate, detect outlines for example or use the “flood fill” that appeared on the quartzcomposer-dev list recently) and had something like a kernel that could sample each cell in the matrix and output a value or a boolean dependent on the white / black balance. I don’t think the core image kernel in quartz composer can output anything other than an image - but a kernel possibly can when used in core image in a cocoa application. Interesting."

that i have found in this site... www.daisyrust.com/2006/08/10/quartz-composer-video-matrix/

i'm not an expert but... i want to learn more...

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

cwright's picture
Core Image Kernels

Woah! I did some testing, and found out that you can return non-vec4 types from your CIKernels. For example,

kernel bool booleanKernel (sampler a) { return true; }

makes a white image.

I'm not sure what other types to (it crashed when I tried to do a vec2 :), but that's really interesting. Does anyone know how to evaluate a CIKernel to get the return value? This is similar to the OpenCV stuff franz was talking about.

franz's picture
CI data ?

and does { return false; } make a black image ? Very interesting.... but it is still an image... returning a boolean from an image would be supreme (ex: true when average brightness is over 0.5) I thought that CIKernels couldn't return non-image data, because data couldn't go back to CPU (or hardly) due to the one-way only graphic bus (AGP, don't know for PCI express)....

Anyway, for motion detection, check out: http://www.samkass.com/blog/C1936590049/E740325982/Media/MotionSensor3.qtz

and this is the code for motion detection (basically substracting 2 consecutive frame):

kernel vec4 passthrough(sampler firstFrame, sampler secondFrame) { vec4 sample1 = sample(firstFrame, samplerCoord(firstFrame)); vec4 sample2 = sample(secondFrame, samplerCoord(secondFrame)); vec4 outpix = abs(sample2 - sample1); outpix.a = 1.0; return outpix; }

however, this patch by sam kass returns an image, not the data. This is the main problem with shaders... unless you write all your logic within the fragment program (in-shader parallel computation) and unless you don't want, say, image motion to trigger sound for instance.

by the way, the function: "sample" could be used to make CIkernel video sampler as well ... (i just realized that...)

cwright's picture
you're right

it's not returning non-image data, it's just treating the return as a value for the pixel in that location. I've not found any way to "evaluate" the results of a CIKernel without just manually processing the image it returns.