Help using the detector plugin!

samjjb's picture

Hello there, I've been asked to create an installation for an upcoming exhibition and have come into it pretty last minute. I've had a good look around the forum for bits of advice here and there and have read a lot of 'learning quartz composer' but am still pretty lost and would really appreciate a point in the right direction as the submission deadline fast approaches!

To sum up the idea, visitors will be instructed to sit in a chair infront of a monitor with a webcam and mic above it. The chair will be positioned in a way that the monitor will display a live feed of the visitors head and torso. There will be a controller probably made with an arduino with two knobs on it. Twisting knob one will apply a blurring effect mapped to the user's face, the more the knob is twisted the stronger the effect. By twisting knob two the visitor will be able to distort their face with their voice or any other sound near enough the mic.

I want to use quartz composer to create these effects using the detector plugin and I've been looking at VDMX to house the feed. I have been experimenting with the detector plugin and have been really struggling to create the effects so any help would be amazing!

Thanks & apologies for the long post!

Alexander Mitchell's picture
Re: Help using the detector plugin!

Hey samjjb,

This is actually something that Quartz can do well.

I have experience using QC with Arduino. I would suggest using the Arduino as a MIDI HID device. I used an UNO with HIDUINO firmware: https://github.com/ddiakopoulos/hiduino

That way you can get a reliable connection to QC without having to bend your mind around Serial IO and then suddenly switching on your installation one day to find that the serial device has a different ID or something.

You can use the standard MIDI library with HIDUINO - just transmits over USB - like a midi controller (HIDUINO is class compliant).

As for tracking - just pipe the data from the output face co-ordinates of Detector into a sprite. The Detector only spits out a structure when it finds a face - so I would have some sort of default.

Then have 2 video layers:

1) Layer 1, that is the unaffected video 2) Layer 2, in front of Layer 1 that has the blur effect.

Use a render in image to get a circle (or any shape) rendering as a video image, then pipe that into the blur effect sprite as a Mask. That way the rendered in image sprit allows only that area of the image to be blurred.

You can easily effect the amount of blur by sending data from the HIDUINO as a MIDI CC message (0-127) and scale it to whatever you feel is a good amount of blur. I would also use an OpenGL blur- so that its running on the GPU to keep fast- there is an FX-Factory Gaussian GL Blur, which obviously comes with FX-factory, but you can install FX-Factory and use the Quartz Composer Plugins without purchasing a license- you only need license to use FX-Factory Editor and within FCPX etc.

Quartz isn't the best with audio- so use something else with this- by using the HIDUINO firmware on your UNO you can very easily get the same data into LIVE or anything else.

There is also an example of using face detector running around here, so maybe check that out and rip out what you need. If you need any help PM me.

BTW: I don't understand why you would need VDMX with this setup. Apart from audio manipulation Quartz can achieve this stand-alone.

Dust put this together, which gives you more than you will ever need to hack together a working prototype: http://kineme.net/composition/dust/facedetector

Its a very complex composition- lots of spaghetti! But as I said works well. (PS: I don't normally look like that- my hair is normally a different colour).

Cheers, Alex

PreviewAttachmentSize
Screen Shot 2014-09-22 at 2.44.59 am.png
Screen Shot 2014-09-22 at 2.44.59 am.png836.95 KB

Alexander Mitchell's picture
Re: Help using the detector plugin!

Just a 1min hack job.

Sorry Dust for the hacking- it had to be done! ;-)

Just press Apple "T" to get settings and switch on blur.

Cheers, Alex

samjjb's picture
Re: Help using the detector plugin!

Hi Alex, yeah I should have mentioned I was playing with Dust's composition but as I am new to Quartz after 5 minutes of exploring, it became too much of a daunting task! Thanks a lot for the hack, I'm going to try it out now..

Also thanks for the great advice regarding the arduino. I'm still in two minds as to whether or not to go down the midi route or OSC as it would be far easier (and free) to use my ipad as the controller, it depends if I have the time!

FYI the reason I was look at VDMX was mostly down to it's ability to handle audio input so well, and also because the interface is far more along the lines of what I am used to - after several days messing around with quartz, I found importing qtz effects and managing them on layers easier to work with than getting lost inside a composition with a bunch of macro patches... Also the built in midi and OSC detection is super easy to set up...

samjjb's picture
Re: Help using the detector plugin!

Not sure what happened to my previous comment!

Thanks for this Alex, it seems as though the positioning of the features is quite far off, any ideas?

I was also playing with the idea of proximity as a parameter to increase the amount of blur as the user gets closer to the camera. I have a kinect lying around that I could use instead of my webcam, have you ever worked with a similar set up? I imagine the kinect might also give a slightly more accurate track...

PreviewAttachmentSize
Screen Shot 2014-09-21 at 19.03.30.png
Screen Shot 2014-09-21 at 19.03.30.png1.56 MB

Alexander Mitchell's picture
Re: Help using the detector plugin!

Hey Sam,

The offset would be a matter of playing with it and getting it working for your video input. I didn't design the overall patch, so I don't know exactly how DUST is doing all the maths.

I normally hack stuff by taking away aspects I don't need - simplifying it until its really simple to grasp.

For control I would go with MIDI- not OSC, you are not going to get a better product using OSC- and there is MIDI controllers for iPad as well: http://mididesigner.com

Good luck!

Alexander Mitchell's picture
Re: Help using the detector plugin!

Just found this from VDMX. Uses a different detector in QC, (Who knew there were two?).

There is no issue with offset- and its a simpler composition- so you should get to know it allot easier! You can cut and copy my blur thing (and make a few distortions as well) into it.

I am actually using the Kinect as my webcam right now anyway. To use it you need to get a plugin: https://github.com/v002/v002-Open-Kinect BTW you don't get any nice skeletal tracking- just depth map etc.

Cheers, Alex

PreviewAttachmentSize
FaceDetect.qtz8.32 KB

samjjb's picture
Re: Help using the detector plugin!

You are a legend Alex! Thanks a lot for the help, I'll let you know how I get on.