OpenCL Sculpt (Composition by gtoledo3)

Author: gtoledo3
License: (Other — see description)
Date: 2010.07.19
Compatibility: 10.6
Categories:
Required plugins:
(none)

This composition is a demo that shows how to setup an interactive sculpt method with the Apple Mesh engine in Snow Leopard.

Shoutout to cwright for a question he answered a long time ago (hmm, 2 years? Time flies...) where the resolution was to take a Kineme3D warp and sample the output with a sample and hold before the renderer, so that a gravity warp would freeze on mouse up, and clear on mouse down.

After that lingered in my mind, I realized I could setup a feedback/multiplexer loop that would allow one to do a type of additive sculpting that wouldn't clear the sample and hold data every time a mouse down event occurred, but additionally clear it if I wished with a secondary mouse button.

A recent discussion about pushing around vertices in OpenCL led me to whip this up, since I was pretty sure the routine could be applied to OpenCL just as easily.

So, this uses that kind of sample and hold based feedback loop with multiplexers to allow one to use a mesh filter in a way that adds the results of deformation to the resulting mesh with each push/pull motion. This is very similar to some sculpting and vertex push functions in modeling programs.

Rev History- .01- Slight tweak to improve clearing mesh filter effect.

PreviewAttachmentSize
OpenCL Sculpt Demo .01.qtz83.68 KB

vade's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

George, thats impressive. You really push QC stuff far. Well done.

leegrosbauer's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Cool! :-)

PreviewAttachmentSize
Sculpo.jpg
Sculpo.jpg186.57 KB
Sculpo2.jpg
Sculpo2.jpg221.42 KB

cybero's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Really interesting example.

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Thanks! One of the best compliments ever.

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Those are some nice pics, great examples of the precision that can be obtained. It's pretty nice how it can work like an extremely tweakable interactive 2D CI Bump distortion, with infinite bump points, isn't it? All of a sudden core image distortion stylize starts looking way yesteryear, lol...

leegrosbauer's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Considerable fun, amongst other great attributes, here! Good one, George! Really nice! Those are both live video input images, btw. Input image on Sculpo2 is filtered prior to being in-put. Right click and hold until a desired cam input image is achieved, then release and sculpt. Repeat to start over.

dust's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

@ lee i know your using video input but just the still of sculpo2 really looks like you painted it. nice one. you should save the vertex points down to .plist and use a still texture so you can use the qc image enlarger to make some nice prints for you wall or a gallery or something. or just make some more screen shots ;)

@george sweet meshy. i prefer modeling more or less organically with push and pull style like this. nice use of variable resolutions.

makes want to break out my arduino and give digital foam another go ;)

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Heheh...

[/quote]

PreviewAttachmentSize
sculpt2.jpg
sculpt2.jpg66.51 KB

leegrosbauer's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

That painterly look is merely a byproduct of using the composition to distort a video input image that's being pre-filtered in CamTwist and then fed into the composition, dust. The coloration and linear appearance come from the image filter ... manipulation of George's composition then smears those attributes. Anyway, it's all very easy for me to recreate at any time. I'll be exploring this composition's capabilities a lot more, for sure.

jersmi's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Wow...

Scratchpole's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

That is very impressive! You really are pushing things a long way George. About time I got with the Snow Leopard!

404visuals's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

This looks like one of the most useful tools anyone has made for QC. Would be interested to see the ability to store blend shapes and animate between them. I look forward to seeing further development! Good work!

psonice's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Cool stuff George. I did something just like this myself ages back with GLSL, as part of an application I was working on (never finished unfortunately, it got sidelined by more profitable stuff). Mine worked in a pretty similar way, but I added a lot of brush tools, plus texture/bumpmap modes as well as sculpt (it was a LOT of work, but was pretty cool :)

You've got sphere based sculpting working too by the look of it though - how do you detect which vertices are under the brush? And are you doing path-based brush strokes? (that was the bit that had me struggling for ages :/ )

There's an absolute ton of cool stuff you can do with this. You can draw an object, then apply effects to it easily. Eg. here I painted a 'pool' (you can see i spent a lot of time painting that ;) and filled it with animated water with a couple of clicks: http://img366.imageshack.us/i/water2ss0.png/ and here I plugged some spinning cubes in as the height map instead of drawing: http://img224.imageshack.us/i/metacubesjj1.png/ (these were done with my own comp, not yours btw).

Keep at it, you'll have endless fun with this one :)

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

K3D attached.

A little different, maybe not quite as controllable because of the way the warp works. Also, won't work as well with shadows (flickers while you warp it), but you aren't in SL, so that shouldn't make a difference.

PreviewAttachmentSize
Kineme3D Sculpt-Demo.qtz59.02 KB

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

I've had this setup for a pretty long time...

OpenCL mesh filtering does the hard work on calculating the vertices for the Sphere example. Since each vertex has actual x/y/z structure that the kernel can manipulate, using x/y from the mouse, and then manually setting z, is the basic method for this. I've also used Y scroll to control Z, but that's trickier, and I didn't do that for this setup because I though it would be awkward for people.

It doesn't automatically follow the circumference of the circle. I think that would probably be possible to setup for this qtz with a little sin/cos lfo setup with amplitude matched to diameter of the circle.

So, this orig post doesn't use a heighfield, which would require a sort of different approach.

However, I made a quick little feedback push image to heightfield demo for demonstration (I know you would know how to get this going, I'm just posting this for whoever cares.) That's a cool technique because you can use different shapes, just like you say. This technique is probably way more obvious than the OpenCL filter feedback loop that happens in my sample comp above (?).

I think I would use GLSL as well, if I wanted to be able to use different brush types, have a heightfield generate the displacement, and be able to use arbitrary objects.

PreviewAttachmentSize
OpenCL_Heightfield_Sculpt.qtz29.44 KB
george.png
george.png122.62 KB

photonal's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

you're doing more comps than I can manage music tracks to use in my vids!! ;-)

great stuff btw!

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Lol...what if I told you I wrote a song a day since I was like 12 on top of this...hah.

gutsblow's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Holy Crap! You are quite becoming the flight404 of Quartz Composer! Keep up the great work.

cybero's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

That painterly look is pretty awesome.

leegrosbauer's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Thanks Cybero. It's done with CamTwist's Watercolor.qtz image filter, which has been modified by changing out the Color Blend Mode patch for a Blend With Mask patch. The input image is split and then fed to both the Color Monochrome patch and the Mask Image port of the Blend With Mask patch. Lastly, the Saturation, Brightness, Contrast, Intensity and EV ports have been published for easy access. You're on your own to do the hack ... it's not my patch to distribute here.

photonal's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

what!?! You do these great QTZ comps, do a song a day and don't run a successful conglomerate too?!? I'm shocked and disappointed!!! ,-)

cybero's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

No worries about the lack of distribution, I've already got CamTwist and its default effects installed.

I love the fact that you can interact with the Meshes.

leegrosbauer's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

I was pleased with video interaction capability of George's composition, too. It was the first thing I tried. Btw, if anyone intends to use CamTwist itself as the video input source (as I did), Quartz Composer has to be run in 32 bit mode. CamTwist doesn't appear as a video source in the QC Video Input patch when QC is in 64 bit mode.

cwright's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

gtoledo3 wrote:
Shoutout to cwright for a question he answered a long time ago (hmm, 2 years? Time flies...)

Holy crap ... I need to find a new hobby :)

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Mac Paint...

leegrosbauer's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

This great composition no longer works for me in 10.8.2. The Grid Demo portion of the composition no longer appears (Sphere Demo seems ok), amongst other irregularities. Anybody have any ideas as to what adjustments might help to get it working again?

gtoledo3's picture
Re: OpenCL Sculpt (Composition by gtoledo3)

Hey there Lee.

Let me take a look at it later today.

If any of the openCL kernels involved have "main" as the name of the function, rename it to something else. If there is a "grid indices generator" patch, make sure that doesn't have "main" as the kernel name either. It also may that the feedback loop needs something to pull on it at some additional place now. Or something else.