Quartz Composer Snow Leopard - New Patches and Features

gtoledo3's picture

There are some pretty cool things in the new edition of QC Snow Leopard, notwithstanding any glitches. Leopard had those too (well, some never DID get fixed).

Here are some things I've noted:

-The Shadow feature on the lighting patch, is a way of doing some dynamic shadows with your lighting. The default color value seems a little heavy to me for most purposes. Backing the alpha value off to taste is a good way of taming this.

There's also a quality control! A setting of "1" will give a nice smooth edge, and decreasing that value will seem to cause the edge to get jagged, like there's an increasing amount of aliasing going on.

When I decrease the quality on the Intel X3100, I see decent gain in fps. On the NVIDIA 9600, not at all.

The shadow patch doesn't work with iterators, and toggling it back and forth can do something to "lock" the state of the Lighting environment, which can affect flat/smooth shading. While this is one of my TOP Quartz Composer feature requests, it's also to be used cautiously.

-DAE/Collada loading. The thing that's fairly handy about this is the way that all of the associated textures get loaded.

Instead of manipulating parts of the DAE with structure patches, we have "Get Mesh Texture" and "Get Mesh Component" as well as "Set Mesh Texture" and "Set Mesh Component". The texture patches are for manipulating the associated texture/image files... allowing you to "break off" the textures, do what you will, and re"set" the texture, or the component. A "component" is one part of the structure or a complex 3D object; this is familiar to anyone who has used Kineme3D.

The DAE models can be dragged onto the editor, which is slick, and familiar to Kineme3D users.

The 3D dae/collada models need to be rendered with a "mesh renderer". So the chain needed is something like mesh->mesh renderer, or mesh->mesh filter ->mesh renderer.

The mesh filters live in the patch list as repository compositions/ "qtz" patches, so each actually has an extensive underbelly that can be investigated, which aren't exposed by looking at the patch right off the bat.

Some of them are colored like providers (assuming that green=black), but some are actually externally driven by default, and hence, are purple (old blue=purple now)! To wit, look at Mesh Jiggle and see the way it interplays with the mouse.

It's worth noting now that when you make a macro, you can control+click, and get an option for "explode macro". THIS is awesome. I would love to see this type of feature become available with these type of virtual qtz patches that are leaned on very heavily in this OS.

-Other mesh stuff: To create meshes you use the... ta da(!)... Mesh Creator! It needs vertices... and you can also pipe it normals, colors, textures, texture coordinates, and colors.

You can feed it with Javascript or OpenCL. THAT is actually pretty cool.. and the mesh utilities actually make use of it as well.

For a good take on powering it with javascript, "Mouse Trails.qtz" is a good one, and it's linked to inside the actual patch list under mesh creator. That one might look familiar to anyone that's paid attention to the travails of tobyspark.

For a less obvious one that uses OpenCL to generate the mesh, go to the Developer folder and look in Library/Quartz Composer/Examples/Patches, and take a gander at "Grid.qtz" and "Quad.qtz". "N-body.qtz" is a more intense, but similar setup in some ways, that shows how to derive a particle system, and how to use a mesh creator/queue with iteration.

-Feedback patch. This is an interesting one. All of the qtz's that have "Inertia" in the title and that are in the same Patches folder, are really good examples of this one in action, as well as the 2D Fluid Simulation, which is in the System/Library/Graphics/Quartz Composer Patches (qtz's that Quartz Composer hinges upon are now loosely strung throughout the OS, so think wisely before editing or accidentally saving over one that QC needs).

Now, don't confuse this with "video feedback"... which is just a classic video/accumulator qtz setup hiding in the patch list, and which doesn't incorporate any new patches (this one actually reminded me of smokris's GL Spline example with a video input plugged in, instead of splines).

-The Interaction patch, and it's weird "outie" port and blue noodle. The Iteration Counter and Iterator Selection qtz's show this one in action.

-This brings up one of the BIG changes in QC. Now we don't have "just" enable ports on Render patches. We have "interaction" ports up above them!

This one strikes me as odd, and the ramifications of it aren't totally obvious yet, as far as how it affects the scene graph.

-Deleting or Cutting a renderer patch doesn't "kill" it in the Viewer. It stays active until you restart the viewer, or do something else. This may be a "feature" to help performance. I find it to be a bug, because it necessitates that one do an extra step to get the Viewer to reflect the ACTUAL state of the editor. This feels like it may be considered a feature (though I think it's a horrible idea).

-Multisampling in the Viewer. This one is cool, and looks like it works on the GPU. Option+Preferences, and there it is in the Editor settings. Check it "ON", do a restart of QC, and there you are... jagged edges somewhat mitigated, if it's on supported on your GPU. It doesn't smooth things out 100% but it's better than nothing, and adds a more professional look to the real-time visual output.

-The opening Template.

This one is actually amazing.... look inside of QC and take a gander at the qtz that powers it. This is a really well done qtz. At the same time, it removes the dynamic template loading, so as much as the engineering of it is beautiful and well done, it reduces function in some ways, while providing some good documentation that wasn't formerly included in the Template choice screen. It's as if this has been made a kind of qtz based brochure of function and documentation.

-... and if you unhide patches, our old friend CISmallGaussianBlur, the dead patch entry is still there! We would have missed you buddy!

-The "mystery" qtz's that wouldn't restore in the old system and that were leveraged in some apps, appear to now be in the System/Library/Compositions in some cases.

-Apple is using the Skanky SDK...maybe :) They are now loading (this pertains to the mystery qtz's working) the Backdrops.plugin and ImageDifferPatch.plugin... from the Quartz Composer Patches folder. So, now we are given GFPlugin(s) from Apple, officially. One thing that's of note is the "GFPluginExclusiveHosts", which is used in the Backdrops.plugin

So, those are some of my thoughts at the moment, and looking towards the silver linings. I've noted a decent amount of other features and/or issues, but this is about all I have in me for this post :)

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Deleting render patches does make them vanish in the viewer, but only if there is something else going on in the composition which causes the visual output to update. Bug rather than feature I would guess, they need to force the scene to refresh when something is deleted.

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Oh and I dont know if its new or not but there is an option called QCEditCompositions in system preferences. This provides a new window which lists compositions loaded by your current composition, and you can double click in that list and it will load that composition up. Edits to it will be reflected in your main composition in realtime.

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Yeah, that's what I meant by "until you do something else". I guess that was ambiguous.

I don't know. There is some weird stuff going on with the way that things evaluate now, and I really would find it hard to believe that no thought was put into it even if it does seem wonky. The thing is, this could theoretically be beneficial in an environment because it seems to keep tabs on the "instructions" that it takes to fulfill a chain of patches, and you don't see quite the same lag time for refresh when you cut and paste in some cases now. I tend to fall on the side that this is intentional, but simply that negatives were overlooked.

That lighting/shadow thing does a similar thing with not updating certain things in the qtz when you have shadows enabled, but it's a bit quirkier.

DAE's in a render in image environment are also suspect in the way that Apple is pulling it off, and their FOV is sorta deprecated as well now, since it basically doesn't work with the DAE loading at all. That's a bit of a disappointment since the FOV is so cool on big scenes. Patches that manipulate the projection style also seem to make shadows not work well, and can get shadows to "not line up".

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Good call on that! That one helps mitigate the big virtual patch reliance, though it also mods everything all over if it's a patch that QC relies on no? I wonder if this is part of the system that prompts you if you've loaded a newer version of a "qtz patch" and you open a qtz that relies on it.

cwright's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

gtoledo3 wrote:
It's worth noting now that when you make a macro, you can control+click, and get an option for "explode macro". THIS is awesome. I would love to see this type of feature become available with these type of virtual qtz patches that are leaned on very heavily in this OS.

This is part of why we never made a macro exploder in KinemeCore ;) Saw this coming a mile away (we still have a Macro Morpher in the works though)..

gtoledo3 wrote:
-Deleting or Cutting a renderer patch doesn't "kill" it in the Viewer. It stays active until you restart the viewer, or do something else. This may be a "feature" to help performance. I find it to be a bug, because it necessitates that one do an extra step to get the Viewer to reflect the ACTUAL state of the editor. This feels like it may be considered a feature (though I think it's a horrible idea).

There's no way this could be considered a feature -- it's a bug, through and through (and I've done my fair share of contrived "it's a feature, not a bug" rationalizations before)

gtoledo3 wrote:
-Multisampling in the Viewer. This one is cool, and looks like it works on the GPU. Option+Preferences, and there it is in the Editor settings. Check it "ON", do a restart of QC, and there you are... jagged edges somewhat mitigated, if it's on supported on your GPU. It doesn't smooth things out 100% but it's better than nothing, and adds a more professional look to the real-time visual output.

We should be adding this to QuartzBuilder in the next release or two -- will work on Leopard as well, on supported GPUs.

gtoledo3 wrote:
-Apple is using the Skanky SDK...maybe :) They are now loading (this pertains to the mystery qtz's working) the Backdrops.plugin and ImageDifferPatch.plugin... from the Quartz Composer Patches folder. So, now we are given GFPlugin(s) from Apple, officially. One thing that's of note is the "GFPluginExclusiveHosts", which is used in the Backdrops.plugin

Apple's always used the SkankySDK (all non-plugin patches are written using the skankySDK api). They did move things, which is interesting though. GFPluginExclusiveHosts isn't new -- it was that way on leopard (so you couldn't use the Backdrops patches in anything other than photobooth and QC) - I wrote about this over a year ago: http://kineme.net/Discussion/Leopard/BackdropHackdrop

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Actually I think it only opens them in a read-only mode, its only good for looking at what is in a composition or making temporary changes to it.

Anyway this applies to stuff loaded with the composition loader patch, virtual macro's are a different kettle of fish. They can be edited and I think it saves them inside the compositions where they are used, for easier distribution of the composition that uses them, as well as globally. Other compositions that use that macro will update next time you load them, which is probably the prompting you refer to.

Anyway Im sloppy and never made much use of macro's in previous qc so I have no idea exactly what aspects of this are new.

Oh I just found the sphere mesh that is used when you are editing a mesh template, its just a dae that lurks somewhere.

vade's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I really wish they would publish my QC 4.0 review over at create digital motion.

sigh

There are so many behind the scenes changes in QC 4.0 that are really nice, and many things are much faster. Render in Image is actually really usable now, and does not slow down much due to moving over to Frame Buffer Objects, which makes a huge difference. You covered a lot of what I wrote about. Mesh handling is so freaking awesome, I just wish my GPU was supported since CPU fallback does not currently work with OpenCL in any form. Argh.

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

The macro morpher will be a sweet companion to the publish to root. There are so many times I have something published all the way up, and realize that I need to put everything in a 3D transform, or some other macro.

As far as the feature vs. bug, I agree, but I'm weirded out by that interaction port, and I was speculating that this may have had some hand in this bug or that's it's a new approach, but I suppose it's more than likely unrelated. The color flips with the patches even seem to suggest "don't think about this how you used to". With the way that you can't monitor certain output nodes unless you use feedback, this interaction port, the fact that the state doesn't refresh, the way that the shadow on/off seems to effect things and not refresh, just made me wonder if something really fundamental has changed in QC as far as "lazy evaluation" goes... like... now it's ultra-lazy :)

I had figured that the multisampling was possible in Leopard from watching the Vade and Tobyspark clip where they talk about it, and having read the various on-List discussions, but only if you wrote it into the app. Would this have actually have been possible to have implemented in the QC app, in Leopard? It's a moot point now, but I'm curious from a technical standpoint.

I was never sure if using GraphFoundation was necessarily synonymous with "skanky" or not, even though I understood that way that the the GraphFoundation plugins become patched into and directly part of QC, whereas "QC Plugins" in the official API are all just working through the QC Plugin.plugin, or whatever it's called. I thought there was a chance that Skanky and GraphFoundation were just overlapping circles on the Venn diagram, because I wasn't sure if you ever use other Apple private frameworks or what, and how that played with the official(or non-official) definitions of everything.

I should have remembered to mention that thread you linked... that was a really intriguing thing. Many of the earlier posts on this forum from you and Steve are outstanding in that regard, not to take anything away from current posts. It's just cool that you guys are there on forum fleshing out the underpinnings of QC.

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Anybody got an example of the 2D Fluid Simulator in use?

My 8 core Mac Pro is very sad that CPU OpenCL in QC is broken at the moment. It wouldnt be so bad if the expensive ATI 3870 card I put in it worked with OpenCL, but no, and the Apple store still shows weeks before the GTX 285 is available in my country.

I shall have to look on the bright side for now, at least my laptop works with OpenCL.

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Yeah thats a shame, especially as most of their Snow Leopard coverage to date has been 'dont rush to install it', Im eagerly awaiting something more positive.

Actually Im a bit surprised how little has happened since snow leopard launch, its been pretty quiet on the subject on VJForums too so far.

All of which means the nicest eye-candy Ive seen on Snow Leopard so far is that example of what hardware accelerated 3D CSS Transforms can do in Safari:

http://www.satine.org/archives/2009/07/11/snow-stack-is-here/

On a semi-related note I ponder with much drool the possibility that Apple could release a tablet with full OS X on at some point, and then maybe the idea of using Quartz Composer in the browser may gain some traction, especially with mesh & interaction stuff. I wouldnt mind making a career out of that.

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

There's also some weird stuff going on with DAE's inside of Render In Image, with shadows, and in other scenarios (likely a few different scenarios).

For instance, I can't replicate the bug 100%, but it's a problem with the Undo feature, possibly in combination with the thing of Renderers not disappearing in the Viewer. Through some steps, I can sometimes get a DAE to look like it's entirely "made of" shadow (which would be awesome if it was actually a feature), when I place a dae that's in a Lighting environment with Shadow enabled, inside of a render in image, connect it to a Sprite, and then undo a certain amount of times. I had it happening 5 minutes ago, and am trying to repeat, but it won't do it again right now. It is the oddest thing.

I haven't used Snow on the X3100 for a bit, but I really thought that I had mesh deformers working on it at one point (not the GM), and it's not even a supported GPU. I could be wrong for sure. I haven't tried that with the GM yet, but I guess I wouldn't be surprised if it doesn't work.

What do you find to be the biggest plus about mesh handling? The ability to generate them, or the deformers? At the end of the day, when it comes to setting up a visual "outcome" , I don't feel like much has been added to my palette besides the anti-aliasing and the shadows... and the shadows not working with the iterator was a gigantic let down for me, as were the GUI changes. Between generating structure for GL Tools, and Kineme3D, I could achieve most of the same visual end results, and there are still a decent amount of things that can't be achieved with the new Apple patches.

Another thing I'm curious about when it comes to the dae/mesh stuff, is why when I rotate a dae building or something, using an interpolate, it looks so darn jerky and unsmooth in movement. I guess this is a look that I associate with lack of GPU acceleration, but maybe that's not correct.

I'm going to go ahead and officially report all of the stuff that's "definitively broken" but I really hate filling out bug reports, especially so many (more non-QC than QC though).

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

What excites me is the potential with the OpenCL stuff, rather than what it can do right now with me at the controls. Im hoping that some people are going to use it to do some very wonderful things, and I will be able to look at the code and stumble my way around modifying it. Plus the potential to harness all my CPU cores & GPU's power for appropriate tasks, gives me hope that I might finally have a realtime tool that actually wants to use the hardware I throw at it.

dust's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

yeah ny chance george you could post any of the cool things you mentioned in a working implementation. in particular the things located in /system/graphics/patches and in the repository.

i have had snow leopard for some time now and well i havent actually really tried to use it yet. the things i like to use don't seem to work ie kineme gl tools. im glad k3d is working cause i can't get a dae to show up.

maybe a simple how to get a dae to show up patch. for some reason the collada plugin will not let you install it on snow leopard yet. i have copied it from leopard and can create them. they are system wide you can open them in image preview and spin them around with a trackball.

for some reason i get nothing when i try to route them into qc. i must be doing something wrong.

the 2d fluid simulator is a must for everyone to see working. if it is a system implementation what uses it ? i would at least like to see it working in apples intended context ?

or a back drop example would be cool. i tried for a few hours one day but was unable to get a correct configuration in qc.

if apple opened up the repository then they should be able to be used in qc.

the only things i have been able to get to work is the mesh javascript examples.

i have been waiting 4 months for this to go public so i can figure out to use all these cool things.

im messing my iphone today but so frustrated so im back messing with qc.

if can get anything mentioned here working i will post, i know they are developer examples but for some reason they are really lacking new examples that work. tons of new repository type things but not very many examples or new ones.

inertia and interaction are pretty cool i can post some things with that latter i got that stuff working.

maybe post a working dae model so i can look at in a text file and see the xml because for some reason i think im exporting them wrong.

if they work this is awesome built in physx and animation etc...

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I have a 2D Fluid Sim modded example that I wrote over after my first Snow update (d'oh), but I should be able to put one together again easily enough. I'm not sure if anything uses it, or if it's something that maybe they actually meant to show better examples of at one point?

Dae's work, but that doesn't mean that any of the possible physics support is, or that any kind of character animation stuff is. In fact, it seems to not be at all. It doesn't seem like any kind of animation info is used by the Apple patches.

I was going to post here, that Google 3D Warehouse will be a new go-to place, for the sheer breadth of models. Due to the nature of Google 3D Warehouse, many of the details are extremely low poly, and low quality, basically. I've personally found the interior models to be of good quality, especially stuff by Surya Murali:

http://sketchup.google.com/3dwarehouse/search?uq=00754149149147553341&sc...

You can also register with the Collada Test Model Bank:

http://www.collada.org/owl/

That's where I got the really nice dining room table model for a render I uploaded on Vimeo a couple days ago.

The "Grid.qtz" example is a pretty good one for showing how to create a mesh, and then load it.

As far as dae's go, you should be able to drag a dae right onto the editor, and hook it to a mesh renderer and have it "just work". Do that with any of the Surya Murali stuff, or really any of the Google 3D Warehouse Collada.zip's (that I know of), and if it's not working, then it doesn't work. Maybe the problem is that you're inherently thinking of doing something animated and it's not supported ( I think something - another patch - likely has to be added to parse through animation info so that it could "do something").

I'll post some stuff, definitely. I have a good amount of SL compositions, and that's a big reason I haven't posted too much lately, but at the same time, they aren't entirely polished since I really didn't think SL was going to be deemed ready for primetime so soon.

I have a lot of stuff working with models that I'm not sure about redistributing, though I'm sure that many of them are able to be redistributed, just have to look (maybe not the Collada Test Bank stuff in some cases....). Some other stuff just needs to be cleaned up a little bit.

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I agree that dae animation and physics and stuff is not supported. Not that Ive actually got any dae's with animation in them to test, but as there are no controls for such things in the dae loader patch, I think its a pretty safe bet.

vade's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I have tried .dae exports with with weights and skinned animation from professional 3D animation software, they error and do not load. Its a shame, but perhaps will be supported in the future.

I also noticed that Preview.app opens Collada files, so that means there is probably a 3D Loading framework tucked away in 10.6 somewhere :)

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Thanks for the info. Collada files show in quicklook too, which is nice.

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

You also get animated qtz previews (not talking about dae character animation to be clear, just a runtime preview) with DAE based qtz's or other qtz's in Finder and Quicklook now!

For example, preview this qtz with Cover Flow, or (highlight+spacebar for) Quicklook. Full animation of the dae based qtz. To be clear, qtz support in the department isn't anything new, but it's good that dae's were integrated and fully supported there.

PreviewAttachmentSize
CTA6000bicentennial 2.zip4.99 MB

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

If you guys look at Tablet Trail.qtz (or others that don't have a Clear patch on layer 1) in Cover Flow mode, in Finder, do you get something that looks like a render in image/no clear error? (graphics glitching out on the edges on this one).

Developer/Library/Quartz Composer/Patches/Tablet Trail.qtz

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

The cat is out of the bag... Finder v002 glitch texture for Snow Leopard.

dust's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

the thing that urks me is that system wide dae support does not extend to qc. why ? im going to file my first bug report with apple. i have tried mapping textures to dae, no diffusion, no specular, no bump, no reflection, no normal, no anay kind of map but 2d uv. ok i will be smart and bake my textures wow i can see the model in preview but qc renders it black. im frustrated, im glad there is k3d cause this mesh filter thing is over rated and really slow, try and add two filters together and watch your system crawl. so i don't want to rant anymore about negative stuff i need to get into a more positive light. here are some test mesh things maybe they will work for better for someone else.

here are some screen shots so you will see what im talking about as far as testing textures the last shot is one actually working in qc. i was actually able to get a few working, but i think the collada exporter is a tad buggy.

vade's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Post your dae file so others can try.

You dont get normal maps or bump maps for "free", those have to run in shaders (wrap your mesh renderer in a GLSL shader and feed the textures to GLSL for your normal & diffuse map).

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

What Vade said... If you don't have normals, and don't have a texture map going, they could definitely look black.

Did you mean to attach qtzs/zips on that?

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

It's weird that the interaction port isn't on "consumer" environments I think... not sure about that.

There's also a kind of fun use of editing something to be in an environment with the mesh filter previews (note that they are inherently in the lighting/shadow enabled environment).

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Oh I was just converting a model that had some basic keyframed transformation animation in it to dae, and the animation played in quicklook! But as expected there is no way to access this in QC from what I can tell. Still it means the underlying dae framework supports more than what is exposed in QC.

gtoledo3's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Does it do it every time or just when you were converting? Wild! I have some dae's that have animation and hadn't noticed that yet. I will be curious to see if it works on mine.

SteveElbows wrote:
Oh I was just converting a model that had some basic keyframed transformation animation in it to dae, and the animation played in quicklook! But as expected there is no way to access this in QC from what I can tell. Still it means the underlying dae framework supports more than what is exposed in QC.

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

It works all the time, the fact I was converting is incidental. At the time I was actually trying to find out why my modelling app of choice seems to be doing something to UV's or whatever that is spoiling my ability to use textures.

Preview and quicklook have consistently shown rotation & scale animations, I havent tried anything else.

dust's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

im in the lab right now when i get home i will upload the files apparently collada is trying to init my baked texture from a picture on my disk. i looked at the xml file and figured out some things. but yeah i did try glsl with mesh and that didn't work either but will try when i get home with apple protocol conformance. actually i have two dae exporters so i will figure out the info but yeah structured out my data so i know its all their plus text file says so as well

dust's picture
mesh file glsl displacement map

ok so on a more positive light, things are working. @vade & @gt you guys are are on it. i guess i was saving a test dae with no texture thus making no color info etc which is ok if you are messing with the mesh filters cause they are only working on the vertex data. now when i baked my model it was initializing my texture from a file url that qc didn't understand. so now i know that if i want glsl to work i need to export my .dae as a 2d uv sampler texture mapped to the color channel of my material network, the rest i can do in qc or well some of it i think. here i will give a example that is cool, maybe someone can explain to me what is going on. i have a low res poly model that i am displacing with a texture, this is awesome because i can gain more details but keep my poly count low. now i am displacing my verts by x amount but the qc lighting patch only sees the original model so the shadow looks as if it is inside my model ????

i like the shadow but i think i will have to make a shadow map or fake it with a png if i am going to do displacement. i feel this is the proper technique when doing realtime 3d stuff. please give me im used to doing multi layer hi res ray traces with clustered rendering farms for compositing any 3d work. im tired of waiting to see the outcome of a render just to find out something is wrong so the last year i have been getting into interactivity and realtime motion stuff so this is all new to me. i mean keeping a poly count low is a general rule of thumb that i don't practice but i am excited about displacement mapping.

so next im going to see about faking the displacement all together and doing a bump map but im looking at nvidia and the code really looks narley.

here are the displacement map and baked texture plus model plus glsl displacement function.

PreviewAttachmentSize
displacment_g15.400000.png
displacment_g15.400000.png450.24 KB
baked.png
baked.png2.55 MB
displace.zip5.88 MB

vade's picture
Re: mesh file glsl displacement map

Thats really nice.

Im not surprised the Lighting patch does not support the programmable pipeline effects at all.

I'd suggest using a normal map rather than a bump and calculating per pixel lighting in the fragment shader. You can also look up parallax mapping which makes things pop a bit more and you can have parts of your normal map occlude other parts so at very oblique angles it looks like it has more depth.

Ill try and throw a demo together but I don't have any modeling apps anymore, plus I don't really know any of them all that well.

I do have working parallax mapping code though. You will get what looks like shadows from that. :)

dust's picture
Re: mesh file glsl displacement map

I have been looking it up apparently the problem with vertex displacement inside a shader is that the deformations of the vertex mess up the normals and faces so and from what im reading you can't the vert data to recalculate the normal for the processed faces which doesn't really make since but then again i just started messing with glsl. i was scared of it for a bit because its all these 4d vectors and i had a go at trying to iterate through 2d vectors of arrays with the stl in c++ and well its hard to visualize.

if i use a light map then the normals are useless anyways. i guess the solution is to calculate the normal from the displacement map but that confuses me even more cause this example is converting the rgb space to grey values for displacement thats why the displacement map is grey and the normal is blue. it kinda makes since to me a little bit.

its making more since though seeing that i understand how to model (visually) create my own shading networks procedurally &(visually) + rig and animate and do all those things that the movie and gaming guys do. they just do one thing though. im a modeler, or im a rigger, or a lighting guy no one does more than one thing. so i understand whats going, im just not used to doing 3d programmatically. so all these functions exist and im trying to make use of them. its actually nice to understand how the 3d suites are working under the hood or giving you your openGL view before you hit render.

I used mud box to make this, if you can get a hack or something it is awesome. i think it is the most simple elegant modeling package there is its kind of like z brush but true 3d plus all the controls are set up like maya which im used to.

so i got bored of looking at that sphere and was to lazy to make a new model cause its all about the next thing not something old for me so i wiped up my own sphere to test with.

so its like push and pull play with mud, i make the model the same way the displacement map is working with sampler2d. its a process you make supper high res model get supper detail paint sculpt paint then extract the normals and displacement from the high res model and apply it to a low res model. (in qc)

plus all the specular, diffusion, bumps, etc are made the same way painting. then put them all in maya and make another shading network with all the 2d textures by spherically or cubically projecting them, once that is done i bake the texture down to one uv map and in this case apply the displacement map in qc. i would like to skip the baking maya step and go strait to glsl. i know it can be done. im just learning it, so it will be some time.

i have all ready made some more uniform samplers and will be making the shading network in glsl the same way i do in maya and mud box. it sucks to get this you have to build 3 shading networks. it would be nice to just make one but to get this look you have to do that its not just wrapping a photo around a model which looks good to.

i want to add dual paraboloid environment mapping to particular regions of the model kind of like the shiny metal with rust all over it look. ? the parallax is a good one to have to tell you the truth. it would come in handy for fish eye stuff camera stuff i think.

so i really think the problem is using displacement and it is inherently glsl because i can apply a normal bump and a displacement with volumetric stuff all day in maya. there has to be away to pull the normals and the extra vert data out of the shading model and feed it back in ?

but thanks for the answer i posted the question to the list as well. it was confusing my why the shadow was not working.

dragoonreas's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I've been testing out snow leopard's new mesh creator patch, and it appears to have a huge memory leak. I'm taking a queue of an audio spectrum and using javascript to generate the vertices and colours for the mesh creator. After leaving the composition running for about an hour, activity monitor reported that quartz composer was using 1.26 GB of real memory and 2.68 GB of virtual memory.

If anybody wants to look into this some more, I've attached the composition I made below.

PreviewAttachmentSize
Audio Mesh.qtz31.56 KB

SteveElbows's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Ive seen a leak too, with a music visualization patch I was mucking with. My composition doesnt use meshes, but it does use OpenCL. I'll try to track it down if I can, but if anybody else wants to try, my composition is attached to the thread about the 2D Fuid Simulator.

I also had someone tell me that my composition was using 580% CPU on a Mac Pro with ATI 4870, which is rather odd as it only uses 25% CPU on my Macbook Pro with 8600 graphics.

cybero's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I've only just downloaded the test file you uploaded, dragoonreas, but something of what you posted reminded me that when I finally had a spare minute from my paperwork and got around to checking in the Console apropos the Snow example from the Developer site bugging out on me, 'twas all my bad - well the plugins that should not be installed as they aren't SL or CL compatible.

Just place any that you might not have seen as suspect into a (Disabled) patches or plugins folder.

Now I'm going to have to leave this running for an hour, shall let you know more when I know more.

Cheers.

:-)

cybero's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Well I've given your file a run through and tweaked the item a little, whilst seeking to retain much of the Options functionality you had in place.

At QC start - preferences set to nothing open at QC start up, the Activity Monitor shows QC as RM - 13.4 MB, VM - 2.66GB - with nothing running, SM - 0 bytes, PM - 3.1 MB & VPM - 22.6 MB.

Then we start the QC file you uploaded running in QC, the following is obtained from the Activity Monitor -

M - 98.2 MB, VM - 4.83GB - with nothing running, SM - 0 bytes to 2.3MB - max - approxswitches to EB occasionally , PM - 82.7 MB & VPM - 220 to 300+ MB + -> VPM switches to EB measure occasionally

The VPM holds pretty steady slowly incrementing in process.

I got this message in the Console

06/09/2009 14:37:35   Quartz Composer[436]   *** Message from <QCJavaScript = 0x18C43570 "JavaScript_1">:
TypeError: Result of expression 'spectrumQueue' [null] is not an object.
06/09/2009 14:37:35   Quartz Composer[436]   *** Message from <QCJavaScript = 0x18C43570 "JavaScript_1">:
Execution failed at time 0.000

When I ran my tweaked variant of your original post I got this console message which I guess is something else for me to look at in the future.

06/09/2009 14:43:04   Quartz Composer[436]   
06/09/2009 14:43:04   Quartz Composer[436]   WARNING: vertex shader writes varying 'VertexPosition' which is not active.
WARNING: vertex shader writes varying 'ViewDirection' which is not active.

I have attached the tweaked version for your comparison purposes, but I think you might well find that the main problem you have is that , just as it is a wee bit inefficient to nest JS patches in Iterations, we might find something similar applying to nesting JS in Macro Patches which JS the structuring/rendering of the vertices requires.

PreviewAttachmentSize
Audio Mesh2.qtz584.86 KB

dragoonreas's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

Sorry it's taken me so long to reply, been a bit busy lately and haven't had a chance to come back to this. Thanks for taking the time to look into this a bit cybero. I implemented your tweaks to get rid of the empty queue error on startup and the option to use a volume peak queue rather than the spectrum (I liked the look the volume peak queue produces).

I found those vertex shader warnings only occur when the mesh creator's drawing mode is set to Volume. I wasn't really testing that drawing mode as I don't actually know what to send to the Volume port. I'm guessing that sending the required data to that port would fix those two warnings.

As you suggested, I moved everything into the root of the composition to see if that made any difference (the new 'explode macro' feature was very helpful for this), but unfortunately the memory leak still persists. I've attached my updated composition below.

One of the reasons I took so long getting back to the Audio Mesh thing was because I had the sudden urge to make a fractal tree screen saver after coming across this screensaver, and subsequently finding that there was no Snow Leopard version.

Once I was happy with it and had checked that it worked properly and didn't produce any console warnings / errors (or have any memory leaks), I set it as my screensaver. Now up until this point I'd only been testing it in quartz composer and it had run fine both windowed and full screen for hours without any problems. But when I set it as my screen saver, I found it uses twice as much CPU as it had in quartz composer, and has a memory leak even bigger than the one in my Audio Mesh composition.

This has really got me puzzled because there isn't even anything in the console logs about what's going on. Again, the composition is attached below if anyone wants to have a look at it.

PreviewAttachmentSize
Audio Mesh 1.1.qtz23.57 KB
Fractal Tree ScreenSaver.qtz92.11 KB

stoney's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I've just discovered that OpenCL does not run on the ATI HD 4870 in OS X 10.6.1. It's supposed to, as it's listed as a supported card.

Any idea what's up with that?

I'm thinking of adding an NVIDIA GeForce GT 120, as it doesn't need aux power cables (both mine are used by the ATI). Is it likely that this card would be used for OpenCL even if it had no attached displays? Some people opine that you can't put an ATI and an NVIDIA in the same Mac. True?

cybero's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

That's puzzling. So it was supported on 10.6 ?

Seems like a lot of cards are becoming adversely affected with successive SL versions, 10.6.0, then 10.6.1

Have you had a look at your Console messages, any invalid plugins and such?

Try disabling those mentioned and try running again.

Have you tried making a backup of the card's .kext and then deleting that .kext ?

Don't know about heterogenous carding in OS X Intel machines.

stoney's picture
Re: Quartz Composer Snow Leopard - New Patches and Features

I've read elsewhere that something is wrong with the OpenCL support for that ATI card, so they disabled it in QC, at least. I poked at it with some modified OpenCL example apps, and it sort of works, but anything significant causes errors. I saw one note that it has been fixed for 10.6.2, but there's no developer beta for that yet, so that may be speculation.