Composition Tool...

This tool doesn't modify the composition at all, it just inspects it.

redundancy analysis would be interesting, but it's a bit difficult. The 3D transformation stuff you mention, while easy to grasp, is actually embarrassingly cheap to do (it's simple matrix multiplication, and for any non-intel video card, it's all done in hardware, and on intel hardware, it's sse-optimized, so it's still cheap until you start working on tons of geometry). Stuff like disabling sprites/billboards/objects that are offscreen are helpful, but difficult to solve (once transformations take place, it's difficult to know if it'll remain on screen or not, esp if you use FoV changes, or GLSL vertex displacement).

image processing (blurs, etc) are smart enough to only happen when the input image changes, so that's already fairly well optimized. iterations are slower than all get out for undisclosed reasons, perhaps having to do with tweaking the QC graph.

you can open and edit the composition plists, it's a pretty straightforward format. however, optimization is tricky to. There's a lot of info on compiler theory that could help, but it's still a bit abstract when applied to QC (taking stuff out of loops = take stuff out of iterators that doesn't depend on iteration count, pre-calculating stuff that can be known in advance, etc.). At the end of the day, there's only so much that can be done automatically. If you add 3rd party patches to the mix, you have no idea whether or not they are stateful, and thus whether or not they can be modified/moved without adversely changing things.

cwright wrote:
Stuff like disabling sprites/billboards/objects that are offscreen are helpful, but difficult to solve (once transformations take place, it's difficult to know if it'll remain on screen or not, esp if you use FoV changes, or GLSL vertex displacement).

Actually, that hits the "nail on the head" of what I am getting at with the 3D transforms, and I have thought before that is what really brings the whole environment "down"...

I would like to be able to setup a really complex scene at say x=-20,y=0, z=+20 ... and slowly zoom in to lets say z=-20. Then I would like to be able to "pan" over to x=0 and have a different scene. Then raise up the y position, for yet another scene....

Those are the ideas I would like to be able to have without the tremendous "hit" it takes on the frame rate and system.

Maybe there is even a logical system of using render in image and multiplexer, with certain math functions to toggle the renders/patches "on and off" as the 3D transform (or your GL Ortho, FOV,etc. patches for that matter) moves to certain positions?

Or for something simpler, say you have one 3D transform with 5 or so 3D transforms nested within. The macro 3D transform would just shift x position back and forth... only the 3D transform that coincides with the current view would be active...

Would doing something like that allow many scenes to be scrolled through, and maximize frame rates? Is almost seems like it would be possible right now, but I don't know how to right off the bat, and I'm not sure if it would actually make things "run faster".

I swear that it seems like some of the comps that take a little bit of time to start to render, go into render mode a little quicker when they have been flattened, and the frame rate is less variable... I trust you when you say it is placebo, but I am definitely going to check this again...

grouping scenes, and disabling them once they were no longer of use, would dramatically improve the framerate (any possibly memory usage, depending on how the nodes respond to being disabled -- most release their resources, but not all).

grouping can happen with a 3d transform, with a simple macro, or any other patch with subpatches -- once the parent is disabled, all the children automatically get disabled.

the math stuff necessary to calculate on/offscreenness is possible, but it's extremely difficult to chain together into a magical "just works" generic solution for everything -- you're much better off manually coding it into a composition yourself, knowing the situation a bit more specifically. There's still no way to generically handle arbitrary 3d objects (since they can be of any arbitrary size), so the generic tool would fail to even disable those (or, it would have to do nasty hacks to intercept its polygon data, and then make assumptions about that data.... by which time, you wouldn't really save all that much)

Oh -- flattening. Wasn't sure you were using that feature of composition tool :)

That should lower load time, and maybe help execution time (assuming that there are actual patches to flatten in the composition). That's simply because there are fewer files to access, and less "glue" internally for QC to apply to make stuff work seamlessly.

If there aren't any flattenable patches (only Virtual Patches get flattened), the output should be identical to the input.

Please post your sample data, so some regressions can be performed on this (perhaps flattening does some other cleanup as well? we're using QC's internal secret flatten stuff for that, so we don't know exactly what all Apple does inside :)

Yep, you have reconfirmed my worst suspicions, that it has to be totally per composition. Which I can already figure out how to do....

Darn Chris, I was betting you had one of those Einstein shortcuts.

I can see that anything with 3D and any of the SSAO/ shadowing of complexity would need to take that into account... save for stuff like those UFO rollercoaster clips I did, or the Kinemekart clip I saw Steve do recently.

I guess from a "coding/efficiency" perspective, it always makes more sense to have a simple backdrop that moves, while the character remains pretty much in the same place on the screen doing minimal movement, with basic image transitions/wipes to another simple scene.

I am kind of going down a different alley here, but I was thinking that something could be developed that would be a cross between your texture patches set, and the GL Height Field apple example, to create generative landscapes using simple photo/texture inputs? That is my non-technical way of thinking, I'm sure there is a more efficient way of approaching that? It is obvious that planes can already be deformed by all kinds of methods, including GLSL shaders and whatnot, but I wonder why the GL Height Field can't seem to be wrapped in a shader?

Well, if apple wanted us modeling, they would have had as a standard more 3D objects than a teapot and a few prims. It's either that or the graphics card manufacturers don't have any standards.

That said, GLSL + libnoise = win.

Well, what is funny, is I am pretty darn sure that there weren't even virtual patches, because I'm not "big" on using that feature. I think the only virtual patches I have installed are the defaults, plus one, just to see how the feature works.

I'll send you some before and after's.... I'll try to find some kind of horrible fps qtz.

What I would say, is that sometimes you may get to a part of a composition where there is a dip in fps or a little quirk because of abusing some kind of CI Filter heavily, or the aforementioned issue of having many objects floating around in 3D space...

I might run something that is abysmally inefficient, running really slow, with occasional rainbow wheels. When you flatten it, it may run slow but you don't seem to get any "halting".

When you flatten it, the frame drop might still be there all the way on some compositions... but on others it seems like it just smooths it, and makes it run more "average".

Yep....

When you think about 3D modeling, from Apple's standpoint, you probably have to get some kind of alliance with a major 3D format, and a real "system" going before just introducing 3D importing "willy-nilly". There are a lot of 3D file types out there, and if Apple was to put it in QC, I would think they would want to have a lot better shading and lighting capabilities than they currently have. That is somewhat of a chore I think.

Don't you think that the lack of 3D support in QC is kind of a nod to their business partners at other software companies? QC is pretty simple (for me) to use after being really familiar with all of the Adobe stuff... so I think the average "non-developer" person would have no problem just using QC for everything if it was a little more robust out of the box.

I was looking at some of the old Apple Quickdraw 3D (I think it was) documents and noting how you could easily export your projects to either Apple or Windows... did we move backwards :o) ... I know that was a much more limited system though.

Oh and GLSL+libnoise=win = -me :o)

I just hate going back and forth, and really don't like working on Windows. I'm not a Windows hater though, it's fine for cross platform apps that I am familiar with, etcetera.... I just try to avoid making situations where I am going onto XP or Vista for any part of daily workflow.

I know how you feel, I'm back and forth between the ease of use and power of quartz composer, and the cross compatibility and web happiness that is Java/Processing.

I wish it worked both ways, but I'm an artist, not a programmer and I just don't have the time/energy/mathematical IQ for C++ or Objective C.

The primitives they've included are all part of the GL standard (there's a teapot function, and quadrics that give you a sphere or cylinder... in Tiger, there was a torus too, but that was mysteriously deleted for Leopard). cubes/planes are, of course, embarrassingly simple too :)

They want more 3D object things, they just wanted a unified way to solve a few problems:

1) have a decent data format (they've decided this for Snow Leopard, according to some mailinglist leaks) 2) have a decent way to manipulate the data (currently, the only structure modification stuff in QC is the javascript patch, which is terrible for non-trivial datasets). This is solved in SL too with OpenCL.

Graphics card makers have lots of standards, (OpenGL, DirectX), that's not a factor in QC's lack of 3D stuff -- it's the above limitations (which are now solved), and the lame fixed-layer stuff (which isn't solved, but can be hacked around I guess).

I'm intrigued by your reference to OpenCL handling large arrays of data there Chris.

I've just been thinking of sitting down and starting to write some 'spreads-style' QC plugins to generate keyed data structures (which Toby's very kindly offered to help me with).

I know there was talk of a new faster QC datatype to handle arrays of values, but this is as much a exercise in Objective-C, from my point of view, anyway.

Do you anticipate OpenCL making manipulation of large multi-dimensional arrays of arbitrary data possible on the GPU, say through a QC plugin?

a|x

shading is what it is. What more do you expect? it's essentially raw GLSL in QC (with the exception of the fancy texture types, like cubemaps etc). Lighting is too -- OpenGL's native lighting stuff is exactly what the Lighting patch does (8 hardware lights, with a location, color, attenuation, ambient, and some material properties like specularity).

The only way to get fancier stuff is to get your hands dirty, and code it yourself -- shaders are shockingly application specific, so having a zillion pre-fab shaders wouldn't actually be all that helpful (much like the primitives in QC, which are pre-fab, and mostly useless... I mean, come on, a teapot? what good is that?)

(cue the rant about how teapots are complex shapes that demonstrate somewhat complex surfaces, I know that already. But how many real applications of them are there?)

QD3D was Yet Another 3D API from the 90's. Good riddance (along with Glide, and custom software rasterizers for games like Quake, Descent, and Doom). Hardware makers have a difficult enough time as it is just supporting GL and DX, having more crap wouldn't make anything better. From a developer's point of view, ditching QD in 1998 was a wise move that allowed Apple to take advantage of Real Video cards a few short years later. And, more importantly, it made porting stuff much easier -- OpenGL code works on Win32, Linux, and OSX without any need for custom tweaking.

If anything, the lack of 3d support in QC is the result of the 2 points I mentioned in another post, combined with a shortage of developers to complete the task. Making a coherent, flexible, fast 3D toolkit is a fairly large initiative, after all :)

That crosses into NDA territory for me, but to be brief, yes, it'll almost certainly be possible as a standard (GLSL has been a standard feature of QC, I don't see why they wouldn't use CL as well).

(though still structure-backed I'd imagine, with all its caveats and limitations -- spreads will still be viable and superior for Really Large Datasets.)

Yeah, shading is shading, but I want a shader/lighting combo that intelligently matches the GLSL type shader lighting controls macros to the the lighting macro light source controls, and also has a shadow control. It would be just like the lighting, with a shadow control, and a GLSL shader "link on/off" without having to manually tweak... like an "intelligent mode".... that would also interact correctly with the 3D transforms...as well as lights from sources, and global lighting too. I know I can publish inputs to both, control them together, etcetera, but I would just like a simple "mode" where the Lighting, GLSL, and Shadows all respond to each other in the correct directions. I know you explained how this isn't really feasible in QC and the point is taken, but I can still dream :o)

I'm sure it is my quirkiness that is causing me to "want" that feature, but it seems like it would be great.

As a matter of fact, GLSL shading in QC is always kind of irritating to me... is there even a decent hair type shader that can run in QC? I don't know, I'm just ignorant of that....

The thing about what I am saying is that, in some cases I think the shader code (GLSL or whatever) would have to actually be able to dynamically change in reference to the number of lighting sources for some types of shaders. (Yeah, yeah, I'll go look at Blender or Maya...lol).

There have been some interesting leaks about the OpenCL and the upcoming 3D data formats in QC (hint ... rhymes with scollada?), but I think that it sucks to have to wait :o( I'm really anticipating the feature set, and what kind of frame rates/stability this OpenCL will add.

Somebody at Apple... send me the seed for free and I swear I won't tell.

BTW, this is getting way off thread, but I think that javascript in the latest build of QC might just be hosed up. Is this even possible?

I notice that some of tb's stuff that worked fine for me in the past is messing up when I change iteration amounts...

When this was happening to me and a couple others on the SSAO stuff I didn't think much about it. But now I it happened with one of the Superspread qtz.'s (I think). The second I changed iterations a couple times (in low value ranges)... and after a few more seconds it crashed.

I'm just wondering if some of these updates are slowly laying groundwork for bigger changes, and if stuff is getting messed up in the process?

My guess is it is totally unrelated though.

"shadows" in GL are a pipedream -- there isn't a magical way to "enable" shadows, because they're very difficult to do. Handling them requires knowledge of light sources and all objects in the scene simultaneously. QC can tell you about the lights, but can't tell you about all the objects in the scene because they aren't all tied together into a single "scene" object renderer thing. Then, what happens if you want soft shadows? what about global illumination? refraction? reflection? parallel planes that are perfectly reflective? These are all variations on a theme, and there isn't a good way to solve any of them quickly (slowly, you can do raytracing, but that's beyond the scope of this discussion). Might as well add Depth of Field to the list too.

GLSL knows about light positions and 3D transforms, as long as you code the shader to look at them (most shaders I've seen posted don't do this, but it's not a weakness of glsl, it's lameness on the part of the shader that's been written) See example composition -- note how the glsl part has exactly zero inputs for the light position, and exactly zero inputs for the 3d transformation stuff, yet it all behaves correctly -- all automagically working.

You're right about the light number thing -- glsl supports conditionals though, so it might be possible to work around this (secret: fixed function stuff nowadays is implemented in glsl!), and have changing numbers of lights Just Work Too. It's just a matter of coding cleverly.

Fur shaders are probably a no-go in QC because all implementations of fur shaders that I've seen rely on being able to render the same object multiple times with vertex displacement to create the fur strands. Once geometry shaders become possible, it'll be easy for the shader to emit new primitives to make this possible (not sure if this'll be a snowleopard feature or not... conceptually it's been around for along time, not sure why no one's gotten around to doing it yet...). You might be able to hack it in via parallax mapping, for example, but I'm not sure how convincing that would look as the angle of intersection got farther and farther from 90°.

Stability typically hasn't been a problem with QC (except for a few corner cases that we've pretty well documented on this site, as well as on the list). I don't know that adding more features ever helps stability, but I don't think that it will hurt it. CL's framerate performance has yet to be demonstrated with anything other than toys, from my personal eye-witness accounts.

there are a number of subtle quirks in the javascript patch that stretch all the way back to Tiger. You probably formatted something strangely to trigger one of them. (hint: changing the main function declaration line's spacing can trigger some bugs). post the crash report, and we can see where it's dying (and possibly why).

Looking at the initial post date I'm guessing this has been superseded and that we can get some of that flattening loveliness just by creating macros of the main composition body and elements.

However, just in case there is such a tool available, where is it ?

:-)

cybero wrote:
Looking at the initial post date I'm guessing this has been superseded and that we can get some of that flattening loveliness just by creating macros of the main composition body and elements.

Flattening isn't really macro related, it's virtual-patch related. Flattening will inline virtual patches, so the composition can be distributed without worrying about virtual patch stuff (the commissioner had several hundred compositions all using virtual patches, and needed a way to quickly flatten all of them).

cybero wrote:
However, just in case there is such a tool available, where is it ?

https://kineme.net/CompositionTool20080724beta

... if you use it and notice that performance is any better with the flattened stuff, report back! It's a curiosity of mine about that... sometimes I think it makes things run more solidly, but I think that's tied to what you're doing in the editor... not sure. I haven't spent any real time profiling what happens in different scenarios of manipulating something that's flattened vs. not flattened. It usually "feels" more solid for some reason (could be total placebo in some situations... definitely got better performance from some particle tool stuff though).

Thanks indeed, cwright & CoreMelt.

Shall update you , gtoledo , et al, regarding improvements achieved.