Unsupported — We cannot guarantee that this software will work properly on Mac OS 10.8 and above. Please be careful.

Release: GL Tools, v20080226

Download 20080226

Release Type: Beta

Version: 20080226

Release Notes

This beta's pretty rad. It adds a few new features, and a couple powerful patches.

The glOrtho patch now allows you to disable ortho mode (so you can switch from ortho to perspective mode without duplicating all the rendering in your composition).

But the coolest part of all is the GL Point Structure and GL Line Structure Patches. These patches allow you to render collections of points, line segments, or a continuous line strip without the overhead of an iterator.

Both patches take the same structure format: a substructure for each point, with an X, Y, and Z value. If the substructures are keyed, the keys "X", "Y", and "Z" are used, otherwise the 0th, 1st, and 2nd structure members are used to derive the coordinate. I'll probably upload some sample compositions to better demonstrate tomorrow morning.

Using JS to drive it is still slower than molasses, but it's an improvement none the less ;)

Before things get too crazy with structures, I'd like to have a plan in place to duplicate the functionality of vvvv's spreads (these are how this sort of stuff should be solved, not by more structure hacks....)

Preview	Attachment	Size
	GLTools-20080226.zip	150.12 KB
	PointAndLineStructure.jpg	117.99 KB

By cwright at 2008-02-26 20:55
GL Tools
glortho
orthogonal
perspective
10.5
Beta

JS needed

could you post the JS to drive the structure ?

i didn't find a way to generate a correct structure type (sub-ed) with non-JS patches -

By franz at 2008-02-27 10:01

beyond sample

actually, i have 3 queue patches that generates 3 structures (one structure for X, one for Y, one for Z) of 100 elements. Any idea on how to make a JS that merges these 3 structures into one in a friendly manner ?

By franz at 2008-02-27 10:29

segue

Nice segue into the vvvv Spreads discussion... ;)

This is why I held off on the cubes patch: I feel that the nicest solution isn't a huge monolithic structure, but a series of simple, linear structure (stream of X's, Y's, Z's, colors, numbers, etc). That way we keep the same inputs, they just take a spread input instead of their basic type.

I'm a bit hesitant on this for a couple reasons:

First, None of the existing patches will handle this new type. We can hack in graceful fallbacks (spread of numbers returns first number when it's accessed as a number instead of a spread), but that may cause unexpected behaviour.

Second, When/If Apple decides to implement this behaviour, it'll effectively fork functionality (unless, miraculously, we design ours exactly like Apple will, or Apple uses our design (laugh), or they never implement such functionality ever). The massive duplication of effort on that worries me a bit.

You could make a JS patch that iterates over the structures, and assembles them into one huge structure. Or, I could make some more StructureMaker patches that would combine elements or something. It's difficult to make a generic solution that works for everything I guess (since I have no idea how people plan on using this -- your use-case is an example of unforeseen usage, and it took all of 30 minutes to become apparent ;)

By cwright at 2008-02-27 10:37

mimic

mimic'ing vvvv behaviour is -to my opinion- the smartest thing to do.

This would give:

A K. GL point patch for instance, whose inputs (X or Y or .... whatever) can accept: 1 regular "single element" datatype (aka normal behaviour) 2 structure containing multiple elemnts of the same datatype (ex: 1 structure containing 100 X coordinates)

If a structure is detected on an input, the according number of copies is generated.

If different structure counts are present, the number of copies is cleverly dispatched " a la " vvvv. See concept here: http://vvvv.org/tiki-index.php?page=Tutorial+SpreadsII

And this would only affect Kineme patches, and leave apple's stuff untouched.

By franz at 2008-02-27 11:27

Structure vs. custom

There are some annoying problems with QCStructures that we could work around at this point to. First, unordered sets is stupid. Second, a simple NSArray (or even C Array!) would do the job, and be extremely fast for very large datasets (there's a 5-15ns penalty each time you pull anything out of an NSArray/QCStructure. While by itself it's a very small cost, doing it thousands or millions of times adds up very quickly).

This would make the port incompatible with existing stuff (autogenerating data in JS, for example).

.... I guess we could accept 3 kinds of inputs: raw numbers (couldn't be input by clicking and typing since it'd be a virtual port...), QCStructures, and some other structure (kinemeSpread ;) that we develop later. does that sound reasonable, or is that over-engineering?

By cwright at 2008-02-27 12:00

sounds reasonable

that sounds clear to me, provided it doesn't make too much work for you.

By franz at 2008-02-27 12:07

Sample JS

Out-of-the-box, structure works with OpenCV's point output structure (not so much for hough lines), as well as smokris' 2000points demo (for showing when JS and iterators are lame ;).

Here's some sample code

var result = new Object();
result.points = new Object();
 
function (__structure points) main (__number patchTime, __number pointCount)
{
   var pc = 4/pointCount;
   for(i=0;i&lt;pointCount;++i)
   {
      result.points[i] = new Object();
      result.points[i][0] = Math.cos(i * pc * patchTime)/2 + Math.cos(patchTime/2)*0.01;
      result.points[i][1] = Math.sin(i * pc + patchTime)/2 + Math.sin(patchTime*3)/100;
   }
   return result;
}

Current problems: JS's structures are unordered (they're not output in the same order that you create them int), so you have to pass them through the Structure Sort patch. As I discovered yesterday, Apple's Structure Sort patch is leaky, so it's not a sustainable practice until Apple updates QC's framework (Troy hinted at the next software update, but I don't know, and I don't want to bind him to anything).

By cwright at 2008-02-27 10:09

CV Sample

Here's a CV Sample: you click to define line endpoints, and those points get tracked. You can wireframe objects and all that jazz. You'll need OpenCV, obviously, for this to work :)

Might act weird when you enter a large number of points or right click... I haven't tested it exhaustively yet, as I wanted to just play with the concept/interface.

Preview	Attachment	Size
	ScribbleTrack.qtz	10.33 KB

By cwright at 2008-02-27 10:19

nice sample

awesome CVsample. I guess it took you quite some time to make -and comment- this graph. Thanks for the effort.

As a sidenote, there isn't any option in the "OpenCV Calc Optical Flow Pyr LK" patch to clear tracked points - or is there ? While there is an option to add points... a reset would be cool (or i missed it...)

By franz at 2008-02-27 11:37

cv request

I've noticed the inability to clear the optical flow patch too.. no way to clear it as far as I know (attempting to load an empty structure does nothing, due to some internal design stuff.) Please note this on the cv beta, so I'll remember when I get back to hacking at that plugin :)

It's a feature I want as well :)

By cwright at 2008-02-27 11:52

there's a bug somewhere - in

there's a bug somewhere - in the JS or the plugin - set the points number to 2000, then to something lower like 10, and the results aren't updated correctly (i heavily suspect the JS)

By franz at 2008-02-27 10:26

js bug

That's because of the init stuff being outside of the function -- it's not cleared each frame, so the stale objects stick around. put the top two lines inside the func, and it should be fixed.

By cwright at 2008-02-27 10:28

Lit wireframe

Is that possible ? It would be really handy (at least for me ;)

Preview	Attachment	Size
	Picture 1.png	143.53 KB

By franz at 2008-03-08 12:19

2-pass

This is a multi-pass effect (one pass draws the wires, the other pass draws the polys). QC doesn't have built-in multipass support, so you either have to render twice manually (once in polygon mode wireframe, and once in polygon mode textured), or do some other similar hack.

By cwright at 2008-03-08 12:28

GLtools enhancement...

Image input for subdivision spline points... Or structure coordinates outputs from subdivision points of the spline patch (something that can be input to the point structure patch )

here the idea : (see the file)

Preview	Attachment	Size
	MAGIC_SPARKLE_LINES_HQ_by_IHEA.jpg	316.37 KB

By yanomano at 2008-04-20 08:51

good idea

That's a wonderful idea (i.e. not difficult to code ;) I'll add that to the todo list.

By cwright at 2008-04-20 08:59

Fantastic !

I like the "something to code when waiting my girl friend things"...;)

By yanomano at 2008-04-20 09:04

Spline Subdivision Points

A cool parameters should be a decrease size option for the points: From first point, From last point, from first and from last... See the file...

yanomano.

Preview	Attachment	Size
	Stalks_2_by_melemel.jpg	21.87 KB

By yanomano at 2008-04-20 09:17

Very nice. These properties

Very nice. These properties would look very nice animated, too.

alx

Quartz Composer Blog: http://machinesdontcare.wordpress.com

Music Site: http://www.toneburst.net

By toneburst at 2008-04-20 09:25

parametric

Even more general, it would be nice to be able to arbitrarily control the size of points directly. Later in this thread, you mention using the alpha channel to control size; Putting aside the possible performance impact of on-the-fly state changes, I think this is really one of the best ways to do this (other than using structures)

By cwright at 2008-04-20 09:31

I misread this the first

I misread this the first time and thought you were talking about using an image input to define locations of spline points..

Which gets me thinking.. This misinterpretation might be useful in getting around the performance issues with using QC structures to represent points (and using the slow Javascript patch)..

We could, for example, write a CoreImage Filter to render a 1000x1 pixel image, where each pixel's color maps to the location of a spline point (red = x, green = y, blue = z), and then feed this to a new GL Spline patch which accepts this image input, renders it to a system-ram buffer, and feeds it to GL.

By smokris at 2008-04-20 09:15

Image to define 3D coordinates

Funny that you talk about that because, i think sometimes to this...vec3 to vec3 this is easy ! Why not using a image vec4 ? Alpha channel value for the size of Gl points or something else ? (Think to a seamless gradient in the alpha channel ...woo!)

By yanomano at 2008-04-20 09:25

GL state swizzling

GL Point size is part of the state, not the data (vertex coords, texture coords, normals, etc are all data), so changing that for every point would be bad for performance. It'd be possible to manually track the point size yourself to not redundantly set it, but it'd still be a bit iffy... I'll try some experiments and see what happens.

By cwright at 2008-04-20 09:27

YES YES YES!! This is a GOOD

YES YES YES!!

This is a GOOD idea! Or you could make the 1D image into a VBO then render directly as vertices. If you could also texture the line with the input image, you'd definitely be onto a winner!

Having setup this system though, there'd be no reason why you couldn't use a 2D image as well as a 1D one. What you'd essentially have is a VBO-based Height Field plugin. And most of the code for this has already been done with the Apple GL Height Field example project in the Developer/Examples folder. It just needs a few tweaks in order to be able to texture the height field with an image, and some other bits of pieces to make it work in a GLSL shader patch, and there you go!

alx

Quartz Composer Blog: http://machinesdontcare.wordpress.com

Music Site: http://www.toneburst.net

By toneburst at 2008-04-20 09:23

definitly a good sunday :)

this is awesome....converting image to 3D structure input make me dream....

By yanomano at 2008-04-20 09:27

I say a 'few tweaks

..to make it work inside a GLSL patch... Of course, I don't know exactly what would have to be done to the Height Field example code to make it play nicely inside a GLSL shader. If I did, I'd probably have had a go at fixing it so it did...

alx

Quartz Composer Blog: http://machinesdontcare.wordpress.com

Music Site: http://www.toneburst.net

By toneburst at 2008-04-20 09:59

possibly

I thought about this initially too; it's kind of niche, but I can see some cool applications of it (as you mentioned, with coreimage filters etc).

Trading GPU rendering -> system ram to work around QCStructure performance will probably be break-even at best (vram/system ram locking is horrific), so that's probably not a good motive. HOWEVER:: abstracting this a bit further, we could apply the R2VA stuff (typically used for heightfields) to make splines instead, and that might make for some interesting uses of a full-hardware geometry generator. This idea of course isn't fully developed, but it would be pretty slick, and much faster than having the CPU do spline interpolation for a 1000 point spline (with 200 subdivisions, that's almost a quarter-million 3D LERPs per frame - perfect for a vertex shader, not so good for a cpu...)

[edit: or, in short, what toneburst said above ;)]

By cwright at 2008-04-20 09:25

KinemeGLline no opacity ?

It seems there is a bug with the gl line opacity.... just not working ....

By yanomano at 2008-04-26 18:20

strange

You're right; It looks like it's hard-set to some inbetween value (maybe 0.5 to 0.7?).

Looks like this is tied to how the blend mode input works; if I add in a glBlendFunc(...), it works as expected, but it overrides the blendmode input. For some reason the blendmode isn't setting the correct blend func now... (blend mode is entirely Apple's code; we just add the port, and have it do its thing. can't really screw that up.)

So, is it kosher to remove the blend mode input and do the right thing manually for the next release?

By cwright at 2008-04-26 18:36

GL Point X AND Y Dimensions?

Does a GLPoint have to be square? The Kineme GLPoint implementation has only a single 'Size' parameter. Just wondering if it would be possible to set X and Y dimensions independently. This would make it easier to use it as a source for a kind of 'screen-aligned quad', with images or the output of a CIFilter applied.

Maybe the plugin could automatically pad-out two of the edges of an applied image with transparent pixels to maintain the correct aspect-ratio, if this is not possible.

Just a thought. I know it's possible to do this using a Render In Image with a couple of billboards inside, but it would be nice to have the plugin do the fiddly work, so we don't have to ;)

alx

Quartz Composer Blog: http://machinesdontcare.wordpress.com

Music Site: http://www.toneburst.net

By toneburst at 2008-07-15 16:38

single-size

Points are uniformly sized -- in OpenGL there's only one parameter, http://www.opengl.org/sdk/docs/man/xhtml/glPointSize.xml This applies to both width and height.

It's difficult to make them line up, since they're in 3D and can be transformed by the ModelView matrix. Renderers also implement sizes differently -- some center (x/y +-size/2), some make it x/y + size, and some don't do it right at all. On mine, it's actually round, not square...