Capturing Geometry For Multipass Rendering

toneburst's picture

Is it possible to 'capture' the geometry created by a GLSL vertex shader in some way, so that it can then be rendered more than once, or iteratively passed through another GLSL shader?

I know multi-pass GLSL has been discussed a lot in the past, but it's just occurred to me that a cool way of implementing this kind of feature in QC might be to have some sort of rendering environment and source patch for Kineme 3D.

You'd place geometry inside the environment patch, and optionally a GLSL shader to mess around with vertex positions, and the result would be captured somehow to some kind of memory on the GPU.

Then, you could use the captured geometry and texture(s) as a source in other places in the composition, so, for example, you could render the same geometry with frontface-culling, then again with backface-culling, without the overhead of having to run the GLSL vertex shader twice.

What do you think?

a|x http://machinesdontcare.wordpress.com

cwright's picture
Re: Capturing Geometry For Multipass Rendering

the extensions required for this is "Transform Feedback" (XFB in Nvidia parlance).

This is the only way to capture GLSL-deformed meshes that I'm aware of. At one point, I was considering implementing the Kineme3D deformers using this. But then I realized something:

To render it, we'd have to upload the gemoetry (system ram to video ram), deform it, and download it (vram to system ram) -- the deform step would go much much faster, but the added cost of upload/download would make it worthless for most meshes people use (it takes several million vertices per model before the upload/download performance hit goes away, and at that point you're not doing realtime anyway).

The best way to prevent the upload/download stuff would be to always keep the meshes around as VBOs. I considered this, but then calculating normals is essentially impossible (without downloading the mesh, of course).

So, my current plan is: don't bother. This will be reevaluated once SL comes out, but until then, it's not as rosy as everyone hopes.

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

Ah, thanks for the explanation cwright. That makes sense.

Maybe it's better just to make custom QC patches for specific tasks where this kind of thing is needed.

I've been thinking maybe when OpenCL comes along, it would be possible to do some of the stuff that I currently do with Vertex Shaders using OpenCL, so that way I could create the mesh using OpenCL, with GPU-acceleration, then store the result in a VBO, and render it several times through different Fragment Shaders etc. You could maybe even use OpenCL to calculate normals.

Actually, do you have any plans to use OpenCL to speed-up mesh operations in Kineme3D in the future?

a|x

cwright's picture
Re: Capturing Geometry For Multipass Rendering

OpenCL radically changes what is and isn't practical, so it's difficult to say what's going to happen.

We've got some internal plans for Kineme3D once SnowLeopard arrives, but we're not disclosing the details just yet ;)

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

Fair enough :) I'm getting quite excited about the possibilities myself now, having previously dismissed it as not useful for anything I might be attempting.

a|x

cybero's picture
Re: Capturing Geometry For Multipass Rendering

Although the CUDA SDK from nVidia doesn't run everything on the Mac that it does on the PC it does allow for getting one's head around some of the key concepts that will no doubt be getting rolled out with 10.6.

The Open CL Technology Brief also makes for a good introduction to OPen CL.

By the way, it is also supported by ATI [part of AMD now?], isn't it?

cwright's picture
Re: Capturing Geometry For Multipass Rendering

OpenCL's "supported" by ATI, but I've not seen anything with their drivers (I've seen both nvidia and intel CL drivers -- nvidia's have been hit or miss depending on dev seed (latest is nice), and intel's were completely non-functional in almost every test (I've heard the wwdc seed finally has functional intel cl drivers though, or possibly just software fallback or something -- the performance wasn't good)).

[citation: http://ati.amd.com/technology/streamcomputing/opencl.html ]

As far as "big names" go, the only absent party in CL is Microsoft -- otherwise, everyone seems to be on board (even ARM, Ericsson, Sony, Texas Instruments, Nokia, IBM, Blizzard, and Electronic Arts were involved in the spec, among several others).

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

That's cool. I'm looking forward to the drivers coming online by the time Snow Leopard is released.

I'm intrigued to know how mesh-creation using OpenCL compares to mesh-displacement by GLSL vertex shader, in terms of speed.

Hopefully I'll be able to get my head around OpenCL's syntax, since it's apparently very 'C-like'.

a|x

cwright's picture
Re: Capturing Geometry For Multipass Rendering

OpenCL can use GL VBOs, so if it compiles to the GPU and operates in VRAM, it'll be indistinguishable from GLSL in terms of performance (with the slight possibility that it will go faster, since it doesn't have to worry about fixed-function state changes like GLSL does).

The syntax looks good to me - I'm excited to start digging in once I wrap up some other projects that are eating all my fun time lately...

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

Quote:
OpenCL can use GL VBOs, so if it compiles to the GPU and operates in VRAM, it'll be indistinguishable from GLSL in terms of performance (with the slight possibility that it will go faster, since it doesn't have to worry about fixed-function state changes like GLSL does).

So could you use it to calculate normals, with GPU-acceleration, with a VBO heightmap, for example?

Or not? First thing that popped into my head...

a|x

cwright's picture
Re: Capturing Geometry For Multipass Rendering

You could -- they'd be faceted normals (I'm not sure there's a simple, general-purpose, GPU-accessible smooth normal algorithm), but it's possible.

Also, I /think/ they're bringing geometry shaders to GLSL in SnowLeopard, and if that's the case, you can do them in GLSL as well, without dealing with opencl at all.

cybero's picture
Re: Capturing Geometry For Multipass Rendering

Quote:

the only absent party in CL is Microsoft

pretty much so, and there's going to be a lot of otherwise perfectly good non qualifying hardware of all sorts of kinds , Mac, Linux, Windows doesn't make a difference, Open CL really is hardware specific.

of course, although Microsoft express no especial interest, nonetheless, CUDA is available Open Source on Windows, although not much help with QC presentations for obvious platform dependent reasons, CUDA.NET will deal with cross platform, non QC , except in a signed Java applet, CUDA development, supported by Mono on OS X

psonice's picture
Re: Capturing Geometry For Multipass Rendering

Calling CUDA cross platform isn't quite right somehow.. it works on different OSes, but only on nvidia hardware. That pretty much limits you to 50% of your audience on any OS (much less if you take intel GPUs into account, but maybe for stuff like this it's better not to :)

cybero's picture
Re: Capturing Geometry For Multipass Rendering

Quote:

Calling CUDA cross platform isn't quite right somehow..it works on different OSes, but only on nvidia hardware.

currently true, psonice.

However given the following list of participants - see Khronos

including 3DLABS, Activision Blizzard, AMD, Apple, ARM, Broadcom, Codeplay, Electronic Arts, Ericsson, Freescale, Fujitsu, GE, Graphic Remedy, HI, IBM, Intel, Imagination Technologies, Los Alamos National Laboratory, Motorola, Movidia, Nokia, NVIDIA, Petapath, QNX, Qualcomm, RapidMind, Samsung, Seaweed, S3, ST Microelectronics, Takumi, Texas Instruments and Toshiba.

I doubt that is going to remain the case for much longer.

psonice's picture
Re: Capturing Geometry For Multipass Rendering

Khronos are behind OpenCL, not CUDA. They're somewhat similar things, but CUDA is made by NVidia and only works on NVidia hardware, OpenCL is cross-platform and supported on quite a lot of hardware (nvidia, ATI, imagination that make the iphone GPU are also supporting it so we could even see it there).

I suspect that OpenCL will totally replace CUDA at some point, as it's going to have much wider support. Microsoft aren't behind it, because they want their directx stuff to replace it (openGL was dropped in vista, but still gets supported by the driver writers i believe, and directx11 is supposed to have some kind of openCL equivalent).

cybero's picture
Re: Capturing Geometry For Multipass Rendering

Quote:

I suspect that OpenCL will totally replace CUDA at some point, as it's going to have much wider support.

Quite likely so, especially given the [growing] list of participants.

CUDA is pretty much nVidia's QED for the benefit of all who want to find out and know about Open CL [ CUDA style at least ]. Cross platform wise there is CUDA.Net, but as you point out, it will only work upon nVidia hardware.

Regarding the other hardware that might , or will support Open CL, as I pointed out earlier, there'll be a lot of otherwise perfectly good bits of kit, Open GL capable, but not new enough to currently have their GPUs and DSPs used for Open CL programming.

That's often as not the case with cutting edge technology though isn't it?

I await Apple's Open CL in Snow Leopard with much interest to see just how the land lies on the Mac platform Open CL wise.

Incidentally, I am given to understand that Snow Leopard might not be PPC compatible, however as 10.5.7 is slated for summer release followed , possibly in September / October by Snow Leopard, all such rumour mill notions might be open to change.

cwright's picture
Re: Capturing Geometry For Multipass Rendering

cybero wrote:
Regarding the other hardware that might , or will support Open CL, as I pointed out earlier, there'll be a lot of otherwise perfectly good bits of kit, Open GL capable, but not new enough to currently have their GPUs and DSPs used for Open CL programming.

Coming from "Theoretically, this is possible"-land, the nice thing about CL (I can't speak for CUDA) is that, with the diversity of supported devices (cpu, gpu, "accelerator cards", etc), it wouldn't be too difficult to take a reference implementation (software/cpu-only), and add some simple "well, this lame GPU/DSP/whatever can handle this simple CL kernel" logic/targeting, and have per-kernel acceleration where possible. The tricky part with this is finding out who's going to write the driver... (vendors typically don't like revisiting old stuff)

cybero wrote:
Incidentally, I am given to understand that Snow Leopard might not be PPC compatible, however as 10.5.7 is slated for summer release followed , possibly in September / October by Snow Leopard, all such rumour mill notions might be open to change.

10.5.7 was released a week or two ago -- you probably meant 10.5.8.

cybero's picture
Re: Capturing Geometry For Multipass Rendering

Yep 10.5.7 - current - did indeed really mean 10.5.8 - soon to be released.

psonice's picture
Re: Capturing Geometry For Multipass Rendering

I'm really looking forwards to openCL. The paint/3d tool I've been working on could be done much, much better with CL than standard cpu code + glsl. If I get time to work on it again some time soon, too busy with iphone stuff lately. Plus I think my ati 2600 card is not supported for openCL.. bugger!

I fully agree on the disappointing lack of support for older cards then :) Well, i guess the latest cards will benefit most, and older cards mostly lack the features or horsepower to make it really useful. There are a few cards (like mine!) that could and should run it but aren't listed, but I guess they only have enough resources to target a certain number of cards.

I'm pretty sure that SL is intel only btw. Another reason to upgrade my old powerbook!

usefuldesign.au's picture
Re: Capturing Geometry For Multipass Rendering

SL is not supporting PPC, that's been said for some time. Time to upgrade for me and a few others on this thread ;).

Can somebody post a link to OpenCL compatible GPUs. I'm not sure to buy a new powerbook or go second hand (not so crazy about glossy screens for one but maybe one gets used to dodging reflections I do lots of photo/DTP stuff). If there's no OpenCL support on older MacBookPros then that's a considerable loss of value in my book.

cwright's picture
Re: Capturing Geometry For Multipass Rendering

Basically, any Intel-based Mac will support OpenCL. if you want decent GPU support of CL as well, you'll want as new an Nvidia GPU as you can get. (I'm pretty sure my MacBook Pro4,1 (pre-unibody) supports CL, though I've not freed up the disk space to install it an see for certain).

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

I'm getting used to the idea that my 1st-gen MacBook Pro is getting a bit long-in-the-tooth now, and the X1600 is probably not going to be well-supported in the future, but I'll be annoyed it it won't at some point support OpenCL (and Geometry Shaders, for that matter) sometime in the near future.

Incidentally, I like how each post in this very long thread is getting narrower and narrower- soon it'll be one word per line...

a|x

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

The list on this http://www.maclife.com/article/news/opencl_supported_graphics_cards page suggests my venerable X1600 won't support OpenCL in Snow Leopard, but I guess that's not necessarily the final word on the matter, and ATI may at some point throw together a driver so it does support it.

On the plus side, looks like the GeForce 8800GT on my Mac Pro WILL be supported. The annoying thing is, it's the laptop that could benefit more from general-porpoise computation on the GPU, I think.

Fingers crossed, as they say..

a|x

psonice's picture
Re: Capturing Geometry For Multipass Rendering

i think that's the list from the apple site. So yeah, basically nvidia 8600 and newer (which includes the last-gen macbook pros), or ati 4850 or newer.

It's very annoying that the ati 2600 cards aren't supported, as they were the standard cards in the imacs for a long time, and the base card in the mac pros too if I remember right. They're also pretty much equivalent to the 8600 cards in terms of power and capability. Hopefully they'll add support later. But knowing apple, it will be either 12 months later or never :D

usefuldesign.au's picture
Re: Capturing Geometry For Multipass Rendering

THX tb.

Now I can comb ebay before I blow my $ on a Brand new MacBookPro.

usefuldesign.au's picture
Re: Capturing Geometry For Multipass Rendering

"But knowing apple, it will be either 12 months later or never :D"

Case of investing in the user base or investing in more potential sales. One has a material benefit, other is more subtle :)

leegrosbauer's picture
Re: Capturing Geometry For Multipass Rendering

At my house: ATI Radeon HD 2600. iMac. Eighteen months old. Twenty-three hundred dollars canadian.

I'll settle for make it work and do it today, Jack.

dust's picture
Re: Capturing Geometry For Multipass Rendering

glad i have one of the new geforce cards on my mbp. im hoping SL will let me put 8gb of ram in it but i got a 2.8ghz so i guess 6gb is all for now. i guess by the time i pay it off, who knows the gpu could be at 2.8 ghz, but hey thats the game we all play. its called planned obselesance, not sure how to spell that but its word, i borrowed it from the automotive industry.

cwright's picture
Re: Capturing Geometry For Multipass Rendering

dust wrote:
im hoping SL will let me put 8gb of ram in it but i got a 2.8ghz so i guess 6gb is all for now.

That's a hardware limit, not a software one. Same for my late 2006 MacBook -- the mainboard chipset will only allow 3GB of ram to be addressed (I've got 4GB installed, for pairing reasons).

You can "install" 8GB (2 4GB SO-DIMMs), but the motherboard itself will only ever be able to address 6GB of it (due to design limitations).

dust's picture
Re: Capturing Geometry For Multipass Rendering

hopefully some firmware update will enable us to put more ram into our laptops,

so im starting on my quest to figure out glsl. i have the kineme gl tools. i would like to use them. now is there a way to make some javascript struct that the super glsl grid works with like in your example. i figure you put in a bunch of time on the gl tools so i should try them before i make my own plug in.

basically i just want to render out a model like the teapot, which i have code for. this is my generated header from cheetah.

define Relief_vertexcount 10201

define Relief_polygoncount 20000

float Relief_vertex[Relief_vertexcount][8]={ {0.00000, 0.00000, -0.02885, 0.99917, -0.02885, -5.00000, 0.00000, -5.00000}...........

int Relief_index[Relief_polygoncount][3]={ {0, 1, 2}, {2, 3, 0}, {1, 4, 5}, {5, 2, 1}, {4, 6, 7}..........

i know this sounds daft but i would like to not have to build a plug in for this. although i am messing with apples version of the rutt etra plug in. i like your gl tools but you use just a sphere, just trying to make my own model that is a tad bit more complex. i don't really need any of that fancy enumerated blending and stuff just want to render out the two arrays above ??? thanks

dust's picture
Re: Capturing Geometry For Multipass Rendering

oops double post... what gl patch can i render into, or make VB0...

well, i looked into making my own mesh VBO and that seems a bit complicated... i found a perl/c++ script that can convert a cheetah file to a open GL vertex Array format so that is cool.

it seems kineme 3d is VBO based i suppose it would save me a lot of headache just to export a fbx ?

was just trying to learn the glsl and wanted to use my own model to start with maybe i will mess with SL mesh for a bit might as well get ready for the future.

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

dust wrote:
i will mess with SL mesh for a bit might as well get ready for the future.

NDA? ;)

a|x

dust's picture
Re: Capturing Geometry For Multipass Rendering

sorry a|x i hope i didn't disclose to much info there. i'd rather it be me than you. i suppose it would cause some internal problems if it where you. just so there is no confusion its ok to talk about CL just not in regards to QC but to Nvidia ? damn it i think i did it again. sorry mate.

its funny i kind of went off topic on the qc.list and someone made a reference to to me saying that it was nice to know there are real people on with real problems attached to the list. and i said i am a robot machine beep beep. and that there are real people on here except there is a bot moderator that limits your file size. the next day i get 4 emails from different moderators so its nice to know there are watch dogs out there and not bots, but yeah please don't turn me in ;)

i got a demo i want to show you when its done. i know chris wood and you are all about the demo scene which frankly i never herd of. ive seen some of those 3d or anagram demos made by kids in the game industry but at first chris said 16bit demo and i was thinking well you could fit most of ipv4 in 16bit and thinking of some network app but it seems to me its all about the art.

is lets say QC legal in comps that is supposing you built the plugins etc... or how does all that stuff work ? where do you draw the line in the realtime events ?

sorry if i sound daft im a noob, i love your work. your clips and what not are very very useful. your volumetric stuff looks really cool, you have inspired me to mess around with MRI images. i have tried cue stacking them and then iterating etc.. eventually i will get to the volumetric side.

my grandfather was a radiologist for many years that eventually went to the mri side of things before his research so i grew around those radio active things. my uncle used to work for him during the mri years and i always thought cause he was a software programmer that he helped make the mri software but i guess it turns out he just made the accounting software and the manuals. he is a wizard now don't ask he goes to wizard camp and got all spiritual after his divorce.

my mind is wide open right now, its like with the glsl there is raytracing and fur now ??? im slowly figuring ci and glsl out its close to C so that is a ++

cheers.

psonice's picture
Re: Capturing Geometry For Multipass Rendering

Let's see how narrow we can make this thread!

Funny, I never figured out before you're dustin of the list :D

OpenCL / snow leopard: opencl was made mostly by apple it seems, not nvidia. Nvidia made CUDA, which is similar but proprietary, but they'll support openCL too. There's no issue really with discussing openCL or SL, unless you've signed an NDA, which I haven't, you presumably haven't, but cwright has. The things to be wary of: if you didn't sign that NDA, how did you get SL? Presumably from a torrent site or wherever, which is worse than breaking the NDA ;D (No worries if you did, I'm sure most of us do it at times :) The other thing is that Kineme won't want to make apple angry, and openly discussing SL stuff will probably make mr. wright uncomfortable, so it would be polite not to do it too much.

Demoscene: Well, it's about the 'art', yeah, but also about showing your skills (demo = demonstrating your skills, a way to show off in the olden days). So a really impressive artistic thing will get some respect, but something fugly but technically great will get respect too. The best groups combine really good code with good art.

Btw, by '16bit' demos, i meant demos with style from the old days when computers were 16bit (like amiga, atari, old 68000 macs, or 286 PCs). 3d and stuff wasn't possible much then, so people made scrolling text in really cool ways and played chip music because mp3 hadn't been invented yet :)

With QC demos, it depends a lot on the party you release it at. Not all parties support mac, and many will only accept a compiled application. The other thing is that using a tool like QC is considered a bit 'lame', because other people will have made their own QC like tools from scratch (look up werkzeug some time - very QC like. If you're open about it, and explain that you're a beginner and want to produce stuff and learn, people are generally happy enough about it though.

If you want to do demo, I would suggest finding out where your nearest demo party is. There's a few in the states (if i'm right in remembering you're from there somewhere) like block party + pilgrimage. Aim to make something for that. Try and find a musician to do the music unless you want to do it yourself, and maybe a graphics guy to do 2d/3d and design.. it's not really necessary, but the demoscene is pretty social and people like to work in teams. It gives you a ton of inspiration too :)

Small parties are probably better - the big ones are assembly (august in helsinki) and breakpoint (easter in germany, but that's passed).

What's normally acceptable for a demo competition: you submit a zip, containing an executable + text file (if you want). There's usually a size limit for the zip, so make sure you're under that (it's normally pretty big, 30mb or more perhaps, unless you go for the small size competitions like 64kb). There's a time limit too, usually 8 minutes or sometimes more. Your demo should play to the end and then exit back to the desktop within that time (or they'll quit it before it finishes :) but it's better to do short, quality demos than long, boring ones. It shouldn't be interactive (if it is, nobody will touch the keyboard/mouse during the compo.)

For a much better idea, head to http://www.pouet.net and look at the demos there. You'll be able to run most of the mac demos, but the PC demos are generally much better so bootcamp + windows XP is a good idea. Or just watch videos, they're not as good but you'll get the idea. The forum there is good, but totally unmoderated so it gets a little.. wild. Fun, but very wild, and sometimes not very safe for work ;) It's a good place to ask about parties, and find out more about what you need to do, but don't set your expectations too high.. sometimes you get a great answer and a good discussion, sometimes you get a huge war started and people posting all kinds of crazy shit :D

toneburst's picture
Re: Capturing Geometry For Multipass Rendering

cool, dust,

keep it up. Just for the record, I'm not really involved in the Demo scene, though I do occasionally borrow techniques from those who are, like Chris.

I'd love to be able to do that kind of stuff, but realistically, without a talent for maths and (crucially), without having started on 3D graphics about the same time I started on solid foods, I'm never going to get anywhere near what those guys are producing.

Incidentally, talking of volumetric stuff, did you see http://vimeo.com/2848308 ?

a|x