I've started to write my own raymarching renderer in GLSL + QC, thought I'd share my output (and some of the code + comps) here.
This is roughly where I'm up to so far:
Moderately complex geometry, lighting + soft shadows. The shadows were hard, I wanted decent quality soft shadows from a point light without too much of a speed hit.
The camera can be moved + rotated in any direction, the light can be moved freely. Adding additional lights with or without shadows is trivial :) Ambient occlusion (true AO, not screenspace) is also trivial, but it seems to look better without it. Animating the object is also trivial.
No .qtz yet, until I've worked out some of the bugs and optimised it a bit. As it stands, the camera can't move outside of a limited space, and repeating objects glitch badly if they're not tightly managed.
Quick rundown of what raymarching is:
You render a single polygon, with a GLSL shader (openCL would also work, but without major benefits and with major compatibility issues ;)
Inside the shader, you determine camera position and direction for each pixel, and you do a form of raytracing in that direction (this is the 'raymarching' step).
To raymarch, you calculate the distance from the current position to the nearest surface. Then, you can 'march forwards' along the ray by that distance, and repeat, until you hit an object.
You then calculate the surface angle at that point where the ray hits, so you can do lighting and the rest.
It's also referred to as 'distance fields' because everything is based on the distance from a surface, or 'sphere tracing' because you effectively calculate the intersection of a sphere and the geometry.
Benefits of this kind of rendering:
It's "fast" (well, for raytracing!)
The scene can be infinitely big and complex. Trivial example: an infinite 3D grid of cubes. This is very quick and easy with raymarching. How many can you do with an iterator? ;)
Advanced lighting effects, reflections, refraction, AO, subsurface scattering are all fairly simple. You can use them all at the same time, with a bit of a performance hit.
The downside:
It's hard :( Just moving the camera becomes a brain twister.
It's slow, compared to normal rendering. 640x480 at 30fps needs a high end GPU.
You can't use 3d models and the like. Everything gets built with mathematical formulas.
I'll post the .qtz up at some point (still improving it + fixing bugs, it's quite a mess..)
But for the shadows, I'm basically tracing a path from the surface point back to the light, and calculating the shadow based on the distance to the nearest object from the ray (with scaling based on distance from the point, so close objects cast a sharp shadow, distant ones are soft).
You render a single polygon, with a GLSL shader (openCL would also work, but without major benefits and with major compatibility issues ;)
Horses for courses isn't it ? :-)
OpenCL will render a polygon, you can put that through to a Mesh Creator and provided it is a Line, Triangle or Point sprite it will cary a texture, but not as well as GLSL can; GLSL will render a Polygon with whatever colour code and texture input you provide.
Yep. There's plenty of good uses for openCL, this just isn't one of them. There are no polygons in this kind of rendering, no meshes at all. So CL wouldn't really gain anything, perhaps a few potential optimisations but nothing major.
And this nice new imac I'm on has a pretty fast GPU, but no openCL support. CL would therefore be about a hundred times slower or so, which was my point really ;)
One advantage of OpenCL in this scenario would be to render to more than one image at a time. For example, you could render a depth-map of the scene at the same time as the lit version, and use the two images to add, for example, a depth of field effect as a post-process. To do that in GLSL, you'd have to render the whole thing twice.
The main output of the raymarch function is the distance. It's pretty trivial to make the output of the shader contain z, intensity (i.e. light map), perhaps material. You can nest that with another shader to apply DOF, material colour/texture and so on.
It has to be 2 passes either way, unless it's possible to do DOF inside the raymarcher. Possibly it is, I'll have to think about it.. you do get information about what objects are close to the ray as you march along it, so in theory you have the info you need. In practice, perhaps it's not possible.
If you were doing raycasting, you could maybe jitter the ray position with a noise function, and vary the jitter amount as a function of distance from a focus plane. I'm not sure that would work for raymarching though, and might just look bad, anyway.
I Just wondered as the latest iMacs I've tried some OpenCL kernel based Quartz Builder toys upon, they worked the toys like a charm, all ATI Radeon, wonderful OpenGL performance.
I guess the OpenCL was working for me upon the CPU, unless I've skipped a beat on the driver / framework support for OpenCL on the new iMacs?
OpenGL can 100% render more than one image at a time, this feature is not exposed in QC on a patch level, but the next Rutt Etra revision uses MRT and multiple attachments on FBOs to one shot render textures for vertices, normals and texture coords all from one render pass.
Just saying, its not a limitation of OpenGL, just of QCs user interface for it.
You mean jitter the position along the path of the ray? It wouldn't work for this, because you'd hit an object too early - it would result in wobbly objects :) Cool idea for raycasting though.
I was discussing DOF with this with another demoscener earlier. Actually, "in theory" it's ideal for DOF effects, because as you step along the ray you know the distance from nearby objects all the time. That's exactly what you need for true DOF, and the information comes absolutely free!
The problem is, you only know the distance. Not the direction, not what object it is. And this happens before you do lighting, so you don't know the colour or brightness of the nearby object.
So far, the conclusion is that it's best to add it in post-process. It's probably best to handle this as a 2-pass render anyway. I'll try to add that next, see if I can get object colour, edge anti-alias, maybe DOF working. I have a suitable DOF shader already, but the performance hit might be bad - this isn't exactly light on the GPU ;)
Speaking of object colour - also hard :( With the standard method, you know when you hit a surface, but not which one.
It's a recent imac, with radeon 4670 (i think). CPU support only. I think George made a handy patch that displays the openCL info, probably it was on the mailing list.
My other mac has a radeon 2600, too old for support :( And a laptop with GMA950. Don't ask :)
Or on second thoughts, maybe raymarching isn't so great. I've been optimising, to get the speed up, and one idea I had was that raytracing some objects would be faster than raymarching. It's easy to combine the two, so you raytrace some objects and raymarch others.
Anyway, long story cut short: it's MASSIVELY faster to raytrace. Comparing the same scene, identical parameters + resolution etc, I got 50+ fps with raytracing, 1.7fps with raymarching.
I thought raytracing was supposed to be slow?! I'm getting 40-50fps, at 720p, on a radeon 2600. On the downside, the maths gets a lot more complicated :(
sweet i was wondering how you would do a ray cast in qc. ray dream studio was my first 3d app, so being able to do a ray-trace in realtime is awesome. one of these days i'm going to study your massive clod.
Ok, here's a quick example. This is pure raymarching, no raytracing. It's a simple plane (the ground), a cube, and a distorted sphere (which intersects the cube) with soft shadows.
I'm on a slow box here (this needs LOTS of GPU power) so the resolution is set low. I've also turned the shadow quality down for more speed (it's the last parameter in the softShadow(...) function).
MaxIters, shadow iterations, and threshold all affect speed + quality.
The position + rotation of the 'mesh' (actually a single quad, but if I use a normal glsl mesh QC "optimises" it out of existence if it move offscreen thanks George for the fix!) controls the camera. Move the mesh to move the camera around. Strangest camera ever! :)
Inside the f(..) function, you can add more objects, using min(..) or max().
Man, how can we get some better fps out of this (besides obliterating functions)?! This is really similar to a cl fractal kernel I put together awhile back, that gave some ok fps... I feel like jumping in and converting, but I know I'll be messing with it all day :) Thanks for the sample.
There's lots of scope to optimise. I've focused on making it work more than speed so far, there's lots that can be done to speed it up. E.g. calculating the normal can be 3x faster - this is pretty major.
The other thing is mixed raytrace/raymarch rendering. Replacing some of the geometry with a raytrace function instead of raymarching can make it MUCH faster. Pure raytracing is even better and you can still raymarch the shadows.
Interesting bit of OpenCL kernel work and a great piece of GLSL work.
I must admit as I perused the composition in between preparing a stir fry I wasn't at first captious of the GLSL shader providing both the sphere and grid GLSL x,y,z,w so started with seeking to replace the OpenCL grid you used with one of my own. [to be fair to myself it wasn't entirely obvious to me from a mere glance that a sphere was intersecting the cube].
I did eventually twig to the fact that no sphere kernel as such existed. LOL.
Just beginning to make fuller sense of this GLSL / OpenCL combo - makes for a great study, like figuring out the way increased iterations results in the rendered grid shifting further back on the stage for instance.
Thanks for sharing, psonice :-)
Your posted composition has given me a fresh insight into how an OpenCL kernel Grid produced Mesh can and will operate. [e.g doesn't wireframe very nicely •~].
On the other hand, it does iterate pretty well producing the faux sphere and cube.
Umm... ignore the openCL stuff :D It does almost nothing (or should do, if it does something it's wasting it's time.. actually I neglected to check as I was rushing at the time..)
You can replace the openCL mesh + filter + mesh renderer with a straight billboard and it'll still work. The reason for the openCL is that QC optimises a billboard/glsl mesh out of existence if it's off-screen, and it does go off-screen.
What the mesh/billboard/whatever is inside the glsl patch does: it's the camera. Moving or rotating the mesh moves the camera around. Scaling changes zoom or perspective. The GLSL patch then resets the vertex positions to fill the screen so you don't see it move.
Also, the iteration counts: it's nothing like a QC iterator. It doesn't iterate the mesh or anything like that.. there's no mesh to iterate. The iteration count is the number of stages in the rendering, more iterations just gives higher accuracy. Low iteration counts means it's very inaccurate.
Best way to think of this: it's software rendering. More or less raytracing, the method is slightly different but it's the same thing really. Except the software is written in GLSL, running on the GPU. That's why it only needs 1 polygon to draw on.
Oh, and the sphere: it's a distorted, rippling sphere, making a hole in the cube. It doesn't look like a sphere :)
It's very different - you can do a complete animated world, with only glsl shader, billboard, and patch time (to feed the animation). Camera, lighting, models, materials can all be done in pure glsl.
But QC is still ideal for this kind of stuff, because trackball and 3d transform are great for moving the camera, and LFOs and timelines and the like are perfect for animating. Plus it's great to just render lighting and stuff in the shader, then do a ton of post-process in regular QC :)
Oops! I noticed yesterday that I made a slight error in the code on that last comp. It's supposed to raymarch the scene, then calculate the normal for lighting. Instead, it's raymarching the scene, then raymarching the scene 6 more times to calculate the normal. I.e. it's probably 5-6x slower than it should be :D
I'll post a new one up at lunch time, with that fixed, a raytraced floor (raymarching the floor speeds it up a lot, and improves accuracy too.. not a bad improvement!) and more interesting geometry. In fact I'll add in domain repetition, infinite objects definitely help show how powerful this technique is ;)
OK, here we go. Unlimited cubes, soft shadows, moving camera + light, and a light that cuts into the geometry within a certain distance.
And HUGELY optimised compared to the last one - it runs much faster at 4x higher res. It's a mix of marching and tracing for speed, with a few other hacks.
It's in a render-in-image patch. It doesn't need to be, it's just handy to turn the resolution down now and then when working on something heavy. Cut + paste it out if you want to change resolution.
I'd recommend opening the 2nd example patch below though, it'll go a LOT faster ;)
It's in a render-in-image patch. It doesn't need to be, it's just handy to turn the resolution down now and then when working on something heavy. Cut + paste it out if you want to change resolution.
I'd recommend opening the 2nd example patch below though, it'll go a LOT faster ;)
i like this first one. looks sort of melting like but yes the fps is way better on the second one. i should study that one. just so i understand your zooming into a cube and using the sphere to cut through it. the ray march is to determine where the point is in relation to the camera so you can cut ?
is the f function the function that does the sort of boolean subtracting or cut i'm calling it ? sorry for so many questions i really want to understand this.
ok i see a bot differently now. your mesh render is where the motion is coming from. so the sphere is more in the middle of the cube. i think it would be cool to place the sphere where the camera is so no matter what kind of track ball you do the cubes get cut, or would an operation like that be to heavy duty ?
Yeah, that's kind of right. The actual scene consists of a ground plane, a fixed cube, and a sphere that punches a hole in the cube. The sphere is rippling, with lots of sin waves, that's why it has that 'melty' look.
The way it works is a bit weird, but roughly: For each pixel on screen the shader calculates the camera position and the direction of the ray for that pixel. It gets that from the position + rotation of the mesh.
Then, it marches along the ray until it hits a surface. It's the way it marches that's unusual. You calculate the distance to the nearest surface for the start point in any direction (this is why it's called a 'distance field'). This is the 'safe distance' you can march along the ray without hitting a surface. Then you move along the ray by that distance, and repeat, until it hits a surface (or comes very close, which is faster than an actual collision).
So, basically, you calculate the ray start point + direction, and feed that to the rm(..) function - this is the ray marcher. That steps along the ray, and each time it uses the distance function (which is f(...)) to find the smallest distance.
The f(..) function has to know the smallest distance to a surface, so normally it's like this:
obj1(...) is a function that gets the distance to that kind of object, so you see i have functions for distance to sphere, cube, plane. It returns the minimum distance out of all objects in the scene.
You change that min(..) function if you want to make objects intersect, eg.
return max(cube(), -sphere());
will subtract a sphere from a cube.
All of this gives you a depth map basically, you just know the distance to the object. You can calculate the normal by testing the distance at points around the place where the ray hits, then do lighting based on that.
The shadows are done by marching from the point where the ray hits, back to the light. Plus some magic to make them soft.
The intersecting sphere is tied to the light position. It's trivial to change it to the camera position - just attach the LFOs with 120s periods to the sphere's x,z position (they're attached to the light's mix of 2 LFOs).
Or stop the camera moving, and do whatever you want with it :) There's no performance impact from moving the sphere.
this is awesome. i bet with some textures it looks really nice. with a system like this do you have to make your objects parametrically ? i have yet to be able to pull of this cl to gl interoperation i think its called to pull in verts to gl from cl ? will this sort of thing work with glsl ?
ok i tried that doesn't seem to bother anything. i will try and add the wobbly noise to the sphere from the first example as i like how it is melty looking.
Generally, yeah, you have to make parametric objects. Texturing is difficult too, you either generate textures in code, or you have to figure out the texture coordinates based on the place the ray hits the object. Not straightforward..
Pulling vertices in is theoretically possible, but you'd have to pass them in as uniform variables I think, meaning it's very limited. The way around that is to render your object the normal way (separately from this render), but inside a glsl patch that writes normal + depth position out to the rgba channels. You could then mix it into your raymarched scene with a straight depth test. You'd have to be very careful to match camera angle, perspective etc. exactly though, not easy.
chris you sent me an example one day called massive clod. i still have it. this had textures and fog, do you think that would be a good place to start if i wanted to add textures.
i mean the ray is returning distance. so would i have to calculate the uv grid from n ?
melty cubes looks nice but it messes up the lighting sort of washes everything out but im sure i can mess with the lighting a bit to make it look proper.
Odd, it shouldn't do. I'll take a look, find out why.
You can comment out the shadow bit in main() btw, which will speed it up. It'll still be lit. The line is something like "intensity *= ss(...);" (ss = soft shadow).
i was just using the code you all ready wrote for noisy cube that you commented out. then return max as you had it so maybe there was some issue with the code you commented out or i would need to return it differently i will try and mess with the shadow.
n is the normal, so you can't use that for the texture coordinate. You could use it for environment mapping though.
I didn't write 'you massive clod', it was parapete (it was for linux, I just ported it to qc). From what I can guess though, the fog is just something like "lighting += distance" and the texture is just based on the coords of the surface (i.e. it's not textured, it's just a surface pattern programmed in).
To calculate the texture coords, you need to determine the position on the surface of the object. Easy example, the ground plane. Get the x,z coordinates at the point where the ray hits the ground. Those are your texture coordinates. If you're hitting a cube, it's harder.
Then you need to change the distance function f() to return a vec3 (distance, texCoords.xy). Instead of using "min(obj1, obj2), you have to do stuff like "return obj1.x < obj1.y ? obj1 : obj2" and it gets ugly with more objects ;)
Urgh. I just tried this, and got it working perfectly. Then I hit save, and QC crashed :( (Probably the CL patch? I need to experiment, to find out what causes it, but it's highly unstable lately and it wasn't before).
What you need to do: Instead of "cube(cube1, pos + vec3(sin....));" use "cube(cube1 + vec3(sin... ), pos)".
i got it working it was the cube.z messing with things. looks fine with noisy cube.x and cube.y; i tried adding some parameters for the soft shadow's float r, float f and float l inputs but for what ever reason. going uniform ssr; for float r input parameters doesn't produce me an input ? very weird.
Now in colour. The walls are acheived by using mix(obj1, obj2) instead of min/max. Some pretty cool effects are possible with mix()!
Colour is tricky. I have to get both the distance to each object, and it's colour, then get the minimum distance + colour of that in the distance function. This means all those nice min/max/mix functions don't work, because you lose the colour.
Yep, only catch really as that it's very hard to predict the outcome at times. The mix value is never what I expect it to be, I have to experiment all the time to get it right :/ The buildings in the video below are made with mix - I'm making 5 boxes, and using a combination of min() and mix() so that the boxes get combined, but sometimes blended to form more complex shapes.
At 720p, with AA, and the quality settings cranked, a few minutes. But it's realtime in general, running about 15fps at the moment at 640x480. It's quite unoptimised too, it would probably run at 30+ once it's tidied up. This is on a radeon 4670, running about a hundred other apps and another heavy QC app running.
Nice contrast of the grey volumes and the coloured lighting. How is the lighting coloured? Is each 'ray' actually evaluated for a colour value or is the ground cycling some coloured texture?
(haven't downloaded any of your code, too busy to get distracted by such mastery ;-) )
It's just a left-over bit of colour from when I worked out how to do materials.. the ground is simply coloured using sin(position.xz) or something like that. I'm going to use the colour channels for material parameters (so r+g are texture coords, b is material index to select a texture) and alpha to store intensity. This way I can put all the texturing in a second shader.
I'm not sure mix() is a linear function. It may be an 'equal power' curve, which might make the results different from what you'd expect from a simple linear interpolation. Not sure on that one, though.
I'm simply subtracting a repeated cube for 'windows', and adding a thin block to the top of each block for the 'roof' (otherwise you get windows in the roof, which looks... odd ;).
Lighting here is simple AO plus a directional light without shadows. Just wondered how it would look. Actually, not bad, except that the ground messes up because of the 'repeated' geometry. You can see the squares where it gets repeated, the shadow on the ground is only aware of what's inside its square, because the normal points straight upwards. Maybe I need to jitter the normal or something.
shadows are really nice ! all from a single GLSL patch ?
Yep, 1 polygon, 1 shader. It's basically software rendering, except the software runs on the GPU.
great, would love to have more detailed info on how you did these shadows. A link maybe ?
Anyway, great stuff, bravo !
I'll post the .qtz up at some point (still improving it + fixing bugs, it's quite a mess..)
But for the shadows, I'm basically tracing a path from the surface point back to the light, and calculating the shadow based on the distance to the nearest object from the ray (with scaling based on distance from the point, so close objects cast a sharp shadow, distant ones are soft).
Looks pretty smooth thus far.
Horses for courses isn't it ? :-)
OpenCL will render a polygon, you can put that through to a Mesh Creator and provided it is a Line, Triangle or Point sprite it will cary a texture, but not as well as GLSL can; GLSL will render a Polygon with whatever colour code and texture input you provide.
Yep. There's plenty of good uses for openCL, this just isn't one of them. There are no polygons in this kind of rendering, no meshes at all. So CL wouldn't really gain anything, perhaps a few potential optimisations but nothing major.
And this nice new imac I'm on has a pretty fast GPU, but no openCL support. CL would therefore be about a hundred times slower or so, which was my point really ;)
One advantage of OpenCL in this scenario would be to render to more than one image at a time. For example, you could render a depth-map of the scene at the same time as the lit version, and use the two images to add, for example, a depth of field effect as a post-process. To do that in GLSL, you'd have to render the whole thing twice.
a|x
The main output of the raymarch function is the distance. It's pretty trivial to make the output of the shader contain z, intensity (i.e. light map), perhaps material. You can nest that with another shader to apply DOF, material colour/texture and so on.
It has to be 2 passes either way, unless it's possible to do DOF inside the raymarcher. Possibly it is, I'll have to think about it.. you do get information about what objects are close to the ray as you march along it, so in theory you have the info you need. In practice, perhaps it's not possible.
GPU Gems?:)
If you were doing raycasting, you could maybe jitter the ray position with a noise function, and vary the jitter amount as a function of distance from a focus plane. I'm not sure that would work for raymarching though, and might just look bad, anyway.
a|x
Cheeky! ;) Entirely my own code:
// pos = point on surface, l = light position, r = shadow softness factor, f = shadow strength, i = step count float softShadow(vec3 pos, vec3 l, float r, float f, float i) { float d; vec3 p; float o = 1.0, maxI = i, or = r, len; for (; i>1.; i--) { len = (i - 1.) / maxI; p = pos + (l * len); r = or * len; d=ƒ(p); o -= d < r ? (r - d)/(r * f) : 0.; if(o < 0.) break; } return o; }What GPU does it have?
An unsupported ATI , Nvidia or an Intel GPU?
I Just wondered as the latest iMacs I've tried some OpenCL kernel based Quartz Builder toys upon, they worked the toys like a charm, all ATI Radeon, wonderful OpenGL performance.
I guess the OpenCL was working for me upon the CPU, unless I've skipped a beat on the driver / framework support for OpenCL on the new iMacs?
OpenGL can 100% render more than one image at a time, this feature is not exposed in QC on a patch level, but the next Rutt Etra revision uses MRT and multiple attachments on FBOs to one shot render textures for vertices, normals and texture coords all from one render pass.
Just saying, its not a limitation of OpenGL, just of QCs user interface for it.
+1
You mean jitter the position along the path of the ray? It wouldn't work for this, because you'd hit an object too early - it would result in wobbly objects :) Cool idea for raycasting though.
I was discussing DOF with this with another demoscener earlier. Actually, "in theory" it's ideal for DOF effects, because as you step along the ray you know the distance from nearby objects all the time. That's exactly what you need for true DOF, and the information comes absolutely free!
The problem is, you only know the distance. Not the direction, not what object it is. And this happens before you do lighting, so you don't know the colour or brightness of the nearby object.
So far, the conclusion is that it's best to add it in post-process. It's probably best to handle this as a 2-pass render anyway. I'll try to add that next, see if I can get object colour, edge anti-alias, maybe DOF working. I have a suitable DOF shader already, but the performance hit might be bad - this isn't exactly light on the GPU ;)
Speaking of object colour - also hard :( With the standard method, you know when you hit a surface, but not which one.
It's a recent imac, with radeon 4670 (i think). CPU support only. I think George made a handy patch that displays the openCL info, probably it was on the mailing list.
My other mac has a radeon 2600, too old for support :( And a laptop with GMA950. Don't ask :)
Or on second thoughts, maybe raymarching isn't so great. I've been optimising, to get the speed up, and one idea I had was that raytracing some objects would be faster than raymarching. It's easy to combine the two, so you raytrace some objects and raymarch others.
Anyway, long story cut short: it's MASSIVELY faster to raytrace. Comparing the same scene, identical parameters + resolution etc, I got 50+ fps with raytracing, 1.7fps with raymarching.
I thought raytracing was supposed to be slow?! I'm getting 40-50fps, at 720p, on a radeon 2600. On the downside, the maths gets a lot more complicated :(
http://www.interealtime.com/misc/demo/raymarch3.jpg
sweet i was wondering how you would do a ray cast in qc. ray dream studio was my first 3d app, so being able to do a ray-trace in realtime is awesome. one of these days i'm going to study your massive clod.
thx for the info !
Ok, here's a quick example. This is pure raymarching, no raytracing. It's a simple plane (the ground), a cube, and a distorted sphere (which intersects the cube) with soft shadows.
I'm on a slow box here (this needs LOTS of GPU power) so the resolution is set low. I've also turned the shadow quality down for more speed (it's the last parameter in the softShadow(...) function).
MaxIters, shadow iterations, and threshold all affect speed + quality.
(and I guess I should have mentioned this...)
The position + rotation of the 'mesh' (actually a single quad, but if I use a normal glsl mesh QC "optimises" it out of existence if it move offscreen thanks George for the fix!) controls the camera. Move the mesh to move the camera around. Strangest camera ever! :)
Inside the f(..) function, you can add more objects, using min(..) or max().
That's cool.
Man, how can we get some better fps out of this (besides obliterating functions)?! This is really similar to a cl fractal kernel I put together awhile back, that gave some ok fps... I feel like jumping in and converting, but I know I'll be messing with it all day :) Thanks for the sample.
There's lots of scope to optimise. I've focused on making it work more than speed so far, there's lots that can be done to speed it up. E.g. calculating the normal can be 3x faster - this is pretty major.
The other thing is mixed raytrace/raymarch rendering. Replacing some of the geometry with a raytrace function instead of raymarching can make it MUCH faster. Pure raytracing is even better and you can still raymarch the shadows.
Interesting bit of OpenCL kernel work and a great piece of GLSL work.
I must admit as I perused the composition in between preparing a stir fry I wasn't at first captious of the GLSL shader providing both the sphere and grid GLSL x,y,z,w so started with seeking to replace the OpenCL grid you used with one of my own. [to be fair to myself it wasn't entirely obvious to me from a mere glance that a sphere was intersecting the cube].
I did eventually twig to the fact that no sphere kernel as such existed. LOL.
Just beginning to make fuller sense of this GLSL / OpenCL combo - makes for a great study, like figuring out the way increased iterations results in the rendered grid shifting further back on the stage for instance.
Thanks for sharing, psonice :-)
Your posted composition has given me a fresh insight into how an OpenCL kernel Grid produced Mesh can and will operate. [e.g doesn't wireframe very nicely •~].
On the other hand, it does iterate pretty well producing the faux sphere and cube.
Umm... ignore the openCL stuff :D It does almost nothing (or should do, if it does something it's wasting it's time.. actually I neglected to check as I was rushing at the time..)
You can replace the openCL mesh + filter + mesh renderer with a straight billboard and it'll still work. The reason for the openCL is that QC optimises a billboard/glsl mesh out of existence if it's off-screen, and it does go off-screen.
What the mesh/billboard/whatever is inside the glsl patch does: it's the camera. Moving or rotating the mesh moves the camera around. Scaling changes zoom or perspective. The GLSL patch then resets the vertex positions to fill the screen so you don't see it move.
Also, the iteration counts: it's nothing like a QC iterator. It doesn't iterate the mesh or anything like that.. there's no mesh to iterate. The iteration count is the number of stages in the rendering, more iterations just gives higher accuracy. Low iteration counts means it's very inaccurate.
Best way to think of this: it's software rendering. More or less raytracing, the method is slightly different but it's the same thing really. Except the software is written in GLSL, running on the GPU. That's why it only needs 1 polygon to draw on.
Oh, and the sphere: it's a distorted, rippling sphere, making a hole in the cube. It doesn't look like a sphere :)
Thanks for taking the time to make all that crystal clear, psonice.
The composition is in many ways a different type of workflow to those I have often been making in the past.
Interesting to study though.
It's very different - you can do a complete animated world, with only glsl shader, billboard, and patch time (to feed the animation). Camera, lighting, models, materials can all be done in pure glsl.
But QC is still ideal for this kind of stuff, because trackball and 3d transform are great for moving the camera, and LFOs and timelines and the like are perfect for animating. Plus it's great to just render lighting and stuff in the shader, then do a ton of post-process in regular QC :)
Oops! I noticed yesterday that I made a slight error in the code on that last comp. It's supposed to raymarch the scene, then calculate the normal for lighting. Instead, it's raymarching the scene, then raymarching the scene 6 more times to calculate the normal. I.e. it's probably 5-6x slower than it should be :D
I'll post a new one up at lunch time, with that fixed, a raytraced floor (raymarching the floor speeds it up a lot, and improves accuracy too.. not a bad improvement!) and more interesting geometry. In fact I'll add in domain repetition, infinite objects definitely help show how powerful this technique is ;)
OK, here we go. Unlimited cubes, soft shadows, moving camera + light, and a light that cuts into the geometry within a certain distance.
And HUGELY optimised compared to the last one - it runs much faster at 4x higher res. It's a mix of marching and tracing for speed, with a few other hacks.
this is pretty cool. interesting the resolution is independent of the screen size ? going full screen didn't seem to bother framerate
way speedier :-)
It's in a render-in-image patch. It doesn't need to be, it's just handy to turn the resolution down now and then when working on something heavy. Cut + paste it out if you want to change resolution.
I'd recommend opening the 2nd example patch below though, it'll go a LOT faster ;)
It's in a render-in-image patch. It doesn't need to be, it's just handy to turn the resolution down now and then when working on something heavy. Cut + paste it out if you want to change resolution.
I'd recommend opening the 2nd example patch below though, it'll go a LOT faster ;)
i like this first one. looks sort of melting like but yes the fps is way better on the second one. i should study that one. just so i understand your zooming into a cube and using the sphere to cut through it. the ray march is to determine where the point is in relation to the camera so you can cut ? is the f function the function that does the sort of boolean subtracting or cut i'm calling it ? sorry for so many questions i really want to understand this.
ok i see a bot differently now. your mesh render is where the motion is coming from. so the sphere is more in the middle of the cube. i think it would be cool to place the sphere where the camera is so no matter what kind of track ball you do the cubes get cut, or would an operation like that be to heavy duty ?
Yeah, that's kind of right. The actual scene consists of a ground plane, a fixed cube, and a sphere that punches a hole in the cube. The sphere is rippling, with lots of sin waves, that's why it has that 'melty' look.
The way it works is a bit weird, but roughly: For each pixel on screen the shader calculates the camera position and the direction of the ray for that pixel. It gets that from the position + rotation of the mesh.
Then, it marches along the ray until it hits a surface. It's the way it marches that's unusual. You calculate the distance to the nearest surface for the start point in any direction (this is why it's called a 'distance field'). This is the 'safe distance' you can march along the ray without hitting a surface. Then you move along the ray by that distance, and repeat, until it hits a surface (or comes very close, which is faster than an actual collision).
So, basically, you calculate the ray start point + direction, and feed that to the rm(..) function - this is the ray marcher. That steps along the ray, and each time it uses the distance function (which is f(...)) to find the smallest distance.
The f(..) function has to know the smallest distance to a surface, so normally it's like this:
return min( obj1(currentPosition), obj2(currentPosition));obj1(...) is a function that gets the distance to that kind of object, so you see i have functions for distance to sphere, cube, plane. It returns the minimum distance out of all objects in the scene.
You change that min(..) function if you want to make objects intersect, eg.
return max(cube(), -sphere());will subtract a sphere from a cube.
All of this gives you a depth map basically, you just know the distance to the object. You can calculate the normal by testing the distance at points around the place where the ray hits, then do lighting based on that.
The shadows are done by marching from the point where the ray hits, back to the light. Plus some magic to make them soft.
The intersecting sphere is tied to the light position. It's trivial to change it to the camera position - just attach the LFOs with 120s periods to the sphere's x,z position (they're attached to the light's mix of 2 LFOs).
Or stop the camera moving, and do whatever you want with it :) There's no performance impact from moving the sphere.
this is awesome. i bet with some textures it looks really nice. with a system like this do you have to make your objects parametrically ? i have yet to be able to pull of this cl to gl interoperation i think its called to pull in verts to gl from cl ? will this sort of thing work with glsl ?
ok i tried that doesn't seem to bother anything. i will try and add the wobbly noise to the sphere from the first example as i like how it is melty looking.
Generally, yeah, you have to make parametric objects. Texturing is difficult too, you either generate textures in code, or you have to figure out the texture coordinates based on the place the ray hits the object. Not straightforward..
Pulling vertices in is theoretically possible, but you'd have to pass them in as uniform variables I think, meaning it's very limited. The way around that is to render your object the normal way (separately from this render), but inside a glsl patch that writes normal + depth position out to the rgba channels. You could then mix it into your raymarched scene with a straight depth test. You'd have to be very careful to match camera angle, perspective etc. exactly though, not easy.
looks really interesting when you apply the noisy sphere from the 1st example with the 2nd. you take a huge fps hit but i think it looks cooler.
chris you sent me an example one day called massive clod. i still have it. this had textures and fog, do you think that would be a good place to start if i wanted to add textures.
i mean the ray is returning distance. so would i have to calculate the uv grid from n ?
or
vec3 n = march.x == 1.0 ? (vec3(f(point+e.xyy),f(point +e.yxy),f(point +e.yyx))-f(point))/e.x : (vec3(tracePlane(camPos+e.xyy, rayDir), tracePlane(camPos +e.yxy, rayDir), tracePlane(camPos +e.yyx, rayDir))-tracePlane(camPos, rayDir))/e.x;yeah doesn't seem that straight forward i guess.
Nice. Try melty cubes next ;)
melty cubes looks nice but it messes up the lighting sort of washes everything out but im sure i can mess with the lighting a bit to make it look proper.
Odd, it shouldn't do. I'll take a look, find out why.
You can comment out the shadow bit in main() btw, which will speed it up. It'll still be lit. The line is something like "intensity *= ss(...);" (ss = soft shadow).
i was just using the code you all ready wrote for noisy cube that you commented out. then return max as you had it so maybe there was some issue with the code you commented out or i would need to return it differently i will try and mess with the shadow.
n is the normal, so you can't use that for the texture coordinate. You could use it for environment mapping though.
I didn't write 'you massive clod', it was parapete (it was for linux, I just ported it to qc). From what I can guess though, the fog is just something like "lighting += distance" and the texture is just based on the coords of the surface (i.e. it's not textured, it's just a surface pattern programmed in).
To calculate the texture coords, you need to determine the position on the surface of the object. Easy example, the ground plane. Get the x,z coordinates at the point where the ray hits the ground. Those are your texture coordinates. If you're hitting a cube, it's harder.
Then you need to change the distance function f() to return a vec3 (distance, texCoords.xy). Instead of using "min(obj1, obj2), you have to do stuff like "return obj1.x < obj1.y ? obj1 : obj2" and it gets ugly with more objects ;)
Urgh. I just tried this, and got it working perfectly. Then I hit save, and QC crashed :( (Probably the CL patch? I need to experiment, to find out what causes it, but it's highly unstable lately and it wasn't before).
What you need to do: Instead of "cube(cube1, pos + vec3(sin....));" use "cube(cube1 + vec3(sin... ), pos)".
i got it working it was the cube.z messing with things. looks fine with noisy cube.x and cube.y; i tried adding some parameters for the soft shadow's float r, float f and float l inputs but for what ever reason. going uniform ssr; for float r input parameters doesn't produce me an input ? very weird.
And another one. More complex scenery...
Another experiment:
Now in colour. The walls are acheived by using mix(obj1, obj2) instead of min/max. Some pretty cool effects are possible with mix()!
Colour is tricky. I have to get both the distance to each object, and it's colour, then get the minimum distance + colour of that in the distance function. This means all those nice min/max/mix functions don't work, because you lose the colour.
I'm getting the hang of it now :) Just an experiment, but it's progress.
Verrry impressive, mate!
a|x
Very nice! mix() is a great function. Excellent for warping geometry, as well as just mixing colours.
a|x
Yep, only catch really as that it's very hard to predict the outcome at times. The mix value is never what I expect it to be, I have to experiment all the time to get it right :/ The buildings in the video below are made with mix - I'm making 5 boxes, and using a combination of min() and mix() so that the boxes get combined, but sometimes blended to form more complex shapes.
Cool work. How long did the render take?
At 720p, with AA, and the quality settings cranked, a few minutes. But it's realtime in general, running about 15fps at the moment at 640x480. It's quite unoptimised too, it would probably run at 30+ once it's tidied up. This is on a radeon 4670, running about a hundred other apps and another heavy QC app running.
Nice contrast of the grey volumes and the coloured lighting. How is the lighting coloured? Is each 'ray' actually evaluated for a colour value or is the ground cycling some coloured texture?
(haven't downloaded any of your code, too busy to get distracted by such mastery ;-) )
It's just a left-over bit of colour from when I worked out how to do materials.. the ground is simply coloured using sin(position.xz) or something like that. I'm going to use the colour channels for material parameters (so r+g are texture coords, b is material index to select a texture) and alpha to store intensity. This way I can put all the texturing in a second shader.
I'm not sure mix() is a linear function. It may be an 'equal power' curve, which might make the results different from what you'd expect from a simple linear interpolation. Not sure on that one, though.
a|x
Adding some details:
I'm simply subtracting a repeated cube for 'windows', and adding a thin block to the top of each block for the 'roof' (otherwise you get windows in the roof, which looks... odd ;).
Lighting here is simple AO plus a directional light without shadows. Just wondered how it would look. Actually, not bad, except that the ground messes up because of the 'repeated' geometry. You can see the squares where it gets repeated, the shadow on the ground is only aware of what's inside its square, because the normal points straight upwards. Maybe I need to jitter the normal or something.