Environment maps and vertex noise

Udart's picture

Hi there, I played around with an environment map shader I found here.

http://www.ozone3d.net/tutorials/glsl_texturing_p04.php

And I quite like the result with the teapot here. I found the studio image somewhere - sorry I don't remember where...

For fun I added some noise to the vertex shader - try turning up the noise parameter. That doesn't work so well but I am a noob so if anybody has suggestions for improvement in the shader let me know.

PreviewAttachmentSize
teapot w map and noise not perfect.qtz952.6 KB

Udart's picture
Re: Environment maps and vertex noise

It's all about calculating normals isn't it? OK that's a bit beyond my reach right now....

Udart's picture
Re: Environment maps and vertex noise

So to continue this conversation I am having with myself: Obviously Kineme have done it right in the 3d tools and the environment shader works quite well here I think. I quite like the facetted effect actually.

PreviewAttachmentSize
Kineme 3d w environment.zip298.29 KB

dust's picture
Re: Environment maps and vertex noise

have you tried using a mesh noise filter before the shader ? sometimes when doing vertex displacements or noise etc.. in the shader lighting and shadow environment patches will not work properly.

Udart's picture
Re: Environment maps and vertex noise

Mesh noise is a bit tricky sometimes. But it's appealing that underneath lies OpenCL code that can be edited - if I knew how. But thanks for the tip.

I have read a bit more, and the explanation seems to be here: http://tonfilm.blogspot.dk/2007/01/calculate-normals-in-shader.html

To keep the lighting you need to recalculate the normals and you cannot do that in the shader it seems (unless you have a grid or another mesh that is created programmatically)

That being said I found an error in the code and so this version is a bit better IMO. At least it works OK for smaller amounts of noise.

PreviewAttachmentSize
teapot w map and noise - a bit better.zip162.52 KB

gtoledo3's picture
Re: Environment maps and vertex noise

You can add lighting to your GLSL code, and it can work with the vertex noise...I think the stock vertex noise shader example from 10.5 even has some type of simple lighting code.

You can also "get" light positions, specular, diffuse, etc., values from a Lighting Environment patch when the GLSL shader is "inside", and make that work with your GLSL shader lighting code.

gtoledo3's picture
Re: Environment maps and vertex noise

Instead of just going "it's possible", I figured I'd post a composition. http://kineme.net/composition/gtoledo3/VertexNoisewithLightingandEnviron...

dust's picture
Re: Environment maps and vertex noise

thats very interesting gt. when i displace the vertex the lightning environment's shadows do not seem to work properly. i tried with a displacement patch and your composition and seem to get strange artifacts with shadows out of place ? (see pics) the lighting itself without shadows seem to work properly ?

i thought this was more or less a bug with qc. your example does show a good use case of how to get at the lighting environment from the glsl shader. i have tried shadow mapping but i still get the same results. in the past there has been some discussion about shadowing in where it has said to not be possible. http://kineme.net/Discussion/GeneralDiscussion/Shadows#comment-4245

either way lighting does work well but i havent had much luck with shadowing any sort of vertex noise displacement. one would think that if you can access the lights position and direction etc.. then some sort of shadowing in a shader should be able to be accomplished, however that is beyond me in with glsl. this looks likes a decent tutorial... http://fabiensanglard.net/shadowmapping/index.php

however when implemented with any kind of displacement the issue still arises, however when displacing the mesh via a noise filter etc... shadows seem to be in the proper place.

gtoledo3's picture
Re: Environment maps and vertex noise

The shadows function in combo with shaders? :) entirely different ball of wax from GL lighting. Man, I wish they hadn't have lumped the things together.

There's no reason that rolling your own shadow mapping wouldn't show the correct distorted vertices by a process like this unless that pass is done first, because you would be getting the scene depth, which would have that info. Some comments in the past revolved around the quality of the depth channel, but it's still possible either way, just fiddly. Also, many shadowing glsl shader examples input shadowmap sampler type, so you have to convert the z lane in the shadow texture to "z" in a 2D texture sampler sense, placing the values in one of the channels.

I know what you mean about the cl thing though. That's why I was hot on fixing it and putting the fixes out when the mesh filters broke :)

gtoledo3's picture
Re: Environment maps and vertex noise

Hey, thanks for that link; that's a real nice simple example and it doesn't use a 2D Shadow texture extension, so that part doesn't need to be rewritten. Maybe I'll set it up tomorrow morning :)

edit- So I'm looking at it, and there's a couple of things that don't make sense to me.

I don't understand this bit - "no rocket science here, we transform the vertex with the camera matrices, the same vertex with the light POV matrix and we get the fragment color".

Now, I understand the theory, but I don't understand where the light POV matrix is being obtained.

Before this step, he takes this matrix:

const GLdouble bias[16] = {   
         0.5, 0.0, 0.0, 0.0, 
         0.0, 0.5, 0.0, 0.0,
         0.0, 0.0, 0.5, 0.0,
      0.5, 0.5, 0.5, 1.0};

...and concatenates it with the modelview and projection matrices, passing it along in the texture matrix. How is that the light POV matrix in anyway, with or without being concatenated with the modelview and proj matrices.

But anyway - Ok, it seems as though that can be simulated in the vertex shader by doing something like this:

const mat4 bias = mat4(
        vec4(   0.5, 0.0, 0.0, 0.0),
        vec4( 0.0, 0.5, 0.0, 0.0),
        vec4( 0.0, 0.0, 0.5, 0.0),
        vec4( 0.5, 0.5, 0.5, 1.0));      
 
   // Used for shadow lookup
varying vec4 ShadowCoord;
 
   void main()
   {
      ShadowCoord= gl_ModelViewProjectionMatrix  * gl_Vertex * bias;
 
      gl_Position = ftransform();
 
      gl_FrontColor = gl_Color;
   }

I'm not getting how that gives the right viewpoint for the shadows... I mean, it doesn't, so I'm not thinking of something correctly/it's getting lost in translation as far as what this person is intending.

I'm generating the shadow map depth ala:

vs:

varying vec4 NormalDepth;
void main()
{
   //Transform vertex by modelview and projection matrices
   gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
   NormalDepth = vec4(gl_Normal * 0.5 + 0.5, gl_Position.z);
}

fs:

varying vec4 NormalDepth;
void main()
{
   vec4 normal = NormalDepth;
   normal.rgb = normalize(normal.rgb);
   gl_FragColor = normal;
}

I mean, I can "see" where it's doing something, and carving away parts of the depth...sort of looking right, but I don't "get" how this person is thinking to control the light POV or why that matrix is being multiplied in (just do it, the other way sucks, it seems to kinda say without reasoning), instead of the reverse cam transform. I guess it's assuming that the light is at eye position, but it doesn't really do that either (?).

The result does something that looks like it's kind of drawing a shadow texture that could be overlayed with the scene, but it's white, and not really right (no pun intended). The demo obviously looks like it works from the pics, I'm just getting mixed up in transferring it.

gtoledo3's picture
Re: Environment maps and vertex noise

I think I see where in getting it wrong now, but I'm not going to go over it until tomorrow. Not transforming the one world to light pov correctly I believe.

edit-

Checking it out... I figure I'll post this in case anyone wants to twiddle with it (still not done). I was checking out the way that the scenes are typically shaded to do the depth comparison, and it's done more like this sometimes:

https://www.box.com/s/htblfzk9czu9icem3h7l

I don't know if that's better, because with the first method I posted, the depth channel is still there + normals, whereas this method encodes every color channel (maybe not alpha, can't remember). Still, I post it since it's what I'm seeing in some examples of shading the scene to do the depth comparison part.

So, I think if the scene is rendered twice in parallel (eg., two render in images), once from the light viewpoint, one from regular eye position, and then the depth comparison is done to throw away stuff that shouldn't be shadowed, it will be the most "basic" implementation of shadow? I was having some trouble getting the scene rendered from light POV last night though... if anyone has done it in GLSL shader by grabbing the Light Position, feel free to chime in :)

Is the shadow map created by warping the projection/modelview matrix of the actual scene, or by rendering the image on a glsl grid and warping the texture matrix according to the light POV? (bueller?) I guess I need to dig through the fine points of some of these examples.

It's pretty darn easy to set this up using the actual code examples, cut&pasting, tweaking etc., but re-translating it to work with stock QC patches can get a little puzzling because...well, you actually have to understand the nitty gritty of every step. I had it casting sort of right last night by just picking an arbitrary viewpoint for the Light POV to test with, and it had that crappy moire on some places.

Going a little further, if a multipass type RII was made so that it would be more efficient of a process, I wonder how it could be exposed to the user to change the camera POV's as would be needed.

gtoledo3's picture
Re: Environment maps and vertex noise

I looked back at the "do shadows in QC using built in patches" a few minutes ago, and finally got the most crappy basic fundamental step going.

I mean crappy (pic attached); I'm not doing anything to smooth the shadows at all, and there's no lighting or materials. Just the object, the shadow map, the POV skew, and the depth comparison to get rid of shadows when needed.

The machine I'm running it on at the moment has an intel gma X3100, which is pretty horrible, no MSAA, etc. It's not quite doing the right thing in the vertex shader with something I know should work, so I'm doing a fiddly workaround. I'll try it out on a good machine tomorrow. Now I don't know why I didn't just do it sooner.

PreviewAttachmentSize
shadow_basic.jpg
shadow_basic.jpg46.44 KB

dust's picture
Re: Environment maps and vertex noise

awesome cant wait to see the shader. are you using a gl read from the depth buffer or RII to render the projective map from the lighting perspective ? or is this the cheap shadow method.

http://kineme.net/Discussion/GeneralDiscussion/Shadows#comment-4248

either way its interesting.

was trying to feed in a gl RII with depth as a shadow map to a shader but failed pretty miserably.

this explains it ok.. but how to make work in qc ? http://www.gamerendering.com/2008/10/23/basic-shadow-mapping/

PreviewAttachmentSize
shadowMaper(try1).qtz7.1 KB

gtoledo3's picture
Re: Environment maps and vertex noise

Oh no, that "cheap shadow" thing is almost a 2.5D kinda thing. I didn't re-look at my old composition, but if it's what I think, the Instructions patch wound up using a kind of drop shadow concept that was pretty similar. I'm pretty sure that's a common graphics technique that was done in print for years.

This is actually rendering the scene from the light POV to create the shadow map. I'm not using the gl read pixels depth image output in any case.

One, I can't use the gl depth buffer of the regular scene to do this, because the info of what's there in the geometry at the light POV angle isn't actually there. Two, using the "gl read pixels" depth on even the altered POV version of the scene doesn't give a good result for me either. I'm passing the depth values using a shader.

Udart's picture
Re: Environment maps and vertex noise

It's awesome that you're giving it a go. I have been reading some of the same docs recently knowing that I'm not at a level where I could translate that into QC yet. But I am very curious to know how you approach it - even if the results are disappointing. Are you using 'GL Look at' and some of the other Kineme GL patches?

Even an inaccurate shadow map would be better than the one in the Lighting patch IMHO. At least we would have it as a separate pass that could be blurred/tweaked etc. depending on the context.