GLSL Help ...

yanomano's picture

I spend a little time to replay with the GLSL shader to displace vertex with luminance.For now the shader works well ( real time with open CV image conversion).the displacement is done in the Y axys from a global luminance value. I need help to modify it to displace in XYZ from each RGB values. Any help ? Alex ;) ?

/////FRAGMENT SHADER////
 
uniform sampler2D DisplacementMap;
uniform float blend;
 
void main(void)
{
   vec4 newVertexPos;
   vec4 dv;
   float df;
   vec4 vari = gl_Vertex;
 
   gl_TexCoord[0].xy = gl_MultiTexCoord0.xy;
   dv = texture2D(DisplacementMap, gl_MultiTexCoord0.xy);
   df = blend*dv.x + blend*dv.y + blend*dv.z;
 
   vari.z += df;
   gl_Position = gl_ModelViewProjectionMatrix * vari;
}
 
/////VERTEX SHADER////
 
uniform sampler2D Diffuse;
 
void main(void)
{
  gl_FragColor = texture2D(Diffuse, gl_TexCoord[0].xy);
}

cwright's picture
try this

/////FRAGMENT SHADER////
 
uniform sampler2D DisplacementMap;
uniform float blend;
 
void main(void)
{
   vec4 newVertexPos;
   vec4 dv;
   float df;
   vec4 vari = gl_Vertex;
 
   gl_TexCoord[0].xy = gl_MultiTexCoord0.xy;
   dv = texture2D(DisplacementMap, gl_MultiTexCoord0.xy);
   //df = blend*dv.x + blend*dv.y + blend*dv.z;
 
   vari.x += blend*dv.r;
   vari.y += blend*dv.g;
   vari.z += blend*dv.b;
   gl_Position = gl_ModelViewProjectionMatrix * vari;
}
 
/////VERTEX SHADER////
 
uniform sampler2D Diffuse;
 
void main(void)
{
  gl_FragColor = texture2D(Diffuse, gl_TexCoord[0].xy);
}

I've not tested this at all, but it'll displace vari on all 3 axes based on r, g, and b values of dv.

yanomano's picture
Thx chris

THX Chris i will test it as soon as i have one minute...

toneburst's picture
Did you try this?

I'd have though, with any old image (rather than a specially-prepared one), this would produce a mesh that was pretty-much a random mess. I may be wrong though. With specially-prepared displacement images, though, it's possible to create things like modulated sphere forms and other base primitives, with additional displacement based on luminance from an image mixed in.

There's a bit of info about this here http://machinesdontcare.wordpress.com/2008/04/

a|x

psonice's picture
vertex displacement

it's possible to do some pretty cool stuff without prepared images if you do something fun in the shader.. otherwise, yeah i'd imagine it would just be an intersecting mess. E.g. I did this quite some time back, i think using one colour to displace on z, the others to twist on x,y: http://img405.imageshack.us/img405/7492/cameradistort2vl8.png

yanomano: I wrote a plugin ages back that prepares standard rgb images for the vertex shader. It's buggy and I never got time to go back and fix it properly, so I never released it, but it's useable. It won't work directly with camera input as it's not in RGB format (well, I've not tested it but I assume not), but I've used it with tons of core image + regular QC stuff without problems. I suspect it'd be faster than the cv conversion method, although I've not checked. Drop me an email at psonice@gmail.com if you want it.

psonice's picture
vertex displacement + video works after all

Had to do some messing, but I now have vertex displacement running on a 64x64 mesh, with some fancy effects added, with live camera input, at 60fps! \o/

quick screenshot (and no, I don't have a mustache, comedy eyebrows, or horns :D That's just what the shader decided to add :)

franz's picture
kewl !!

hey, that's a good news. what's the trick finally ?
are you using a height field ? I remember some tests i did based on the GLHeightField plugin ( from Dev. folder) and it worked pretty well.

snapshot #3 is 128*128 grid running @ 60FPS on Ge.8600

feel like sharing infos ?

PreviewAttachmentSize
Picture 1.jpg
Picture 1.jpg93.76 KB
Picture 2.jpg
Picture 2.jpg93.73 KB
Picture 3.jpg
Picture 3.jpg120.05 KB

psonice's picture
colorspaces

I don't know why we're bothering with this effect, it just makes us look ugly :D

The trick was just to remove colorspaces from the texture before it's passed to the vertex shader. I noticed that the only things that would work all had "genericRGB" as the colorspace, so I made a plugin that just strips the colorspace and passes the texture back to QC. It still needs a bit more with the texture properties patch, and it seems to crash with anything other than a ^2 texture, but it works with pretty much anything now.

I think FBOs are probably still a better (and faster) way to do it (and I think that's what the GLHeightField plugin uses).

If anyone wants to fix up the code so it can be released properly, drop me a mail at the address above and I'll send it to you. I made it before learning cocoa properly, so it probably leaks memory and QC complains about it not supporting a callback somewhere, but it's very simple. It could probably be extended to support camera input properly too (at the moment I have to render in a render-in-image to force the pixel format to BGRA first).