Shaders in a plugin ?

benoitlahoz's picture

Hi !

Tinkering again with these new worlds for me that are OpenCL, Objective C and OpenGL, I'm trying to make a plugin with a shader (the one we use to make our "shadows"), but even while reading more and more of vade's beautiful (but hard !!! :-)) code (and GT's Heightfield) I can't manage to find a "simple" way to do this.

I would like to output an image on a plane. No mesh complications...

Is there an example I could study to learn about it ?

Thanks again to you all !

franz's picture
Re: Shaders in a plugin ?

v002 erode. it uses a FBO to render on. For multipass shaders things will get more complicated.

benoitlahoz's picture
Re: Shaders in a plugin ?

Thanks Franz !

It seems much more simple, yes !!!

benoitlahoz's picture
Re: Shaders in a plugin ?

Well... Hum...

I finally tried with v002Glitch and the AnalogGlitch plugin. It seems so preciously done...

But, I have a trouble with passing colors to the shader (CGFloat* to GLfloat I guess) through the "renderToFBO" method, using glUniform4fARB.

It seems that I have a trouble with header files too : warning: 'TolalusuloPlugIn' may not respond to '-renderToFBO:width:height:bounds:texture:texture2:colorIn:colorOut:threshold:blur:fade:bypass:useImage2:colorInvert:'

If someone could tell me what I am doing wrong it would be of great help (once again !).

- (GLuint) renderToFBO:(CGLContextObj)cgl_ctx 
             width:(NSUInteger)pixelsWide 
            height:(NSUInteger)pixelsHigh 
            bounds:(NSRect)bounds 
           texture1:(GLuint)texture1 
           texture2:(GLuint)texture2 
            colorIn:(CGFloat*)colorInComponents 
           colorOut:(CGFloat*)colorOutComponents
          threshold:(GLfloat)threshold 
              blur:(GLfloat)blur 
              fade:(GLfloat)fade 
            bypass:(GLboolean)bypass 
          useImage2:(GLboolean)useImage2 
         colorInvert:(GLboolean)colorInvert {
 
   GLsizei width = bounds.size.width,   height = bounds.size.height;
 
   // this must be called before any other FBO stuff can happen for 10.6
   [pluginFBO cachePreviousFBO:cgl_ctx];
   [pluginFBO setBounds:bounds context:cgl_ctx]; // this generates a new texture for us
   [pluginFBO attachFBO:cgl_ctx];
 
   glClearColor(0.0, 0.0, 0.0, 0.0);
   glClear(GL_COLOR_BUFFER_BIT);         
 
   glColor4f(1.0, 1.0, 1.0, 1.0);
 
   glEnable(GL_TEXTURE_RECTANGLE_EXT);
   glBindTexture(GL_TEXTURE_RECTANGLE_EXT, texture1);
 
   glActiveTexture(GL_TEXTURE1);
   glEnable(GL_TEXTURE_RECTANGLE_EXT);
   glBindTexture(GL_TEXTURE_RECTANGLE_EXT, texture2);
 
   glEnable(GL_BLEND);
   glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
 
 
 
   // Importe les couleurs
   GLfloat   colorIn[] = {colorInComponents[0], colorInComponents[1], colorInComponents[2], colorInComponents[3]};
   GLfloat   colorOut[] = {colorOutComponents[0], colorOutComponents[1], colorOutComponents[2], colorOutComponents[3]};
 
 
   // bind our shader program
   glUseProgramObjectARB([pluginShader programObject]);
 
   // set program vars
   glUniform1iARB([pluginShader getUniformLocation:"inputImage1"], 0); // load tex1 sampler to texture unit 0 
   glUniform1iARB([pluginShader getUniformLocation:"inputImage2"], 1); // load tex1 sampler to texture unit 0 
 
   glUniform4fARB([pluginShader getUniformLocation:"inputInColor"], [[colorIn objectAtIndex:0] floatValue], [[colorIn objectAtIndex:1] floatValue], [[colorIn objectAtIndex:2] floatValue], [[colorIn objectAtIndex:3] floatValue]); // pass in uniforms
   glUniform4fARB([pluginShader getUniformLocation:"inputOutColor"], [[colorOut objectAtIndex:0] floatValue], [[colorOut objectAtIndex:1] floatValue], [[colorOut objectAtIndex:2] floatValue], [[colorOut objectAtIndex:3] floatValue]); // pass in uniforms
   glUniform1fARB([pluginShader getUniformLocation:"inputThreshold"], threshold); // pass in uniforms
   glUniform1fARB([pluginShader getUniformLocation:"inputBlur"], blur); // pass in uniforms
   glUniform1fARB([pluginShader getUniformLocation:"inputFade"], fade); // pass in uniforms
   glUniform1iARB([pluginShader getUniformLocation:"inputBypass"], bypass); // pass in uniforms
   glUniform1iARB([pluginShader getUniformLocation:"inputUseImage2"], useImage2); // pass in uniforms
   glUniform1iARB([pluginShader getUniformLocation:"inputInverseColor"], colorInvert); // pass in uniforms
 
 
   glBegin(GL_QUADS);
   glTexCoord2f(0, 0);
   glVertex2f(0, 0);
   glTexCoord2f(0, pixelsHigh);
   glVertex2f(0, height);
   glTexCoord2f(pixelsWide, pixelsHigh);
   glVertex2f(width, height);
   glTexCoord2f(pixelsWide, 0);
   glVertex2f(width, 0);
   glEnd();      
 
   // disable shader program
   glUseProgramObjectARB(NULL);
 
   glDisable(GL_TEXTURE_RECTANGLE_EXT);
   glActiveTexture(GL_TEXTURE0);
 
   glDisable(GL_TEXTURE_RECTANGLE_EXT);   
 
   [pluginFBO detachFBO:cgl_ctx]; // pops out and resets cached FBO state from above.
   return [pluginFBO textureID];
 
 
}

vade's picture
Re: Shaders in a plugin ?

Don't use v002 Erode as a template, its out of date and has some issues with how state is re-set in 10.6. Look at the blurs, and use the FBO classes if you must.

Also ensure you define the renderToFBO in an Execute category on your plugins class.

Just FYI, before you get to deep, read, understand and respect the licensing terms of the v002 plugins? If its not suited for your needs, don't use them.

gtoledo3's picture
Re: Shaders in a plugin ?

What does "the one we use to make our shadows" mean? Did you get that shader from mesh kit?

FWIW, QC supplies it with info, and does other stuff to accomplish what we wind up seeing. Even in that "full" implementation, it kinda sucks for a lot of things, and can be tricked into being inaccurate.

Unless you can get valid depth info with enough resolution, or raw vertex info or everything, I don't really think you can make a decent shadow implementation. Even if you got that shader working in the context of a QCPlugin that outputs image, what would your intent be... to output to a billboard that lays over the entire scene?

It would be cool to have some kind of great QC shadow implementation, but I think that's going to come from A- reading and passing a high quality depth image of the QC scene, and B - outputting it in a way that doesn't kill the resolution and C - ignoring the shader that currently makes shadows in QC as part of that.

I may be off base on some of this, but it's my current thought on it, and I'm sticking to it for now.

If you look at apple tech notes/reference library, you can probably suss out how to utilize shaders in a plugin from scratch without using v002 as a guiding point. When I used last published v002 that used shaders (that I've seen anyway), I had to change some stuff to get it to not error in 10.6. I think it was something with how the file gets loaded (I think some method was deprecated?), and also with the error logging. Since then, I just looked at Apple's site, and did it from scratch, since I happened to be browsing and saw some breakdown on that topic (I cannot remember where the link is, but I know it's there).

vade's picture
Re: Shaders in a plugin ?

Have you looked at the newer source for more recent releases? It all compiles clean.

benoitlahoz's picture
Re: Shaders in a plugin ?

Thanks vade and GT for your replies and your works... I've read ang agreed with the licensing terms... and read the "license file distribution" thread too.

To reply to GT's question : the shader "we use to make our shadows", is only used to track, fill and transform shadows by compositing. Kinect is surely better at silhouette tracking though.

It was built by the CNRS (french scientific national research center) during a bigger project we were experimenting from 2008 until the end of 2011.

It is open source, and a Jitter version of its implementation was made by one of our partners (we were 3 partners : CNRS, 2 theater/numeric companies) but it is very slow.

My theater company is working with Quartz Composer, then we translated the shader from CG to GLSL to use it inside a GLSL patch. We have very good results with a good IR light.

What I would like to do is to make an usable plugin and implement other functions inside the plugin, so we can use it in a simple way.

Also, it would be more simple to distribute it (freely and open source).

As I am going deeper in Quartz understanding I tried to make an openCL treatment of the resulting "shadow" of the shader, then I realized that I needed a C treatment of my resulting points, so I went to make a plugin in XCode, then I'm trying to come back to the basis of the project purposes : having a simple and distributable (for performing arts and educational purposes) tool for shadow treatment.

I guess I went to a certain point with vade's AnalogGlitch plugin, but I have this trouble I can't manage to resolve about the "renderToFBO" method. I have some kinds of headaches with this and getting mad !

Currently, I'm only displaying a checkerboard, with an exception about renderToFBO and a "initialization makes interger from pointer" in the renderToFBO call. Argh !

I'll look at your "blur" plugin, vade. I want to thank you again for sharing your work, which is really impressive (specially your enlightning way to share methods between plugins).

Greetings,

Ben

vade's picture
Re: Shaders in a plugin ?

No worries!

In your interface for your plugin (in your header) you need to make sure you have

@interface myPlugin (Execution)
- (void) renderToFBO: (snip...)
@end

Underneath the normal class declaration.

The (Execution) is called a category tells Obj-C that the method belongs to the Execution category/ phase (see your @implementation / .m ), and that renderToFBO method will be called from within there. If you've made changes to the renderToFBO method (adding arguments to suit your need), you need to make sure all declarations of the method and everything calling it matches. Are you getting compilation errors or warnings?

benoitlahoz's picture
Re: Shaders in a plugin ?

Wooooow... Sorry for this : I kept your "texture" instead of my "texture1" in the call of the method...

Well... Now I have a display. But it's kind of weird in colors, and nothing works... ! But well, I have a display :-)

[edit : actually, I'm passing 4 GLfloat for each colors, is it the good way to do it ?]

vade's picture
Re: Shaders in a plugin ?

Pass the CGColorRef. Also it looks as though you are using more than one image from QC for your effect. I would highly suggest making a "passthrough" shader, with one image input, and get that working. Baby steps. Plus it allows you to verify what stages are, and are not working.

benoitlahoz's picture
Re: Shaders in a plugin ?

"CGColorRef" OK !!! Thanks !

(and thanks for the advice ! You're right, I'll do this.)

benoitlahoz's picture
Re: Shaders in a plugin ?

Hi !

I have a new trouble (baby steps... ). It seems that I can't manage to link the shader with the program :

2011-03-23 11:34:05.586 Quartz Composer[1401:a0f] *** OpenGL error 0x0503 (stack overflow) in function "-[QCPlugInInputImage unbindTextureRepresentationFromCGLContext:textureUnit:]"
2011-03-23 11:34:05.595 Quartz Composer[1401:a0f] >> WARNING: No such uniform named "tex0"

I took both "v002Blur" and "v002Glitch" as examples. I transformed the shader so it doesn't use anymore "if" conditions, to make it simple... But it still doesn't work. Don't know what I'm doing wrong...

vade's picture
Re: Shaders in a plugin ?

Order of operations and GL State leaking. You need to really understand how OpenGL works, and how state changes effect the pipeline. Take your time ad go through step by step.

benoitlahoz's picture
Re: Shaders in a plugin ?

Thank you very much Vade. I guess I'm too impatient.