Stencil buffer in QC ?

benoitlahoz's picture

Hi all !

I've been trying to implement a GL_STENCIL_TEST in a 2Dgeometry plugin. After fighting to understand what was going wrong, I decided to test with the GL Context Info kineme patch and discovered... a stencil buffer with 0 bits...

OpenGL Extensions Viewer tells me that I have 0 or 8 bits in the stencil buffer.

Does anybody know what could be the origin of my trouble ? Is the pixelFormat or other in QC disabling the stencil buffer bits ? Is there a way to set the stencil buffer number of bits ? Or to change the pixelFormat (don't even know if this is related...).

I read that the swap between the Intel card and the NVidia on wasn't optimized though...

Thank you !!!

gtoledo3's picture
Re: Stencil buffer in QC ?

You have to first create a stencil buffer and make sure that you've setup your plugin's internal pixel format correctly to set the stencil depth. It would be typical to use one of the calls to setup a depth/stencil buffer that's joined as opposed to one of the "only" stencil buffer calls, but I guess it depends on what you're doing. Also, this is all something that likely will be happening inside of your plugin, then you'll be outputting an image that has no stencil buffer once you've used it for whatever you're using it for.

... which brings me to, I'm sure there's a good reason, but the stencil buffer is typically used for masking tricks, and shadowing (though it isn't needed at all, it's just used in some techniques). Hearing "stencil buffer" in connection with "2D geometry" makes me wonder if you're getting a little sidetracked, or if there's a way simpler way to do whatever you want.

benoitlahoz's picture
Re: Stencil buffer in QC ?

Actually I'm trying to render non-convex and self-intersecting polygons, without having to triangulate them in the CPU. This pixel config you're talking about seems exactly what I'm looking for !

I have the code of the GL_STENCIL passes, that must work if I all understand well but...

How can I "setup the plugin pixel format" ?

gtoledo3's picture
Re: Stencil buffer in QC ?

When you set your pixel attributes, it will be something like:

NSOpenGLPixelFormatAttribute attrs[] =
{
NSOpenGLPFADoubleBuffer,
NSOpenGLPFAAccelerated,
NSOpenGLPFAColorSize, 32,
NSOpenGLPFAAlphaSize, 8,
NSOpenGLPFADepthSize, 32,
NSOpenGLPFAStencilSize, 8,
NSOpenGLPFAAccumSize, 0,
NSOpenGLPFAMultisample,
NSOpenGLPFASampleBuffers, (NSOpenGLPixelFormatAttribute)1,
NSOpenGLPFASamples, (NSOpenGLPixelFormatAttribute)4,
0
};

Take that with a grain of salt, it may not be appropriate to use these exact attributes. Once you've established the buffer, then you should be able to use stencil related functions. That said, I've never had occasion to use the stencil buffer - I've never had a time where it hasn't been easier to just draw my geometry the shape I need and forget about masking, or mask via a texture or shader instead. Someone talked me out of stencil buffer stuff at an early stage in my OpenGL learning so I have no clue how well it's supported on OS X.

For example, I had something I could have used stencil buffer for a month or so ago, and instead wound up passing the texture to a shader, searching for the color value of the part of the texture/mesh that needed to go away, and used the "discard" function in GLSL.

benoitlahoz's picture
Re: Stencil buffer in QC ?

Thanks a lot ! I'll try it ! Stencil bitwise op could be slower than a shader may be, as stencil require that I draw one triangle per vertex... I don't know, as I usually rely on Kineme GL to display custom-plugin-made vertices structures... So I'm still a big noob in OpenGL... I'm just trying :-) But I must admit that I would be very very happy if it works.

EDIT : Sorry GT, but... when and where do I set my pixels attribute ? I use to work with the standard CGLContext...

gtoledo3's picture
Re: Stencil buffer in QC ?

benoitlahoz wrote:
Thanks a lot ! I'll try it ! Stencil bitwise op could be slower than a shader may be, as stencil require that I draw one triangle per vertex... I don't know, as I usually rely on Kineme GL to display custom-plugin-made vertices structures... So I'm still a big noob in OpenGL... I'm just trying :-) But I must admit that I would be very very happy if it works.

EDIT : Sorry GT, but... when and where do I set my pixels attribute ? I use to work with the standard CGLContext...

I'm assuming that you're outputting an image, or doing your calculations, creating a texture, and then applying that to objects that your plugin creates.

If QC doesn't setup a stencil buffer, there's none to access. I don't see why it would create one, but you never know. Like I said, it's a marginal technique. I don't know what you're up to exactly, but usually you're going to test the depth buffer, and then use the stencil buffer to mask depending on depth. So, you could probably do the exact same thing with a clip plane, shader, or texture, which will all likely work with QC's current OpenGL pipeline.

benoitlahoz's picture
Re: Stencil buffer in QC ?

No, I'm drawing directly in the GL context. I'm looking for a way to fill non-convex polygons.

As

  1. I don't want to triangulate on the CPU (I implemented poly2tri for Constrained Delaunay, but it would be too slow to edit the polygons in real-time)
  2. I can't use gluTesselator in QC, as I read somewhere on a kineme post

I would like to find a fast and efficient way to do it...

gtoledo3's picture
Re: Stencil buffer in QC ?

benoitlahoz wrote:
No, I'm drawing directly in the GL context. I'm looking for a way to fill non-convex polygons.

As

  1. I don't want to triangulate on the CPU (I implemented poly2tri for Constrained Delaunay, but it would be too slow to edit the polygons in real-time)
  2. I can't use gluTesselator in QC, as I read somewhere on a kineme post

I would like to find a fast and efficient way to do it...

Hmm, this is where my knowledge starts to dwindle, since I haven't really crossed this hurdle before.

You can use glu in QC, but it may be a scenario of "everything happening in your plugin, then output image and/or other data", or that it doesn't work in QCPlugin for some reason... I'm not sure why you couldn't in a regular OpenGL scene, so I might try it before assuming it can't be done. The thing is, GLU has an association with slowness, rightfully or wrongfully, so you might want to check the library and see what it's actually doing in the Tesselator and base your code on that. I don't know too much about gluTesselator, use cases, performance, etc.

All that said, one will decompose concave (non-convex) polygons into convex usually by using some kind of CPU based algorithm. There may be some GPU/OpenGL based solution, but I don't know of one. The other "trick" would be decomposing to the true least amount of polygons.

Are you just trying to make some kind of image processing thing that looks like delaunay? I was thinking about this recently when pondering delaunay/voronoi and past forays, and was thinking of this "cheap" pseudo algorithm:

  1. Maybe draw a grid and use a delaunay type/looking triangulation.
  2. Color each grid vertex the same color value as the pixel value present in the input texture at the same location. If making voronoi, draw a cone in the appropriate position.
  3. Use some GLSL noise in the vertex shader to perturb the points of the grid a bit, but not so much that the vertices aren't still basically in the same area of the raster. ( Maybe something like the ashima noise ? http://kineme.net/composition/gtoledo3/AshimaVertexNoisewithLightingandT...). This should give it "the look", because the points won't be in a uniform grid anymore.
  4. If something needs masking, that would just happen with the input texture. Since the vertices are populated with pixel values from input texture, if something is black (or whatever), it will make that whole triangle also be black (or whatever...alpha, etc.).

....except that I wasn't exactly thinking of this in terms of grid/verts, but as a raytracing thing all in the fragment shader because of the context I was working in, and just with voronoi.

Maybe you're doing a free hand drawing thing though, in which case, none of the stuff I just mentioned applies :) I remember you looking into delaunay though, and this line of questioning seems to be possibly related to that, so I mention the idea.

benoitlahoz's picture
Re: Stencil buffer in QC ?

In fact, Delaunay is not any more a problem, as I implemented the poly2tri lib in the plugin. It works very well but is quite slow for polygons with many vertices (drawn with bezier for example).

The plugin I'm developping (and fortunately I'm at the end of it, apart this tricky rendering problem) is a kind of (little little !) Illustrator-like editor using a custom library I made. The custom lib, which was the first project I had, was aimed to be an interface between my different plugins using 2D geometry (to avoid manipulating arrays of vertices, but polygons objects), then I finally decided to make a quick editor plugin to feed other of my plugins (e.g. a Box2D quite large implementation I've been working on for a long time) with easy method on the geometry usable at any place.

So... yes it is based on "free hand" drawing, or user custom shapes he can create dynamically, with the mouse or anything else that can output an X and a Y. I put in input an option to triangulate and it works well, but I would like the renderer to render correctly the filling, even if the user want to move points on a complex polygon without triangulating.

I was thinking about using an FBO (or a VBO, I just don't know more about it !!!!) if I can render directly without passing by a texture, so I believe I can attach a custom pixelFormat on it ? Can I ? I should dive in vade's code to begin to learn about it...

Stencil buffer seems very interesting for a lot of purpose, but I discovered it while searching ways to do my render, and its major quality for me is to be able to perform bitwise operations on multiple "rendering" passes. There's an elegant solution to render concave polygons with the stencil buffer here : http://what-when-how.com/opengl-programming-guide/drawing-filled-concave...

The only thing is that you have to render one triangle per vertex. Don't know how far it would slow the rendering process, but I would like to test it...

Below, you can see one of my renders -which is exactly what I want to avoid ! :-)

PreviewAttachmentSize
Concave_Polygon.jpg
Concave_Polygon.jpg70.05 KB

gtoledo3's picture
Re: Stencil buffer in QC ?

benoitlahoz wrote:
In fact, Delaunay is not any more a problem, as I implemented the poly2tri lib in the plugin. It works very well but is quite slow for polygons with many vertices (drawn with bezier for example).

I'm not really familiar with that lib, but I think the slowness is probably because many implementations attempt to create points in bright places of the image, and then do the triangulation on those.

Whereas what I'm describing is to start with a normal distribution of points (a grid), do the triangulation on those at the start, warp point positions with noise, and rgb values are just laid over the grid... In reflection, I guess what I'm suggesting for that is very similar to a "crystallize" filter, except with a delaunay pattern. RGB values would pop on/off as opposed to the mass of points following the object of interest. The end result would be different from what many delaunay filters do now, but I think it would have a similar aesthetic and be way faster.

Sorry to sidetrack though.

Quote:
The plugin I'm developping (and fortunately I'm at the end of it, apart this tricky rendering problem) is a kind of (little little !) Illustrator-like editor using a custom library I made. The custom lib, which was the first project I had, was aimed to be an interface between my different plugins using 2D geometry (to avoid manipulating arrays of vertices, but polygons objects), then I finally decided to make a quick editor plugin to feed other of my plugins (e.g. a Box2D quite large implementation I've been working on for a long time) with easy method on the geometry usable at any place.

So... yes it is based on "free hand" drawing, or user custom shapes he can create dynamically, with the mouse or anything else that can output an X and a Y. I put in input an option to triangulate and it works well, but I would like the renderer to render correctly the filling, even if the user want to move points on a complex polygon without triangulating.

I was thinking about using an FBO (or a VBO, I just don't know more about it !!!!) if I can render directly without passing by a texture, so I believe I can attach a custom pixelFormat on it ? Can I ? I should dive in vade's code to begin to learn about it...

A vbo is a vertex buffer source, an FBO is a frame buffer source. The main reason to use a VBO is that vertices that you're handling will be mapped to GPU memory instead of RAM.

You've explained the concept a bit, but it also sounds complex, and I'm sure I don't entirely understand how all of the patches are supposed to work together. It sounds like you'll either need to pass an image output, or apply the texture back to vertices. I guess it depends on how it's supposed to composite/depth test with other stuff in QC.

Quote:
Stencil buffer seems very interesting for a lot of purpose, but I discovered it while searching ways to do my render, and its major quality for me is to be able to perform bitwise operations on multiple "rendering" passes. There's an elegant solution to render concave polygons with the stencil buffer here : http://what-when-how.com/opengl-programming-guide/drawing-filled-concave...

Ah, that's a cool use of that! It's easy to see how it would work by putting some triangles over one another and turning GL Logic Op (from GL Tools) to Invert.

I guess if you're outputting 2D anyway, you could just draw everything back to four verts and size it to Viewer size, or maybe a user input sizing or something. As long as you expose blend modes and depth testing, remember to clean up any opengl stuff you've invoked etc., everything should be pretty sweet for working as expected.

benoitlahoz's picture
Re: Stencil buffer in QC ?

gtoledo3 wrote:
I'm not really familiar with that lib, but I think the slowness is probably because many implementations attempt to create points in bright places of the image, and then do the triangulation on those.

Actually, I'm not using image input or output at all : I'm getting the tirangulation from an array of points. It's sometime slow where I have a lot of vertices (like in bezier curves) or when I have holes. I wish I could program in OpenCL to speed the process !

gtoledo3 wrote:
A vbo is a vertex buffer source, an FBO is a frame buffer source. The main reason to use a VBO is that vertices that you're handling will be mapped to GPU memory instead of RAM.

You've explained the concept a bit, but it also sounds complex, and I'm sure I don't entirely understand how all of the patches are supposed to work together. It sounds like you'll either need to pass an image output, or apply the texture back to vertices. I guess it depends on how it's supposed to composite/depth test with other stuff in QC.

Thanks, I'm a real noob in OpenGL... The patches are all using the same geometric objects and functions that simplify some checks and processes, so I can pass a custom oject through a structure output, without having to recheck, for example convexity, number of edges, orientation, etc.

Then, I can use my lib to transform OpenCV contours, prepare it to Box2D, or interact with it. I've made an editor with many possible interactions, and I'd like to be able to render non-convex polys directly... That's why I was hoping in stencil buffer :-)

gtoledo3 wrote:
Ah, that's a cool use of that! It's easy to see how it would work by putting some triangles over one another and turning GL Logic Op (from GL Tools) to Invert.

Thanks, I'll try it. But, if it works it will only for black and white polys I guess... Or maybe there's a way to create a mask with the resulting polygon (glColorMask ?) ?

gtoledo3 wrote:
I guess if you're outputting 2D anyway, you could just draw everything back to four verts and size it to Viewer size, or maybe a user input sizing or something. As long as you expose blend modes and depth testing, remember to clean up any opengl stuff you've invoked etc., everything should be pretty sweet for working as expected.

Well, I'd like to be able to extend my lib to 3D once... And, I would like everything to be connectable with standard QC patches...

cwright's picture
Re: Stencil buffer in QC ?

benoitlahoz wrote:
2. I can't use gluTesselator in QC, as I read somewhere on a kineme post

You can use this, but you need to make sure you set the CGL context to the current context first, since glu doesn't use the cgl_ctx stuff that the regular GL things use. There are a few exceptions in gluMacro.h that take cgl_ctx (and thus don't require you to change the context manually), but I don't think tesselators are one of them.

gtoledo3's picture
Re: Stencil buffer in QC ?

I was just suggesting that the kineme logic op would give you a quick visualization of what kind of values the stencil buffer logic op bit wise trick you reference generates, not that it should be used in your final implementation :)

benoitlahoz's picture
Re: Stencil buffer in QC ?

Do you mean that I have to reassign the cgl_ctx in the method that calls gluTesselator ?

It would be great if it works... I have troubles with the gluTesselators callbacks though...

For the moment I managed to have my polygons rendered correctly by using poly2tri in a light way, and the clipper lib for self-intersecting polygons. It works but it is quite slow while moving polygons with many vertices (many doesn't mean at all the kind of proportions we can have with OpenCL !!!)...

Thanks a lot for your help, @cwright and @gtoledo3.

cwright's picture
Re: Stencil buffer in QC ?

I mean you have to use CGLSetCurrentContext and CGLGetCurrentContext (get the current context, store it, set to cgl_ctx, make glu calls, then restore to the original context). It's not fast but it should work.

CLGContextObj oldContext = CGLGetCurrentContext();
CGLSetCurrentContext(cgl_ctx);
gluDoStuff();
...
gluDoMorestuff();
CGLSetCurrentContext(oldContext);

benoitlahoz's picture
Re: Stencil buffer in QC ?

Wow ! Thank you !

I guess it will be faster than my triangulation.