reducing jaggies

jersmi's picture

Any advice on a way to reduce the bleed / aliasing shown in this pic? The shapes are composed of 2 sprites and 2 GL triangles, using 'difference' CIBlending mode.

I'd be happy with an fx cheat-- i saw vade's notes and plugin re: antialiasing, but that's a little out of scope at present.

PreviewAttachmentSize
jaggies.png
jaggies.png47.33 KB

gtoledo3's picture
Re: reducing jaggies

Without seeing the actual qtz, it's hard to say if this will help... it is an image kernel that is a real help in adding some polish in certain scenarios, and it doesn't impact performance negatively (it must have some impact, but it's negligible).

It's not really anti-aliasing, but gives the similar effect when it comes to sprites. It won't smooth anything besides the edges of the image, so it's not an all purpose solution by any stretch.

I use this on sprites...pipe your image source, run it through this, and then to your sprite. You will have enable anti-aliasing on the renderer (patch settings), and I think use over or add. Never tried it with a kineme GL Triangle or Quad...

FWIW, the Vade plugin/workaround doesn't work on my Intel, even in a separate app context (not implementing it in QC)... so it wouldn't be "cross compatible" if that is important to you at all, even if you got it working. Maybe there is a solution to that, but I don't know of one.

(those unibody Pro's are looking more and more delicious...)

PreviewAttachmentSize
softRect.qtz1.85 KB

jersmi's picture
Re: reducing jaggies

George-- thanks so much... ...it doesn't work... :(

Here's the comp if you want to have a look. to be polite I thought i should isolate the problem in a smaller comp, but i didn't cuz i thought maybe if someone had a look they could spot something i missed.

the focus is on the gl triangle and diagonal jaggies. i'm sure this is an old story in the world of cg...

btw, Vade's plugin doesn't work for me, either.

PreviewAttachmentSize
LRblinker2.qtz130.11 KB

jersmi's picture
Re: reducing jaggies

ps. George, if you do have a look and you have any clue how i could insert value historians after the controller macro to record keyboard strokes (and relaibly loop with interpolation, etc.), please please please let me know. i know about the boolean issue with value historian, but i can't get any action i can use so far...

jersmi's picture
Re: reducing jaggies

pss. creating keypresses with the controller patch in this setup lags substantially when i go to fullscreen. it would appear i have plenty of headroom with fps. any clues?

sorry for adding issues off topic... that's it, i promise! :)

leegrosbauer's picture
Re: reducing jaggies

gtoledo3 wrote:
It won't smooth anything besides the edges of the image, so it's not an all purpose solution by any stretch
Frankly, it's likely to be pretty darned useful for me. Made a clip. Thanks!

toneburst's picture
Re: reducing jaggies- Supersampling

The other method for reducing jagged edges is supersampling. The basic idea is, you render everything inside a Render In Image patch, at 2x the final output size, then connect the RII to a billboard at normal size, and the Graphics card's builtin filtering takes care of the rest. Obviously, there are performance implications for this, and on some GPUs, the final render size may be larger than the maximum supported texture size so it won't work, but I've found it works pretty well for me. I've made a little macro for setting the render size, and including logic to check if the final render size exceeds the capabilities of the machine's GPU. I'll post it, if you're interested.

Incidentally, since the softRect method works by making the edges of the sprite transparent, you might find you have issues with z-fighting.

a|x

leegrosbauer's picture
Re: reducing jaggies- Supersampling

toneburst wrote:
I've made a little macro for setting the render size, and including logic to check if the final render size exceeds the capabilities of the machine's GPU. I'll post it, if you're interested.
That would be great!

jersmi's picture
Re: reducing jaggies- Supersampling

Thanks, toneburst... I'd really appreciate seeing your macro-- the logic to check graphics card capabilities is the part i can't quite imagine here.

The z-fighting suggestion gave me hope-- both primitives overlap on the same z-plane-- but separating them didn't help at all.

Could it be a core image filter issue? i'm actually using your CIBlend Modes clip...

gtoledo3's picture
Re: reducing jaggies- Supersampling

Unfortunately, I don't think the softRect method works with this. Your suggestion is a really good one for this scenario.

I spent a bit of time yesterday throwing a value historian on this.

The concept is this... you put a value historian on the editor, then go into settings, and add the necessary amount of ports. Then you want to take the left/right coming out of your "control" macro, and hook them to the historian, then hook the historian outputs to the luminosity...just like you have now. You also want to hook up the outputs of "animation x position" and the "size"...and then on to the inputs that you have them currently attached to...

Then you don't get any "boolean" exception, etc..

The only problem I notice with using the value historian like this (with key commands), is that it doesn't always seem to catch the "key up/release" motion. I've tried doing some number smoothing via the smooth patch (before the historian, and sometimes with rounding, to make the "on/off or 0/1" signal stay around a bit longer, and that "works" most of the time... but that is probably a bit of a glitch with the historian, because it shouldn't really be necessary.

Soooo, you set it to record/reset false... now you are recording when you start/stop the renderer. (you could also set a stopwatch to start/stop the recording on the patch)

Once you have recorded, you hover over your duration. Then you click on the value historian, cntrl+click and set your timebase to external. Setup an interpolation patch with the exact amount of time that is reflected in your value historian. Set the beginning value to 0, the end value to whatever the total duration is. Then set the time to that duration, put it on loop/linear. That will keep repeating your value historian values over and over again.

I would have posted an example... but again, I was noticing that my "key releases" weren't getting recorded properly, and didn't want to get into breaking out smooth patches/rounds to see if it would help...

toneburst's picture
Re: reducing jaggies- Supersampling

jersmi wrote:
Thanks, toneburst... I'd really appreciate seeing your macro-- the logic to check graphics card capabilities is the part i can't quite imagine here.

I've attached the macro below. The logic isn't too complex- it can all be done with a single Math Expression patch

(max(Width * 2, Height * 2) < MaxTex && Toggle == true) ? 2 : 1

Inputs Width and Height are self-explanatory MaxTex is maximum texture-size supported by your Mac's GPU, as reported by the OpenGL Info patch Toggle is a switch to turn supersamping on/off. The output is either 2 (if SS is on and dimensions are within usable limits) or 1. This is multiplied by the Rendering Destination pixel Width and Height, and fed into the inputs of the RII patch.

Quote:
The z-fighting suggestion gave me hope-- both primitives overlap on the same z-plane-- but separating them didn't help at all.

Maybe I should have explained what 'z-fighting' is. Apologies if you already know, but it happens when a transparent part of an image still occludes items behind it. There's a Kineme patch (part of GL Tools, I think), that helps work-around the problem, I think, but the correct way to deal with it is to ensure that sprites are drawn in order of distance from the viewer, with furthest-away drawn first, and nearest last. This isn't always easy to achieve though especially when you start to move objects around in relation to each other.

Quote:
Could it be a core image filter issue? i'm actually using your CIBlend Modes clip...

The CIBlend Modes clip just packages together all the standard CoreImage methods for compositing two 2D images together. There are separate Quartz Composer patches for all of them, I just wanted a way of quickly switching between different blend-modes, without having to re-connect.

Hope this helps,

a|x

PreviewAttachmentSize
tb Supersample Dimensions (Clip).qtz3.87 KB

jersmi's picture
Re: reducing jaggies- Supersampling

Thanks a lot, tb, for the macro. i'll have a look and i appreciate the explanation.

And gt, re: value historian, i had it set up exactly like you describe, but i'm getting wacky durations from records-- record for ~10 sec, duration says ~600 sec, things like this. as well i get behavior from interpolation driving the vh that makes no sense to me numerically. also, i guess vh timebase cannot be kept at external when recording?

if it's working for you, i can't see what i'm doing wrong.

my concept of the vh-- correct me if i'm wrong-- is that on record, the patch writes 0 to n-1 samples to a buffer, then play stops record and reads samples back from the buffer.

i was optimistic and set up a vh patch with inputs/outputs for all qwerty letters. not working yet...

my probably over-simplified concept of writing to a buffer comes from how i understand the buffers in my audio app of choice, plogue bidule. maybe i should look into making a buffer patch in bidule to record QC keyboard data via OSC then send back to QC...

jersmi's picture
Re: reducing jaggies- Supersampling

turns out the color bleed problem was something really specific-- i'm using the spot color patch: the "closeness" setting was set <.5.

sorry, but not too cuz this is all valuable stuff, and besides, it still doesn't address fixing antialiasing with the gl triangle.

gtoledo3's picture
Re: reducing jaggies- Supersampling

I'm having problems with the VH not acting correctly as well. It seems like it didn't have this problem, or maybe I never did anything that made it exhibit it before? I will doublecheck that I have the latest version... It doesn't seem to be recording completely accurately. I'm not having the problems with time duration like you are describing though. I'm having problems with ... not 100% sure!

I wanted to strip the basic concept of shutting renderers off + smooths down to basics, so I was "whipping" up this qtz... which doesn't work on replay correctly. I have liked doing it like this in the past, because it allows for some "decay" time... but it no workee now. I wonder if it worked correctly in one of the earlier versions?

PreviewAttachmentSize
vh prob.qtz4.12 KB

gtoledo3's picture
Re: reducing jaggies- Supersampling

Hmm...

Actually, I semi-redact this!

I am starting to think about how I have set this up, and it's not the best. However, I'm still not entirely sure why it works when I'm recording, and not the same on playback.