Bayer Explorer (Composition by smokris)

Author: smokris
License: MIT
Date: 2008.09.19
Compatibility: 10.5, 10.6
Categories:
Required plugins:
(none)

Inspired in part by psonice's lo-fi analogue processing, I tried doing some lo-fi digital processing: Bayer Dithering, as a CoreImage filter.

The most exciting part is that CoreImage doesn't support Arrays, Data-dependent Conditionals, or the Boolean "&&" operator.. But it's kinda possible to implement it, using some huge Lisp-style expressions.

For example, for a 4x4 Bayer, I'm doing this:

float bayerColor(float colorin,int bx,int by,float BayerIntensity)
{
   float mul =
       by==0 ?
         ( bx==0 ?  1.0/17.0 : ( bx==1 ?  9.0/17.0 : ( bx==2 ?  3.0/17.0 : 11.0/17.0 ) ) ) :
      (by==1 ?
         ( bx==0 ? 13.0/17.0 : ( bx==1 ?  5.0/17.0 : ( bx==2 ? 15.0/17.0 :  7.0/17.0 ) ) ) :
      (by==2 ?
         ( bx==0 ?  4.0/17.0 : ( bx==1 ? 12.0/17.0 : ( bx==2 ?  2.0/17.0 : 10.0/17.0 ) ) ) :
         ( bx==0 ? 16.0/17.0 : ( bx==1 ?  8.0/17.0 : ( bx==2 ? 14.0/17.0 :  6.0/17.0 ) ) )
         ));
 
   return colorin * (mul * 2.0 * BayerIntensity + (1.0 - BayerIntensity));
}
 
kernel vec4 bayerPixel(sampler Image,float BayerIntensity)
{
   vec4 pixel=sample(Image, samplerCoord(Image));
   int bx=mod(samplerCoord(Image).x,4.0);
   int by=mod(samplerCoord(Image).y,4.0);
 
   pixel.r = bayerColor(pixel.r,bx,by,BayerIntensity);
   pixel.g = bayerColor(pixel.g,bx,by,BayerIntensity);
   pixel.b = bayerColor(pixel.b,bx,by,BayerIntensity);
   return pixel;
}

Challenge for the reader: figure out a better way to do this. :^)

PreviewAttachmentSize
BayerExplorer_03.qtz20.91 KB

toneburst's picture
Ah, good old Ternary

Ah, good old Ternary Operator conditionals.... So easy to read...... ;) I don't think you have an alternative if you want to do conditionals in a CIKernel though.

Looks nice, incidentally.

a|x

cwright's picture
opto

toneburst wrote:
I don't think you have an alternative if you want to do conditionals in a CIKernel though.

Ahh, come now. that's silly, plain and simple. [edit: should have read: "there are ways around this in this case"]

Mod is your friend. and int's (in CoreImage) are your enemy.

the 2x2 can be written like this:

float bayerColor(float colorin, float bx, float by, float BayerIntensity)
{
   float mul = (1.0 + mod(2.*mod(bx,2.0) + 3.*mod(by,2.), 4.0))/5.0;
 
   return colorin * (mul * 2.0 * BayerIntensity + (1.0 - BayerIntensity));
}
 
kernel vec4 bayerPixel(sampler Image,float BayerIntensity)
{
   vec4 pixel=sample(Image, samplerCoord(Image));
   float bx = samplerCoord(Image).x;//mod(samplerCoord(Image).x,2.0);
   float by = samplerCoord(Image).y;//mod(samplerCoord(Image).y,2.0);
 
   pixel.r = bayerColor(pixel.r,bx,by,BayerIntensity);
   pixel.g = bayerColor(pixel.g,bx,by,BayerIntensity);
   pixel.b = bayerColor(pixel.b,bx,by,BayerIntensity);
   return pixel;
}

If you modify the ordering on the higher numbered ones, there are similar breakdowns for those as well (I'm leaving that as an exercise for the reader). No conditionals needed, just mod and the right coefficients.

toneburst's picture
I stand corrected. Probably,

I stand corrected. Probably, though I actually have no idea what you're talking about... ;)

Non-programmer here, remember....

a|x

cwright's picture
both

we're both right.

You are correct, in that conditionals in CI are ugly, and there's really nothing anyone can do about it.

I was showing how, in this particular case, Conditionals aren't actually necessary.

I'm not enough of a math/provable software nerd to know how scalable such ideals (non-branching code) are though...

Jens Groh's picture
smokris wrote:The most

smokris wrote:
The most exciting part is that CoreImage doesn't support [...] Data-dependent Conditionals [...]
Not true. Use the compare() function.

cwright's picture
nice

compare()'s a bit tricky to use (since it only compares against zero, not for/against equality), but that might clean this up a bit too.

from the docs:

genType compare (genType x, genType y, genType z)

For each component, returns x < 0 ? y : z. Note that genType is a placeholder for an arbitrary vector type.

smokris's picture
Data-dependent *Block* Conditionals

Jens Groh wrote:
Not true. Use the compare() function.

...but that's still an expression --- just an alternative to the ternary operator. I was complaining about the inability to use curly braces, which would have made the code a little more legible.

(I should have said "Data-dependent Block Conditionals" in my original post.)

Jens Groh's picture
ternary operator vs. curly braces

smokris wrote:
[...] the inability to use curly braces, which would have made the code a little more legible.

You're right.

By the way, I find it totally misleading that the syntax check does not complain when I use the ?: operator for normal variables. ?: is never marked as an error, but it actually works only with "non-pixel-dependent", that is, 'uniform' qualified variables. (Correct?)

cwright's picture
uniforms

uniforms are a GLSL thing, not so much CoreImage (though coreimage translates down into GLSL I think, unless the filter is too complex)

you can do pixel-dependent ternaries, like this:

kernel vec4 brightpassFilter(sampler image)
{
   vec4 color = sample(image, samplerCoord(image));
   color.a = color.r + color.g + color.b > 0.125? 1.0 : 0.0;
   return color;
}

works as expected (is this what you were meaning? or some other usage?)

Jens Groh's picture
works as expected

(Hmmm, I thought the custom CoreImage filters we can create in QC are always programmed in GLSLang, aren't they?)

Strange, I thought I had tried just that and it didn't work. Must have been a different problem then (maybe the clamping thing I stumbled upon recently).

Thanks for clarifying this!

toneburst's picture
CoreImage and GLSL

Quote:
uniforms are a GLSL thing, not so much CoreImage (though coreimage translates down into GLSL I think, unless the filter is too complex)

It's more the other way around- there's stuff you can do in GLSL that can't be done in a CoreImage Kernel. The CoreImage Kernel language is a subset of GLSL (ie GLSL with some of the functionality stripped out).

Having said that, it's still pretty powerful, especially when you take into account the JavaScript wrapper functionality that came in with QC 3.x, which allows you to pass the same image through a chain of custom Kernel functions, apply any Image Unit filters you might have installed on your system, resize, crop and rotate images and use JavaScript to process controls values etc.

I'd be interested to know which runs faster, generally-speaking. I tend to use CIFilters for simpler 2D image manipulation, because I don't then have to use a Render In Image if I want to pass the result into another patch for further processing.

I'd really love to see something similar added to the GLSL patch, so you could construct multi-pass GLSL shader effects in the same way, but I'm not sure this is technically possible.

a|x

toneburst's picture
Would you mind if

I wrapped this up into a qcFX for VDMX? I think it would work quite nicely as a VJ effect. I'd make sure you were fully accredited as the author, of course.

a|x http://machinesdontcare.wordpress.com

smokris's picture
Not at all.

Sure, that would be awesome.

The initial composition I posted is pretty inefficient (especially in 8x8 mode), so I'd advise giving cwright's suggestion of dropping the ints and using mod a try.

toneburst's picture
Cheers!

I'll give that a go. Not sure how to do the 4x4 and 8x8 ones, but I'll have a go.

a|x

psonice's picture
Mmmm dithering

I love proper old style ordered dithering for some reason. And as it happens, I have a CI filter that converts stuff to a c64 palette... I'm thinking that and this would go together like quartz and pixels =)

I'll dig it out sometime soon and post up a composition.

Also, good to hear I've inspired somebody :)

vade's picture
great

I was doing some work on making LUTs that matched the older gaming systems. This dithering is fucking awesome. Ive just finished porting part of a older gaming emulator core to a QC plugin, so right now NES games (and soon others) run within QC. :)

Fun. The two would work really well together.

gtoledo3's picture
Re: Bayer Explorer (Composition by smokris)

Wow, this is cool. I missed this when it was posted.

mradcliffe's picture
Re: Bayer Explorer (Composition by smokris)

Test Comment.

weevil's picture
Re: Bayer Explorer (Composition by smokris)

I'm desperate for an atkinson-style core image filter.