how to change only one color of pixels

hiltmeyer's picture

hello could some one please give me a hint in the right direction. i want to colorize only the white(or a other) color of an image to a different color and leave the other colors as they are in the image. i have no clue how to do this. thanks hiltmeyer

usefuldesign.au's picture
Re: how to change only one color of pixels

If you strictly want to effect the white (r=1,b=1,g=1,a=1) areas of the image +only+, the easiest way is to create a sample of just that part of the image, kind of like you would in Photoshop/GIMP with a colour range or magic wand (tolerance=0) selection.

In QC this is probably best achieved with a threshold patch (Noise Industry have a couple if you have them installed). Otherwise it's easy enough to make your own in a Core Image Filter programming patch. Just compare the colour component of each pixel to white. In my example I've compared to a threshold value [0,1]).

You can then colour the pixel in the same patch by multiplying the pixel by an input color value. In order to have access to the b&w threshold sample as a mask, I used a second CI Filter patch to do the colouring in.

As you can see in my comp the edge is a little dirty, so you may need to refine the filtering. As it totally depends on the source image as too what kind of filtering I haven't gone any further with it.

PreviewAttachmentSize
CI Threshold to color.qtz34.49 KB

usefuldesign.au's picture
Re: how to change only one color of pixels

Just reread your post and read "white(or (an)other) color of an image."

Smoothstep matching to an input color value is probably a way this could be done. No time for me to do it for now though.

jersmi's picture
Re: how to change only one color of pixels

This website has posted a really nice chroma key CI filter.

http://www.daisyrust.com/category/quartz-composer/

cybero's picture
Re: how to change only one color of pixels

nice example for colour threshold though :-)

usefuldesign.au's picture
Re: how to change only one color of pixels

Encouragement, cybero! I find the GLSL language quick-guide PDF a tremendous help ;-) I'm very much the novice at CIFilters so far.

Re: smooth stepping in the previous post — on the the alpha value not the color.rgb values. Still no time to try it…

dust's picture
Re: how to change only one color of pixels

you can do this pretty easy with a cl kernel. basically you would want to read the image into the kernel then check for equivalence based on what ever threshold you set meaning. it would be something like color = the image source and pos. just look at standard default kernel it shows you how to read an image in. then you would want to either get a pixel average or check pixel by pixel for each color. meaning for rgb separately.

I think actully an average mean would be faster. thats basically just saying if (((color.r + color.g + color.b)/3))==threshold then color = new color. where as the threshold is the point in which you set the color white to be within. now you can do this in cl with live video input, just remember to resize the video Input to something really small and you will relatively have realtime latency free color replacement going on. If you do this sort of thing based on a frame differential. meaning testing for the pixel white when comparing a before and after frame image together then using the white average to output the position of said motionor what ever. kind of like how optical flow works but without all the really fancy math.

when I get to my system I can post you an example

hiltmeyer's picture
Re: how to change only one color of pixels

thanks for all the info. dust it would be really nice to get a example of the code. i have no clue about programming kernels and stuff but i would love to learn it. i think this would be the first step into it.

usefuldesign.au's picture
Re: how to change only one color of pixels

Either mean or product can do it, doubt the speed is going to get effected much since there's some CI Filter overhead that is the biggest hump —but I'm not experienced with CI Filter — build and profile, as always.

To do the colour replacement operation neither of those threshold methods is going to do it. Need to compare pixel and key colour by finding the difference for each colour component and testing if it's within whatever tolerance range you set.

eg. (pseudo code) ( abs(px.r - key.r) < threshold )? replace_colour.r : 0.0 where abs (absolute) multiplies arg by -1 if arg <0

You could just do this using the step function which automatically assigns 1.0 or 0.0 depending on comparison of two arguments. Then multiply the answer by the replacement colour.

ie step(edge, x) ; 0.0 if x < edge, else 1.0

so for eg.

pixel.r = step(tolerance, abs(px.r - key.r) < threshold) *replace_with_color.r;

You can combine single operations into vector operations where each component of an vec3 or vec4 gets the same function applied. You need to define the result and 'x' as the same dimension of vector. 'step' can be scalar (a floating point number) or a vector of same dimension.

So in the example, you'd have to make tolerance a vec4 to use different tolerances for each channel of the RGBA color.

eg vec4 tolerence = (.05, .01, .025, 0.0)

Check the chroma key CI Filter jersmi linked to for more detail. It has to be doing this kind of testing to filter a keying colour.

Hope that helps. CI language takes a while to get comfortable with since it's defining an operation that operates on every pixel, it's kind of back-to-front for me because I'm used to more procedural/object coding and it doesn't have much room for any kind of conditional logic. Step and smoothstep are as good as it gets from what I can tell.

gtoledo3's picture
Re: how to change only one color of pixels

This is a qtz, based on stock Apple example, on setting up something like you're talking about. I've changed it so that it uses video source, and added the constant color patch so that you can choose what color you want to substitute in.

What is critical is that you make sure that your background image is the same size as the foreground image. In this example, a crop patch handles that. Another approach might be to use a Resize patch, if you were keying in imagery, and not color.

When you use this, you are going to see that adjusting threshold and smoothing makes all of the difference in the world.

I set the qtz up for a little different color sub than in the picture. For me, it turns skin tone into a "incredible hulk"-like hue.

PreviewAttachmentSize
ChromaKey_gt.qtz11.57 KB
ChromaKey.png
ChromaKey.png492.1 KB

gtoledo3's picture
Re: how to change only one color of pixels

Geez...

There is the thing of "if you don't have anything positive to say, don't say anything at all". I really wish I could do that all the time! This gets the job done. However, I will say that it's stuff like this that made learning QC confusing, because there are many choices that aren't the most direct way to achieve the result desired. Even if one needed to add in a shape mask as well, this isn't a great way of doing it. (Resizing the window makes everything go to crap.)

The apple developer examples are full of wonderful info, and there are very few questions that I ever see asked that aren't covered in some stock Apple example, and they rarely feature questionable construction.

dust's picture
Re: how to change only one color of pixels

well this is the first step......

CL

__kernel void main(__rd image2d_t Image, __global float *mpa)
{
 
   int2   pos = (int2)(get_global_id(0), get_global_id(1));
   float4   color = read_imagef(Image, CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_NEAREST, pos);
   float monochrome = (color[0] + color[1] + color[2]) / 3;
   int tid = get_global_id(0) + get_global_id(1) * get_global_size(0);
   mpa[tid] = monochrome;
}

now just for fun this is how you would parse the color data.

javascript

function (__number Average) main (__structure MonochromeStructure)
{   
   if (MonochromeStructure) {
      var result = new Object();
      var Sum = 0;
      for (i in MonochromeStructure) Sum += MonochromeStructure[i];
      result.Average = Sum / MonochromeStructure.length
      return result;
}
}

this is just simple example of the first step. my patch isn't set to replace color so this doesn't do any replacement but thats easy enough once you understand how kernels work.

i was afraid of them for a while don't know why. kind of silly if you ask me, it just seemed so foreign to me all these vectors functions and stuff. now that i understand things a bit better ci, cl, glsl are all very similar so once you understand one of them the others kind of just fall in place.

to make this work smoothly feed a 10x10 px image into cl kernel then feed cl kernel into the js. the output of the js will give your average mean pixel that can be used to replace.

maybe its best to this all in a cl or ci but sometimes i like mixing all of them.

PreviewAttachmentSize
pixelS.qtz8.47 KB

gtoledo3's picture
Re: how to change only one color of pixels

I'm going to be really interested in seeing what is up with this one, because the approach is a totally out of the typical as far as chromakey stuff goes, but makes sense from a pixel test standpoint (vaguely, without looking at the composition). I'm curious to see how you work doing a 10x10 pixel test into being able to cut out all of a given color in an image, and then how the image is "rebuilt", so to speak (this is ambiguous to me from the post). It's killing me that my SL partition just went fubar and it will be a couple hours before I'm able to see it (at best).

If this is slower than the CI method, but somehow more accurate, it would be really cool. If it's faster AND more accurate, then it's very cool, because it's sort of an entirely different approach to chromakey than is typical. Cool line of thought.

dust's picture
Re: how to change only one color of pixels

so yeah this works really well with motion detection. as i have been using this set up as a sampler trigger for fluid simulations.

if you put this kernel inside a render in image with pix set to 10x10 then use it as the input to the above cl kernel you will be surprised how fast it is. it can also be even faster if the js stuff is negated and done in cl as well.

cl

__kernel void ImageMonochromePixels(__rd image2d_t Image, __rd image2d_t pImage, float treshold, __wr image2d_t dstimg)
{            
   int2   pos = (int2)(get_global_id(0), get_global_id(1));
   float4   Color = read_imagef(Image, CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_NEAREST, pos);
   float4   pColor = read_imagef(pImage, CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_NEAREST, pos);
   float monochrome = (Color[0] - pColor[0] + Color[1] - pColor[1] + Color[2] - pColor[2]) / 3;
   float4 mcolor = (monochrome<treshold)? (float4)(0, 0, 0, 1) : (float4)(monochrome, monochrome, monochrome, 1);
   write_imagef(dstimg, pos, mcolor * 100.);
}

image is your first frame and pimage is your second frame. (use queue or sample and hold to get a before after and frame) set threshold as you choose.

this is basically the same kernel as above but the output is an image and not structure...

then just attach a conditional to js output checking for white and you have a very fast motion detection system. i found using a smooth with default settings attached to the output of the conditional seems to work best.

alternatively you could do the motion detection in a ci kernel like this...

ci

kernel vec4 Difference(sampler Image, sampler pImage, float Treshold)
{
   vec4 d =     sample(Image, samplerCoord(Image)) - 
            sample(pImage, samplerCoord(pImage)) + 
            vec4 (-Treshold, -Treshold, -Treshold, 1.0);
   return clamp(d * 100.0, 0.0, 1.0);
 
}
[code/]
 
just remember 10x10 ;)
 
 
 
 
 
 

usefuldesign.au's picture
Re: how to change only one color of pixels

This example doesn't actually key white (as asked for in OP). It does a nice job of keying the blue in the sample image I used in the comp I posted though. Since there is a conversion to YUV? colourspace and I'm not a video engineer so I can't comment as to why white isn't usable as keying colour (or how to modify the code to do it in that colourspace).

Also there's no need to generate and import an image to set a replacement colour. Just input a colour, and the dimensioning issue go away like magic.

Of course you could use the input background image for a texture to key in something more interesting than a colour but wasn't asked for in OP. Staying positive I wont go into a rant about an irrelevant solution ;-)

Actually the way I posted in my original comp gives the advantage of outputting a mask, so the replacement colour image can be further modified with stock CIFilters and rendered over the background image on a second billboard. I still suspect the most satisfactory way will be to smoothstep the alpha on the edges though. The Apple/gt comp smoothsteps the blend variable which is effectively smooth-stepping the alpha (in-house) to get the nice edges.

PreviewAttachmentSize
ChromaKey_ud.qtz39.56 KB

usefuldesign.au's picture
Re: how to change only one color of pixels

gtoledo3 wrote:
There is the thing of "if you don't have anything positive to say, don't say anything at all". I really wish I could do that all the time! This gets the job done. However, I will say that it's stuff like this that made learning QC confusing, because there are many choices that aren't the most direct way to achieve the result desired…

I think the "If you can't say something nice (or nicely)… " is a good axiom (my Nana used to say that), but like anything it can get taken a bit too far at times. Having said that I've made some notes on the comp you posted.

I think all the sample code posted in this thread will help a novice to see more code around the same problem and will help them understand the CIFilter language basics which can't be bad. Not sure which comp you don't like, is it the one on daisyrust.com?

gtoledo3 wrote:
The apple developer examples are full of wonderful info, and there are very few questions that I ever see asked that aren't covered in some stock Apple example, and they rarely feature questionable construction.
Yes they are a great start, wish there are more of them! How do you remember them though, I've forgotten them after a year? Could you link to the comp you're referring to I can't find it in the ones I've downloaded.

gtoledo3's picture
Re: how to change only one color of pixels

You know what, I think we are/were talking about two totally different things now that I see where you're going with it. I think that my thought on what the original post was about is different than yours. I definitely may have misread what it was asking.