Free Warping Points ?

mfreakz's picture

I would like to build a warping/distortion tool for pictures. The idea is to set a point in an image (for exemple: the face of a character) an to warp different X,Y,Z for this point. For the begining i use a set of "Kineme GL Grid Generator" linked to a "Kineme GL Grid Renderer". I plan to manage with a structure maker to create an animation of warping points... The matter is that Grid generator use a defined numbers of lines and columns. I would like to use preset distortion for different image (character faces) using mouse to point manually each point to warp. I need to build a generic tool to warp each kind of images/character faces, so i need to place the warping point with a mouse patch for exemple.

On an other hand, i see another exemple of warping technics using a "GLSL Shader" and a DisplacementMap Movie input, there is an exemple of GLSL_Movie_Displace here: http://kineme.net/WWDC2008#comment-2152 Could you help me to build such a patch ? Is there another technic to do that. I would like to use a mouse to set one or many points and to warp them. Warping settings will be the same, but point position should be free and configurable. Any idea ?

psonice's picture
Mesh warping

It sounds like you need to warp the actual mesh rather than just a texture if you're doing it in z also. Warping the mesh is very easy, you just need a GLSL grid + GLSL shader, but mouse control will be a lot more difficult I think.

Off the top of my head, I would say the best way to do it would be to have a number of points set by the user, each point will have a start point (where the user puts the dot, say by the nose) and some distortion parameters (at least how far on x,y,z it is moved, perhaps you will want scale and rotation x/y/z too?)

That will give the basic data you need to distort the mesh, you will then have to pass that into the vertex shader (I'm sure there's a limit on how many parameters you can pass into the shader, but possibly it will run in software if there are too many - it will be slower, but maybe acceptable).

In the shader, for each vertex you would have to run through the list of points, determine the distance of the vertex from the point, and move the vertex depending on the point translation + rotation. You'd presumably have some kind of strength falloff for the point, so it affects vertices further away much less than ones that are close (otherwise moving one point will move the whole grid instead of a small area). You'd want a bell curve or similar for the falloff to make it work smoothly.

I can imagine it being a really horrible shader to write, so good luck :)

mfreakz's picture
Set point without a Grid ?

Well, i would like to make a kind of warping/distortion only on 2D images. X and Y are the only value that i want to change. I would like to make picture 2D distortion like in After Effect, but without a defined Grid.

My first try was using the couple: "Kineme GL Grid Generator" linked to a "Kineme GL Grid Renderer". It is basicly the tool i need. The matter is that i just can change value for one fixed point, or change the total number of points (by changing grid settings). I wonder if there is a way to "pick manually" a point in the picture, without using a grid. Exemple: 1) Load a picture (the face of a character) 2) Set a point manually (with the mousse) on the chin of the character picture. 3) Be able to change/warp that point only... 4) Be able to set a defined number of points for fast face animation (One point on the chin, two other on the eyebrow, etc...)

Picture could change, but user can set a fixed and named number of useful points on the character face to animate/warp them.

psonice's picture
Much easier

You don't need a grid at all in this case, you can do it in a pixel shader or CI filter.

Just use a list of points for the distortion, and you'll need to know whether the point is active (when the user places it on the chin, it becomes active and distorts), the original position (i.e. the coordinates for that place on the chin) and the distortion info (how far the point has been moved or rotated or whatever you want to do).

In the shader, for each pixel you run through the list of points, and do pretty much what I said before but distort the sample coords. Calculate the distortion strength by working out how far away the original position of the point is, and then transform the coordinates by whatever the distortion is.

Actually, that still sounds pretty hard ;)

Michael Neely's picture
Free Warping Points

This would be very cool, especially if we could control the point locations via an animated value from an LFO, Interpolate, or Timelines patch. I like the idea!

The ability to click and add points in the parameter view to a thumbnail of the image would be cool as well.

mfreakz's picture
Basic Exemple ?

Well, i don't know much about Pixel Shader or CI Filter. Do you have a .qtz exemple to use those kind of patches ? I could learn more if i've got a very simple exemple, even if it isn't focused on this particular topic.

Pixel shader needs to code function or/and inputs/outputs ? If you could spare me a moment, could you help me with a very basic composition (i could try starting and increasing the concept alone). Thanx for all.

psonice's picture
No example

I've not tried this, at least this way. Thinking about it more, what I suggested was a bit wrong anyway, and wouldn't work all that well. The problem is that it will only take a small region, and move it by a certain amount in one direction. It would work to some extent, but I don't think you'd get that "smearing" effect where you can push the image around, like the old power goo app. It might still be enough though.

I guess it would work better using a line instead of just a final point position, and working out the distance from the line - then you could move the points around a lot more freely. There's a developer example that shows how to work out distance from a line with fall off - look in the CI filters examples, it's called AALine or something like that.

I have actually done something similar for my paint app, where it's possible to smear the picture around freely. It wouldn't work with animation though, and that part's not finished. It's (hopefully, one day, when I have some time) commercial work though, and way more complicated that what you'd need, so I can't really share that sorry.

mfreakz's picture
Another idea ?

Do you think there something to do with the "Kineme 3D parametric surface" instead of GLSL Grid Patch ? Could it help for choosing point position ?

initi's picture
Re: Free Warping Points ?

woud be really nice to have like this java aplet http://www.dt.org/html/meshwarp.html means the same?

smokris's picture
Re: Free Warping Points ?

You can do this using the GL Grid Generator and GL Grid Renderer provided with GLTools.

Blunder82's picture
Re: Free Warping Points ?

I'm also looking for a way to create my own grid and decide the anchor points of the off the warp should start. Has anyone come up with an idea/solution yet?