Tiltshift Plugin for AE, Final cut & Motion

gnomalab's picture

I'm trying to create a plugin effect Tiltshift "with FxFactory to use in Aftter Effects and Final cut. The filter is quite close to the result I want. but I do not tweak the lens blur Fx characteristic of Tiltshift. There are many tutorials for this FX in PS or AE, but are based on a combination of filters and layers to get the result. Examples: http://www.tuaw.com/2008/10/07/how-to-use-photoshops-lens-blur-tool-for-... or http://luipermom.wordpress.com/2008/12/29/tutorial-tilt-shift-con-photos... (in Spanish

My intention is to create a plugin all in one. Launch the plug and go. My first test was using Gaussian Blur, worse today and made a change by Box Blur and gives much better results. But I think it would be much more successful getting a lens blur fx. Here is a demo of my plugin running in AE: I searched but did not and managed to find an appropriate combination using NI blurs, or VADE blur plugins.

my question is, how I can create a filter with the properties of lens blur?

PreviewAttachmentSize
GN tiltShift whith Box lbur.png
GN tiltShift whith Box lbur.png468.65 KB

toneburst's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

gnomalab wrote:
my question is, how I can create a filter with the properties of lens blur?

Looks good :)

I don't mean to be discouraging, but surely the problem is that you can't create truly realistic-looking lens blur without somehow extracting the depth information from the scene. I know there are plugins and applications that attempt to do this, but it's pretty hardcore coding, I think.

Having said all that, with certain scenes, your approach works really well.

Incidentally, I first saw this effect on this video

I remember reading about it at the time, and I seem to recall there was a lot of masking involved. Looks great though!

a|x

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

cwright posted an interesting DoF filter example somewhere around here.

There's a lens blur plugin floating around, but it's ultra slow.

toneburst's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

gtoledo3 wrote:
cwright posted an interesting DoF filter example somewhere around here.

That was for 3D geometry though. Wouldn't work on 2D video.

a|x

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Mmm, my mistake for not fully checking out the video in the original post.

usefuldesign.au's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

toneburst wrote:
gnomalab wrote:
my question is, how I can create a filter with the properties of lens blur?

I remember reading about it at the time, and I seem to recall there was a lot of masking involved. Looks great though!

Z-maps are used to generate the mask in cases I have seen. So if camera is moving a lot relative to geometry of scene you need to be able to animate the Z-map and therefore the mask, which is easy enough in a 3D app but is a long round trip just to make a mask.

psonice's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Like smokris said, you need a bokeh effect rather than a normal blur. QC has one built in - it's a private patch called something like 'disc blur' (and it's slow compared to normal blurs, but hey :)

If you want to create it yourself, the filter isn't too complex. It's a bit like a gaussian blur filter, but you sample each pixel within the area of a circle and weight them equally. You won't get a really good bokeh effect with the white circles around small light areas though unless you have HDR images/video, because with a small dynamic range once the small white area is spread out it ends up grey.. you could fake this I guess though.

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

For his time the result is pretty good if you compare with other more complex animation presets in AE. Obviously, the type of scene is most important to function as tiltshit. There should be no vertical elements of the landscape ahead. if not, is just a simple blur mask.

Here is a comparison of fx done with animation presets and masks in AE and the other with the latest version of tiltshift Gen. Now works best e smucho easier to implement than a long chain of layers and filters. Perhaps to be more realistic is necessary to improve Blur Focal distance. I'll see if I find the DOF filter, would be more similar.

Thanks for your answers.

iaian7's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

You can get surprisingly improved results by simulating a linear light composite - convert using pow() of 2.0 (1.0+offset), compute circular blur, then convert back via pow() of 0.5 (1.0-offset/2.0).

Be sure to clamp image input when using pow(), and keep everything in a single CIfilter to preserve floating point information. I haven't had time to publish my QTZ work in many months, but once I do, it should all be on my blog http://iaian7.com/quartz/

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

yes!, Disc Blur looks much better for this. Disc Blur I used it in the beginning, but I do not know why it ... desice In reality and escigido NI Disc Blur, which has pixel correction and repeat edges.

Thanks for the tip!

iaian7's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Vade did some work late last year in bringing point splatting based blurs to Quartz Composer, not sure if he'll be releasing it or not...

http://abstrakt.vade.info/?p=213

Seems like the best solution for Bokeh simulation, especially if you want custom blur shapes. For now, I'm using linear light compositing (see my other post below).

iaian7's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

It's not as polished as I'd like it, so don't take this as an official release or anything...but since I won't have time to officially post for another couple months, I'm attaching one of my variable blur projects.

Takes an image, mask, and several other variables to compute a high quality variable blur. It can work very well for faking tiltshift photography, but NOT for simulating DOF using a depth mask - without point splatting, image kernel based filters just can't cut it. Believe me, I've tried. :(

Anyway, hope this helps? It includes a simple radial mask, but wouldn't be hard to replace with a linear gradient.

smokris's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Wow. This is beautiful.

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

This IS really nice.

The stock axial blur can be useful in many instances for making it seem like a backround is blurry, and foreground is focused...

PreviewAttachmentSize
CIAxialBlur Unit.qtz313.14 KB

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

here the new result:

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

The latest FX Factory released yesterday has a shit ton of per pixel blurs.

NI Gaussian Blur With Mask

NI Gaussian Blur Channels with Mask, its meant to do exactly what you want.

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Also, respectfully, the v002 blur license agreement specifically prohibits packaging the plugins for a for sale, commercial license. If you are interested in using FXFactory and making some plugins, I'd highly suggest emailing Gabe or myself (gds at noiseindustries.com and anton @ noiseindustries.com )

We actually have in progress some things you will find very useful, and can we tend to make custom plugins for partners.

Here is a work in progress lens blur (hope gabe does not see / mind :)

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

This ended up being a real pain in the ass and I abandoned it. The issue here is that in order to do do this realtime, you need to work on a downsampled mesh to generate the bokeh from.

IE, your input image is 640x480, your mesh, to be realtime, needs to be on the order of 20x20 to 100x100 (not that its a lot of verts, but its the blending, and texturing/fill rate that slows you down).

The issue then becomes since your mesh is downsampled, bokeh pop in and out of existence. You could do tricks like interpolating or blending time slices, but it all ends up being a hack, and is never quite right.

Pixel / Fragment shaders are heavier, but produce much nicer results.

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Thats a nice comp by the way :)

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

uauuu is perfect! DOF don't really needed. Tilt shift is something else. I will work with what you sent me,iaian7. maybe if there is to be replaced by a linear gradient or perhaps include both a Menu: Radila and Linear ... :-)

It really is helping!

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Hi Vade, my intention is not to distribute plugins. I just want something for personal use. Since I have found no fx and decided to do it myself and so glad life. Create this AE process is much slower and you need to use layers, masks, and animation presets ... Although it would be that there was a commercial plugin TiltShift. It would be very useful and fashionable! :-)

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Ah cool, no worries then, sorry I misunderstood :)

Either way, if you have suggestions though, let us know. These variable per pixel blurs are on our list. Right now we have core image based and GL (faster, but limited to 8 bit) versions, all highly optimized (separable, etc). The lens blurs are in progress so stay tuned :)

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

It belongs on a separate thread, but could you consider enlightening us about the geometry shader stuff I've seen that you must be poking around with (via lisp)? Reaaaally curious.

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

perfect! I will be careful ... The next step would be to add handling of time to create fx stopmotion as these examples: http://vimeo.com/videos/search:tilt% 20shift/page: 1/sort: HIGHLIGHTS / format: thumbnail

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

gtoledo3 wrote:
...but could you consider enlightening us about the geometry shader stuff I've seen that you must be poking around with (via lisp)? Reaaaally curious.

Do you mean this sort of thing? http://vimeo.com/9242544

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Probably not. Thats a fun effect though. I did some realtime stuff with luma keys and nice footage for a music video using stuff like that.

The goem shader stuff is nice but is not suitable for QC only because there is no way to use it other than a custom plugin. Its also complicated as fuck. Im not sure where lisp comes in (typo? maybe you meant on the GL list?). We are using it to fix some issues with the Synth plugin, and just to have it if we need it. Originally it was for a bokeh style effect since points can't typically be larger than 64, so geom shader points to quads with textures gets around that.

Whats odd is it works in clipping coordinate space, rather than modelview coordinate space. It makes doing some things a lot more of a pain in the ass. However, its definitely fun to use.

toneburst's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

vade wrote:
These variable per pixel blurs are on our list. Right now we have core image based and GL (faster, but limited to 8 bit) versions, all highly optimized (separable, etc). The lens blurs are in progress so stay tuned :)

I'm intrigued to know why the GLSL version is limited to 8-bit. Is it because of hardware-accelerated filtering not working on 32bit/channel images? If so, you should be able to create your own filtering algorithm in a GLSL shader.

I'm sure you've thought of that though.

a|x

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Yes, simply because core image already accomplishes arbitrary bit depth, and filtering manually will slow it down, plus to be accurate QCPlugins only have output images of 8 or 32 bit, so it means the plugins down the stream would also have to implement their own custom filtering internally if they leverage GL at all. For example, I can filter manually within a custom plugin, but that does not mean a different plugin will be able to do it properly. Core Image gets around this, but not all consumer patches draw in GL using Core Image, so this means filtering on actual rendering is out of your hands.

I made those, kind of, for myself and you guys, for realtime usage. cough They are meant to be fast.

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

I meant paste.lisp.org, but I had seen you comment on it on some Apple list I subscribe to as well.

Synth plugin? If that's in the latest NI batch, I have totally missed it. Real outstanding work, I have to add. There are so many new patches in that one, I haven't had a chance to cover all of them, and the ones that are obviously traditional v002 things tend to have nice extra touches.

It's interesting that you say it's not suitable because it would have to be a custom plugin? I guess you mean a non-standard API plugin (a patch, if you will)?

You originally were employing it for bokeh effects! So, I guess it does come around to being pertinent to this topic. What did you end up doing instead, with the bokeh effect I've seen (I can't remember if it's v002, NI, or both).

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

The link didn't convert successfully...

vade's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Well, it has to be a custom plugin (standard public API), because there is no way to use a geometry shader with the GLSL patch in QC. So you have to manage all of that low level stuff yourself.

Right now, for the Bokeh I am doing something incredibly similar to the QTZ core image plugin posted above. I took a look at it and its kind of amusing how similar it is :)

gtoledo3's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

Oh, ok, I thought it was a given that Geometry Shaders don't work in QC's GLSL patch, and that's why you were making a plugin with it. We're on the same page.

As for the Bokeh and the CI above... I guess "there's only one way to rock".

Please don't actually listen to the song.

dust's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

haha american mtv went reality but now we have youtube.

so i thought tilt and shift where to correct for perspective distortion. like when using medium format camera to film a tall building you would use the tilt to correct the perspective distortions. this also makes it so you can do interesting things with the PoF and DoF like selective focusing which looks to be more of what this plug-in is emulating rather than tilting the perspective. i'm not a photographer so i'm sorry for being daft but wouldn't selective focus sound like a better name than tilt and shift ? i mean you could add a tilt and shift with QC/FX. its really only rotating etc... so again sorry if i'm daft maybe tilt and shift is a term photographers use for selective focusing. in that case disregard what i just said.

gnomalab's picture
Re: Tiltshift Plugin for AE, Final cut & Motion

vade wrote:
...

We actually have in progress some things you will find very useful, and can we tend to make custom plugins for partners.

Here is a work in progress lens blur (hope gabe does not see / mind :)

, Examples Lens Blur aberation look good. seems that the edges do not repeat pixel. another question, Is it possible to simulate a lens aperture? triangular, exagonal etc...