These are real-time versions of some time smear effects. On a brand new MBP they run best at around 24 iterations. If anyone has any suggestions to improve performance and get more iterations, I would love to hear them.
Try using Image Texturing properties to keep it all stock while gaining performance. This is more efficient than using Crop in this instance, and only requires a slightly different approach with your setup.
Check the attachment. I get a sizable performance boost on my system (if I use an FPS patch meter to check average, it averages 10FPS with crop, and 17fps with Image Texturing Properties, at the default composition opening size of 1440x828.
At higher iteration/slice counts, the discrepancy becomes more pronounced. At 64 iterations with Image Texturing Properties, I'm still getting 14FPS, while the Crop method obtains a little over 6FPS.
Well remembered - a video patch works really nicely, but would it really reproduce the desired result as per dwskau's Vertical & Horizontal Smear though ?
I can get it pretty close, but not the same , though that's just on the basis of a 5 minute fling.
I especially like the way you can use a different delay map and work wonders with the delay factor and sampling range. Smooth.
Image Texturing Properties , probably one of the most under used and yet very useful patches that comes as standard with QC.,
I think it would probably be possible to exactly reproduce the dwskau's delay 'smear' effect achieved with a high degree of success and it is actually pretty amazing how well it improves processing performance.
[Just taken a look at your comp and boy the difference that patch makes, pretty incredible.]
...but I'm still waiting to see a video version with decent framerate + quality (I vaguely remember getting 8 seconds/frame for 640x480, 1 slit per pixel!)
...but I'm still waiting to see a video version with decent framerate + quality (I vaguely remember getting 8 seconds/frame for 640x480, 1 slit per pixel!)
Yeah. Try something like that comp the OP posted. Now try it with 480 iterations (for vertical, this is best case, horizontal needs 640). It's still not entirely correct, as the comp should scan through both time and pixels for the best effect, which is even worse..
This is ASD's lifeforce demo converted to slitscan with QC, it shows what I'm talking about:
No time to check just now, but I'll have a good look at that when I get time :)
I'm guessing though, that you're not using all 640 buffers in every frame? That's what I needed to do - which means copying 640x640x480x4 bytes to the GPU every frame. Unless you have a 1GB VRAM card, it just doesn't work :( (and actually that much data moving across the bug can kill performance for the whole OS!)
I think with your plugin, plus the texture properties patch instead of crop, plus lower res it might get towards reasonable framerate.. I'll have to try it :)
Scanning through time + pixels: what I mean is this..
For each row of the image, you take a slice through time. This slice is also a slice through the image at that time.
As you go down (or across) the screen, you scan through time, taking a slice from different points in time.
Because you're taking the same physical slice from each buffer, you're only using a small part of the image. You can vary which part of the image you look at, so you're scanning across the image too (you take the same row or column from every buffer during the frame, then move the position for the next frame).
That's how I did the animated slitscanning in that video (not my idea though, I stole it from somewhere else :)
Ok, I'll give it a proper look when I get time. I'm still very much doubting it's pushing 640 buffers to the gpu each frame though - at 640x480 that's 0.75GB per frame! I've done that before, and it absolutely kills the framerate (and the OS if you're not lucky!)
Yeah....! I first saw you post that one on VDMX of VIDVOX a long time back. I was shocked when I saw how fast it worked. I love how you can feed it different delay maps as well. It makes it really easy to shift stuff around like a jigsaw puzzle.
The textures have to be pushed into vram to be used in QC though, no? And that means pushing them across the data bus. And pushing too much data across the bus kills pretty much everything :(
Anyway, let me have a play with it and see for myself :)
OK, I've had a play with it and I'm with you now. It's not actually providing access to the buffers beyond selecting from frames using the map. So a 640-frame buffer is no issue.
Its... well, cool. But it can't actually do what I was discussing :) Still, it's excellent for basic (and even quite fancy!) slitscan + delay effects. I'll have a lot of fun with that one :D
Some questions + suggestions:
Does it support >8bit colour for the delay map? Having a 640-frame buffer is great, but you can't select >256 of those frames without >256 shades of grey.
Do you plan to extend it at all? If so, I suggest this: take 3 values from the delay map, r = delay amount, gb = pixel coordinate.xy. This way you can do very advanced effects, remixing both spacially and temporally (this is what I was talking about before). I guess coords will have to be remapped from 0..1 to 0..image width. If all images in the buffer are the size size it should still be quite fast.
Can't you just put the source image on a loop (eg., like, a tile that is constantly moving horizontally), to get that "moving across" thing you're talking about? I've done that before, in just a sheer "guess/poking around moment" and it sounds like what you were describing up above (?).
I know you can feed this "checkerboards" of various shades of grey and get distinct regions in the delay map, or move squares (or any shape) around in an RII and have it get map delay to that x/y area.
After having used this for a long time, one thing that would be advantageous, if not standard at all, would be an option to be able to push the first frame across all samples at once. For live stuff, it doesn't really "work" in many scenarios to have to have the "queue" fill up, but it would be a-ok if it looked like a normal image when you flip it on, and then it starts slitscanning. Yeah, you can put a billboard behind or something...
I did a little Finder search and found this composition I made when I first got this plugin. It uses a v002 zoom blur filtered video feed as the delay map. Totally "wrong", but a really psychedelic effect.
Hmm.. panning the source would get exactly half-way to the effect I'm talking about. If you take the stack of images as a 3D volume, with normal x+y plus z being time, that would give you a diagonal slice scanning through the volume. The effect I referred to originally would be a slice through y and time, scanning through x, which still isn't possible.. or is it? Damn, this stuff gets confusing!
No, scrap that, I think it IS possible if you can scroll at exactly 1px/frame.
Either way, my suggestion of having R = delay, GB = pixel position has me much more excited. With some cunning all kinds of amazing effects are possible :)
I had a play with it before using a motion detection patch I wrote ages back, so areas with motion in the current frame have no delay and 'calm' areas have long delay. Didn't quite work, but it was cool watching somebody walk past, then walk past again some time later :D
Cool :D Didn't have time to try it earlier (there's only so much that can squeeze into lunch break!)
At present, for each input pixel it takes the buffer index from the input luminance, and the samples the pixel at the same coordinates as the input pixel from that buffer image, yes? I'm suggesting a variation where the sample coordinates can be changed, using the green/blue values from the delay map.
It would need a CI filter or whatever to create gradients in g + b channels to achieve a 'normal' result (like what you have now), but it'd be amazingly powerful! You could remix not just time, but time and space :D
This would have to be a separate plugin I suspect, since it would be a whole different beast and quite hard to understand perhaps :)
** edit: I'm happy to try this myself if you'd be willing to share source? **
http://kriss.cx/tom/video-delay/ by Bangnoise
Try using Image Texturing properties to keep it all stock while gaining performance. This is more efficient than using Crop in this instance, and only requires a slightly different approach with your setup.
Check the attachment. I get a sizable performance boost on my system (if I use an FPS patch meter to check average, it averages 10FPS with crop, and 17fps with Image Texturing Properties, at the default composition opening size of 1440x828.
At higher iteration/slice counts, the discrepancy becomes more pronounced. At 64 iterations with Image Texturing Properties, I'm still getting 14FPS, while the Crop method obtains a little over 6FPS.
Well remembered - a video patch works really nicely, but would it really reproduce the desired result as per dwskau's Vertical & Horizontal Smear though ?
I can get it pretty close, but not the same , though that's just on the basis of a 5 minute fling.
I especially like the way you can use a different delay map and work wonders with the delay factor and sampling range. Smooth.
Image Texturing Properties , probably one of the most under used and yet very useful patches that comes as standard with QC.,
I think it would probably be possible to exactly reproduce the dwskau's delay 'smear' effect achieved with a high degree of success and it is actually pretty amazing how well it improves processing performance.
[Just taken a look at your comp and boy the difference that patch makes, pretty incredible.]
nice effect great optimization george i get double fps
I get well over double, I can get 60 fps at 96 iterations! Awesome optimization, thanks!
Coming to this a bit late...
http://www.memo.tv/timestretching_in_quartz_composer
Hopefully you're familiar with
http://www.flong.com/texts/lists/slit_scan/
slitscanning <3
...but I'm still waiting to see a video version with decent framerate + quality (I vaguely remember getting 8 seconds/frame for 640x480, 1 slit per pixel!)
? Have you read this page?
Yeah. Try something like that comp the OP posted. Now try it with 480 iterations (for vertical, this is best case, horizontal needs 640). It's still not entirely correct, as the comp should scan through both time and pixels for the best effect, which is even worse..
This is ASD's lifeforce demo converted to slitscan with QC, it shows what I'm talking about:
Nice video.
I agree that the OP's method is doomed, try my plugin linked by the first respondent. Composition with 640 frame buffer attached.
Can you elaborate on scanning through both time and pixels?
cheers, T
your plugin is damn great, and blazingly fast ... are you using a texture rep. or a buffer rep. ?
thanks - buffers, 640 (or however many you want) frames eat way too much memory to keep them on the GPU.
No time to check just now, but I'll have a good look at that when I get time :)
I'm guessing though, that you're not using all 640 buffers in every frame? That's what I needed to do - which means copying 640x640x480x4 bytes to the GPU every frame. Unless you have a 1GB VRAM card, it just doesn't work :( (and actually that much data moving across the bug can kill performance for the whole OS!)
I think with your plugin, plus the texture properties patch instead of crop, plus lower res it might get towards reasonable framerate.. I'll have to try it :)
Scanning through time + pixels: what I mean is this..
That's how I did the animated slitscanning in that video (not my idea though, I stole it from somewhere else :)
Stop doubting and guessing wrong and just try it ;)
I'm still trying to make sense of your pixels v time...
Ok, I'll give it a proper look when I get time. I'm still very much doubting it's pushing 640 buffers to the gpu each frame though - at 640x480 that's 0.75GB per frame! I've done that before, and it absolutely kills the framerate (and the OS if you're not lucky!)
Yeah....! I first saw you post that one on VDMX of VIDVOX a long time back. I was shocked when I saw how fast it worked. I love how you can feed it different delay maps as well. It makes it really easy to shift stuff around like a jigsaw puzzle.
buffer representation sits on the CPU. texture representation sits on the GPU. Tom said he used buffers..... so that would be ...
http://developer.apple.com/mac/library/documentation/Cocoa/Reference/QCP...
The textures have to be pushed into vram to be used in QC though, no? And that means pushing them across the data bus. And pushing too much data across the bus kills pretty much everything :(
Anyway, let me have a play with it and see for myself :)
Acquire frame -> store in main memory -> calculate new output in main memory -> pass to QC as a buffer. Nobody dies.
OK, I've had a play with it and I'm with you now. It's not actually providing access to the buffers beyond selecting from frames using the map. So a 640-frame buffer is no issue.
Its... well, cool. But it can't actually do what I was discussing :) Still, it's excellent for basic (and even quite fancy!) slitscan + delay effects. I'll have a lot of fun with that one :D
Some questions + suggestions:
Does it support >8bit colour for the delay map? Having a 640-frame buffer is great, but you can't select >256 of those frames without >256 shades of grey.
Do you plan to extend it at all? If so, I suggest this: take 3 values from the delay map, r = delay amount, gb = pixel coordinate.xy. This way you can do very advanced effects, remixing both spacially and temporally (this is what I was talking about before). I guess coords will have to be remapped from 0..1 to 0..image width. If all images in the buffer are the size size it should still be quite fast.
I'm very impressed with how fast this is!
Can't you just put the source image on a loop (eg., like, a tile that is constantly moving horizontally), to get that "moving across" thing you're talking about? I've done that before, in just a sheer "guess/poking around moment" and it sounds like what you were describing up above (?).
I know you can feed this "checkerboards" of various shades of grey and get distinct regions in the delay map, or move squares (or any shape) around in an RII and have it get map delay to that x/y area.
After having used this for a long time, one thing that would be advantageous, if not standard at all, would be an option to be able to push the first frame across all samples at once. For live stuff, it doesn't really "work" in many scenarios to have to have the "queue" fill up, but it would be a-ok if it looked like a normal image when you flip it on, and then it starts slitscanning. Yeah, you can put a billboard behind or something...
I did a little Finder search and found this composition I made when I first got this plugin. It uses a v002 zoom blur filtered video feed as the delay map. Totally "wrong", but a really psychedelic effect.
Hmm.. panning the source would get exactly half-way to the effect I'm talking about. If you take the stack of images as a 3D volume, with normal x+y plus z being time, that would give you a diagonal slice scanning through the volume. The effect I referred to originally would be a slice through y and time, scanning through x, which still isn't possible.. or is it? Damn, this stuff gets confusing!
No, scrap that, I think it IS possible if you can scroll at exactly 1px/frame.
Either way, my suggestion of having R = delay, GB = pixel position has me much more excited. With some cunning all kinds of amazing effects are possible :)
I had a play with it before using a motion detection patch I wrote ages back, so areas with motion in the current frame have no delay and 'calm' areas have long delay. Didn't quite work, but it was cool watching somebody walk past, then walk past again some time later :D
Hey glad you finally tried it.
Yep it stores the delay map as 32-bit luminance. If you're running into problems with that send me a composition that's not doing as you expect.
I must be being very slow here... gb = pixel coordinate of what?
Cool :D Didn't have time to try it earlier (there's only so much that can squeeze into lunch break!)
At present, for each input pixel it takes the buffer index from the input luminance, and the samples the pixel at the same coordinates as the input pixel from that buffer image, yes? I'm suggesting a variation where the sample coordinates can be changed, using the green/blue values from the delay map.
It would need a CI filter or whatever to create gradients in g + b channels to achieve a 'normal' result (like what you have now), but it'd be amazingly powerful! You could remix not just time, but time and space :D
This would have to be a separate plugin I suspect, since it would be a whole different beast and quite hard to understand perhaps :)
** edit: I'm happy to try this myself if you'd be willing to share source? **
Hey yea got you totally now. Would be simple enough to do, I'll give it some thought.
Re starting with a full buffer of the first frame - not a bad idea, if I'm making changes I'll do that. Thanks!