Core Image Slowdown

toneburst's picture

I'm having major issues with composition slowdown when using even a really really simple Core Image filter kernel. The attached QTZ has the iSight feeding a Queue, which is rendered onto billboards with an Iterator. Optionally, the image is pre-filtered through a simple luma CIFilter, before going into the Queue.

Turn off the CIFilter, and I get more-or-less rock-solid 60fps. Turn it on, and the framerate fluctuates wildly, sometimes slowing to as little as 10fps for a couple of seconds, then going back up to 60fps. The CIFilter is only working on a tiny image, since I resize it before passing it through the filter.

Can anyone else duplicate this?

I'm running on a MacBook Pro

Snow Leopard 10.6.1

NVIDIA GeForce 9600M GT 512MB

a|x

PreviewAttachmentSize
Gridfun 121109.qtz16.99 KB

offonoll's picture
Re: Core Image Slowdown

Hi! I am changing direct to filter and back and I don't have any problem at all. My frame rate with 20 rows and 20 columns goes 20fps and few cells goes to upper 60fps

I run the same computer you have and SL 10.6.1 / NVIDIA GeForce 9600M GT 512MB

If you need something else, just ask me. :)

.lov.'s picture
Re: Core Image Slowdown

whoo, this run with 1fps (10*10 cells) on my 9400M based Mini (10.6.2)

cybero's picture
Re: Core Image Slowdown

Direct 2.8 - 3.5 fps CI - 4 - 6 fps nVidia 256MB 9400 iMac 10.6.2.

It really does vary, especially when the rows are being fleshed out.

When all grid images are in place, then we get the stable results shown above.

Rows 10 Columns 6 - any more than that and everything really does slow enormously on my iMac.

gtoledo3's picture
Re: Core Image Slowdown

Yeah, this is horrible. 2.4fps! Same system as you, but 10.6.2. For me, it's giving the same performance with the 640x480 as with the 64x48 image, so that's just stupid.

This is a trivial qtz too. I have a setup around almost exactly like this that I've used a ton in Leopard, and it flies. So, if you're peeved about how it works in 10.6.1, it works even less well in 10.6.2. At least you can turn off the CI Filter and get 60fps.

Crap.

toneburst's picture
Re: Core Image Slowdown

It's a weird one, isn't it... It's clearly not just Iterator overhead. I'm guessing something about the image going through the CIFilter forces some kind of extra format/colourspace conversion that slows things down horribly.

Or maybe it's something else. Either way, I'm not f*cking impressed...

a|x

smokris's picture
Re: Core Image Slowdown

FWIW, on 10.5.8, 8600M GT 512mb, this composition reliably gets 60fps, whether the filter is on or off.

cwright's picture
Re: Core Image Slowdown

With severe performance regressions like this, please post to the mailing list -- Apple's likely not aware of how "helpful" their performance "improvements" have been, nor are all the other "DUUUUDE SL's So FAST!111" dweebs that I've heard from since WWDC2008 (that have mysteriously shut up since GM... hmm... ;)

(smokris has access to a bunch of 10.5 and 10.6 versions, and a few different architectures, so he can produce some nice performance spectra -- other users with other hardware are of course welcome to pool in their data to flesh out the picture :)

cwright's picture
Re: Core Image Slowdown

Iterator overhead (from my profiling thus far) is largely CPU bound (they added a bunch of additional "optimization" stuff that actually doesn't help in 99% of real world cases), and it's essentially limited to halving the framerate at worst.

This is definitely something else (as you've already stated -- I'm just backing you up with my observations removed from CI filtering entirely)

gtoledo3's picture
Re: Core Image Slowdown

Yeah, I'm pretty intimately familiar with this type of setup, and while many things have irritated me in SL, this gives me tremendous concern. This type of setup FLIES in Leopard- 60~90fps with CI or not.

Double crap. You know, I've been peeved at some of the "glitches" in QC, but have written many of them up to the fact that I'm really anal about fine points. This is just HORRIBLE. Then, for it to perform WORSE in 10.6.2.... time for another cup of coffee.

Yep. I'm booted in Leopard. This sucks, I'm not mincing any words. Is there enough time in a day to keep writing bug reports? Why don't they just hire people that understand how QC works, and how people use it? Whatever.

Anyway, the Leopard stuff is with 10.5.8 -9L30, and the SL testing is with 10.6.2 latest seed, whatever that is. NVIDIA GeForce 9600M GT/4GB memory/3.06GHz processor. The difference in performance is just pathetic.

Yay for looking at crappy Google Warehouse stuff. Maybe QC can just become a front end for Google Warehouse... again "whatever" in the most apathetic sense of the word.

gtoledo3's picture
Re: Core Image Slowdown

FWIW, whatever is going on in 10.6.2 makes it "even steven". It sucks either way.

gtoledo3's picture
Re: Core Image Slowdown

Ok, at this point I'm going - is it possible to make an iterator plugin that emulates the old iterator, or is like the old but with real performance improvements that don't suck, ala Quartz Builder? Is that even reasonable? It horrible to think about manually "converting" to a plugin for standard patch function, but the performance of the iterator was already lacking. This just makes things silly. I expect there to be rough periods with any kind of big change in features, but this is getting absurd.

Besides the straight out crappy iterator performance, I'm sure that old things are broken - I never regressed into a simple version of my "time blur shuffle" (which is basically an emulation of a freeframe effect), to make it easy for people unfamiliar to understand, but that one doesn't work at all anymore (give me the benefit of the doubt on that one).

I think there is a solid case for eliminating the new "functions" in the iterator, because they're pretty useless in actual application anyway, especially if they play any part in this horrible performance. The counter stuff looked promising at first, but is a bust in my opinion, as far as interfacing it to do anything actually useful.

I'm peeved, but I don't mean any of it as a "downer" on Apple. I think the efforts were highly admirable, just that more was bitten off than they could chew. Everyone makes mistakes, but at the same time, they can't have something like this linger. They need to figure it out, and fast, because they are going to wind up with a Vista on their hands. Many SL users still don't understand what the point of the upgrade was to begin with, so it's a big yawn to the fair-weather consumers, and then people that are likely more aware of fine points, and the types that point out Macs as being great computers have to bite their tongue. I know I'm going off right now, but this really disturbs me. The first computer I used was a Lisa, and yeah, there have been some pretty boneheaded things that have happened with Apple over the years, but they are fooling themselves if they think they can saturate the market with little crap pieces of plastic forever. If their OS starts to tank, they're history.

cwright's picture
Re: Core Image Slowdown

It's possible, but it's a lot more work than it might appear: The bulk of the slowdown comes from QC's graph evaluation stuff (which was slow in 10.5, but slower in 10.6 most of the time) -- modifying this is an extremely intimate procedure with lots of gotchas.

I'm still working on reversing the 10.6 iterator all the way -- once graph eval is completely understood by us, we plan on making "GraphTools", which will have a Fast Macro and a Fast Iterator (essentially doing things the 10.5 way, but with 10.6 niceties like stateful patches etc, or in a completely new way that's hopefully better than 10.5 or 10.6). This is a few months away from getting started, as I'm wrapping up other projects and haven't fully transitioned to 10.6 myself (and until these fixes are available, it's rather unappealing to me, causing a convenient catch-22 ;)

cybero's picture
Re: Core Image Slowdown

Leopard 10.5.8 - CI - 8 to 13 fps | Direct 8 to 10 fps PPC G5 nVidia 5200 64MB Graphic RAM - I'm staggered - WT*IGO - This comp flies [relatively speaking ] on my old leaf blower.

Is this, it can't all be down solely to OpenCL frameworks stuff, if it has anything to do with that at all.

Bizarre and also very annoying.

Thanks for the dose of sanity a|x .

toneburst's picture
Re: Core Image Slowdown

Hey- I've just discovered something that's actually been fixed in Snow Leopard.... I was starting to think NONE of the old bug had been fixed at all, just a load of new ones created

My CoreImage Feedback effect, which used to cause some kind of horrendous memory leak, now appears to work fine.

a|x

PreviewAttachmentSize
CIFilter Feedback.qtz7.98 KB

gtoledo3's picture
Re: Core Image Slowdown

Yeah, that's true. I was happy to see that.

cwright's picture
Re: Core Image Slowdown

another gem is typing "atan2(x)" into a number port -- this would crash on 10.5, but doesn't on 10.6! (which kinda sucks, since I'd use that if I needed to crash QC in a hurry.. now I have to resort to OpenCL, which is hit or miss ;)

.lov.'s picture
Re: Core Image Slowdown

Ohh, yes, and its super-fast :)

gtoledo3's picture
Re: Core Image Slowdown

Well, now all you have to do is pick the mesh filter template :)

toneburst's picture
Re: Core Image Slowdown

cwright wrote:
now I have to resort to OpenCL, which is hit or miss ;)

Oh, I don't know... you just have to wait a bit longer. Patience, my friend, patience....

a|x

toneburst's picture
Re: Core Image Slowdown

It's quite handy. I remember now why I gave up on it though- you can't change the dimensions of the buffer image once it's been created, which limits its usefulness a bit.

a|x

toneburst's picture
Re: Core Image Slowdown

That sounds exciting, cwright.

a|x

toneburst's picture
Re: Core Image Slowdown

Going back to the original QTZ; I definitely think there's some kind of memory leak or cumulative f*ckup going on here, because if you stop and restart rendering, the fps rises to a reasonable rate, but then the rate starts dropping, but not consistently.

The other possibility that occurred to me is that the GPU is overheating, and this is what makes the framerate drop. I have noticed the fan ramping up, just like it does when I do any OpenCL stuff, and the back of the casing, behind the keyboard gets REALLY hot.

Do you think I should report this as a bug, or am I really asking for a lot expecting this setup to run at a consistent (and reasonable) framerate?

a|x

toneburst's picture
Re: Core Image Slowdown

And it's definitely more Core Image-related than anything to do with the actual CIFilter code itself- even with just the default 'passthrough' filter, I get exactly the same slowdown.

It's got to be something to do with texture format/colorspace conversions that's clogging things.

a|x

toneburst's picture
Re: Core Image Slowdown

OK, this is the same thing, but I've eliminated the Resize patch, substituting a Render In Image with the Width and Height parameters reduced, and I'm doing the monochrome conversion in a GLSL shader inside the RII.

Pretty-much rock-solid 60fps here now. So, why is the CIFilter (or the combination Resize>CIFilter) sooooooooo slllllllooooooowwwwwwwww in comparison?

a|x

PreviewAttachmentSize
Gridfun 131109 02.qtz20.19 KB

.lov.'s picture
Re: Core Image Slowdown

Yep, its much faster, runs around 26fps on my Mini with Nvidia9400M.

smokris's picture
Re: Core Image Slowdown

toneburst wrote:
Do you think I should report this as a bug, or am I really asking for a lot expecting this setup to run at a consistent (and reasonable) framerate?

This is a reasonable expectation. I consider it a bug (a regression from Leopard, even). Please file at http://bugreport.apple.com/ --- and also post it on http://openradar.appspot.com/ so we can file duplicates.

philwues's picture
Re: Core Image Slowdown

Hi

I made this for VDMX. Import into the VDMX mediabin. Add a new live input (webcam,...).

When the movie is playing, go into layer controls and select the video channel of the live input.

You now see a non-working slowdown. It doesn't have any latency and is flickering.

When you open the other version directly, it works just fine. Any ideas what's going wrong?

cwright's picture
Re: Core Image Slowdown

this is a known issue -- the work around is to use a Render In Image to generate what you queue.

philwues's picture
Re: Core Image Slowdown

Hmm I don't understand, cause I'm stupid...

Doesn't work. Can anyone help?

PreviewAttachmentSize
Philwues-render1.qtz3.85 KB
Philwues-render2.qtz3.44 KB