ci

QC and very large images

psonice's picture

Any suggestions for keeping QC efficient when working with large images?

I've got a little side-project (asteroid hunting of all things!) that involves processing a big bunch of large (as in DSLR size) images in 32bit mode. I've worked out the memory requirements for this at around 128mb per image.. ouch.

Well, QC seems to handle tiling the images transparently in the background, and with only one image plus a few copies loaded at any one time, it should run a little slow but otherwise OK right? Well, no.. in fact when I ran my composition it ate 6GB of ram. 6GB! I only have 4GB in total, my poor mac was nearly killed :D The composition isn't that heavy though, it should use a couple of GB or so.

First question then: how the hell do I keep memory usage to a minimum?

One possible issue is that I'm doing feedback rendering. Using CI with feedback is well known for causing a memory leak (I REALLY fear the results of that leak in this case :D) but I've re-worked my feedback using GLSL to get around it. I don't see any leakage with smaller images, and this seems to be stable once it's used a big chunk of memory.

Another problem is that the GPU's maximum texture size seems to be 1k x 1k for 32 bit mode (same memory footprint as the normal 4k x 4k in 8 bit mode). Again, QC seems to manage it seamlessly in the background, but I get issues with CI filters around the tile edges if I try to sample outside the current tile... any fix for that?

For now, I've created a macro that splits the image into 1024x1024 chunks so I can process them one at a time and recombine at the end. Not ideal, but it should get me running at least.. I'll have to add some overlap to the tiles if I can't figure out the CI issue. It's still going to eat too much memory though, so any advice is much needed :)

CIPinLight Feedback (Composition by gtoledo3)

Author: gtoledo3
License: (unknown)
Date: 2009.12.04
Compatibility: 10.5, 10.6
Categories:
Required plugins:
(none)

This is a protocol image filter that sets up a feedback loop using the CIPinLight Blend mode, an Accumulator, and an Image Transform. This is something that is fairly useful as a virtual patch.