|
streaming data into QC with Structure from File (File Tools)What i try might be pretty bold, but worth it if it would work. I try to stream data from property list files into QC with the kineme Structure from File patch. Basically just sequential named binary plist files (data_1.plist, data_2.plist, ... , data_x.plist). Each plist contains 3D coordinates for a voxel type animation. For smooth real time animation, i need to get 30 frames per second. Everything works fine, but I do not get more then ~5-10 fps update rate from the structure to file patch, whereas the composition runs in 60 fps. Hard disk transfer speed is not the bottleneck, each plist file is only 140kB, which is about 4.2 MB/s total. It seems the Structure from File patch does need more then one cycle to read the data and shoveling it into a structure and to make it accessible to following patches. The performance inspector indicates that the structure from file patch requires ~53% of the time and an average of 0.02 sec . anybody any idea why? |
Loading a structure from disk is simple code-wise, but expensive in terms of CPU time.
I'm guessing that it's more of a Cocoa-framework-shouldn't-be-used-like-this type issue.
The execute function is really simple, and has no loops, so it's not like our code sucks or anything.
As a workaround, what happens if you combine all the plists in to a single large plist, with each sub-list being an element in an array. Then, to switch structures you simply change which index you're using. Then you're only loading once.
(And, of course, as I look over the pasted code, I notice a memory leak, so repeated loading is a Bad Idea. Fixed for the next version though...)
thanks for the reply I tried the method of combining all data in one large plist, but the apple tool (plist editor) is not capable of dealing which such large volume of data. it is 740 MB total as single binary plists. I might have to write custom code to do that. When i tried with a shorter combined plist, it was working with structure from file, but long loading times as expected. An additional boolean output at the patch, giving a pulse once the structure is created would be useful.
v.
why not try with XML ? it might be a better choice to handle LARGE datasets. Anyway, a 740 Mb file will take ages to load.
QC's XML has a progress indicator, and sends a bang when finished loading.Might be usefull to sequentially load multiple files.
As far as i remember, it is also asynchronous, so loading won't choke your FPS. Normally.
well, in xml format the data is 6.8 GB big. making it binary gets rid of a lot of overhead.