Multiple video streams

steve holmes's picture

Hi everyone I am developing a composition that plays back multiple video streams of a classical concert and lets the user select different camera angles. The different video streams need to be synced together and the user can switch between the views. I have been using vade's movie player plugin with the playhead of the master clip driving the other clips playhead position. The clips will be 1024x576 and using and OctoCore MacPro with a 8600 GT I can only get 2 or 3 video files playing back smoothly.

I was wondering if there was a different way of doing this and whether anyway else had any luck with similar setups. I would like to be able to playback a total of 12 different feeds.

Is the bottle neck here QC or HDD access, as I may need to get a couple of SSD's to do some tests with.

Cheers for the help. Steve

dust's picture
Re: Multiple video streams

you are going to want to feed a recursive directory with movie downloader into an image multiplexor in order to keep things running smoothly. You can do this with stock movie downloader as well. set movie to external time and use a interpolator to drive the composition connect the movie duration to end value of the interpolator so no matter what clip gets loaded it will play to the end. Its up to you to decide when to cut. it will be much easier to do it this way than to mark positions in one clip. so there will be prep work. you will want three multiplexers all with directory scanner hook up your keyboard to trigger which shot you want etc... lets say the multiplexers are "close" medium" and "wide" shots all with corresponding indexes so when you click on the number 1 it plays the number 1 index of all your multiplexors. so for clip one both your close wide and medium shots for clip one are running at the same time. use a fourth multiplexer to select which clip will be shown close medium or wide. I have an example composition on my site if you want to look at it.

http://si02p.info/index.php/comps/application

gtoledo3's picture
Re: Multiple video streams

Try loading the structure, and toggling the index to flip movies... avoid the multiplexers. You will note that each movie is evaluating...It's a hard call. Convenience can outweigh that sometimes. What is "good" is to hard-code the filepaths using javascript, or ironically, a multiplexer, and then to pull into the movie loader off of that.

In any event, I think that's fairly typical to see poor performance in this scenario. I can't speak to if it's QC or the HD access, but I would tend to think it's a Quicktime/QC combo situation.

Your main hope for pulling off 10~12 feeds is going to be to use as low a rez as possible, and not to use CI that "blows up" the image, or anything else that's very intensive in your composition (iterators, driving it hard on the front with an external input patch, etc...)...

usefuldesign.au's picture
Re: Multiple video streams

12 videos playing back at 1024x576 sounds undoable on one box even if it is an octo-core. Why do all the clips need to be playing if user is selecting the view? If you want 'live-preview' windows like a TV studio, why not downsample a copy of each video file to something much smaller (for example 64x36) and play them back all the time. For full-size video switch between fullsize files with ext patchtime.

Am surprised about Georges comment that multiplexing multiple movie players results in them all getting "evaluation" and slowing it all down, what's happening to lazy evaluation in that case?

If you really want 12 videos playing 1024x576 at decent fps for less cost than an octo-mac you could buy 12 appleTVs and sync or slave the compositions to a master (multiple methods available), you'd need to hack the ATVs to OSX though ;) I'm guessing they would suck more greenhouse-producing-gas than a octo-mac too.

usefuldesign.au's picture
Re: Multiple video streams

Dead link ;) is your server gonna be up again, dust? Thx

gtoledo3's picture
Re: Multiple video streams

My mistake for being sloppy with my reply... what I meant is not that the movie patches are strictly evaluating/rendering, but that the "port" seems to hold the image. Once you've triggered it, you can toggle over the output of the movie player or input of a multiplexer, and you will see that it has held the last image that was passed through. If you chain CI filters between the movie player and the multiplexer, it seems that each node has an associated image that's being cached. The composition also has to remember extra movie position/duration info (that's probably fairly trivial)...

You are right on about downsampling. That's where I was leading with resolution. The smallest images possible in both dimensions as well as resolution are going to be the best.

dust's picture
Re: Multiple video streams

opps that was a typo si02.info or pixel8r.info

dust's picture
Re: Multiple video streams

you might want to think about making some sort of logic gate or use the toggle plugin that way you can toggle off the enable of the sprites for the shots your not using. that way they wont get evaluated. and at any given time you are only dealing with one stream of video. it might be some complex routing but the idea i guess in my head is to always have the same shot angles ready to be triggered that way you don't have to program your edit decision list. i mean in a realtime improvisational environment ideally you don't want to be trying to figure out what clip you are going to cue and does that shot coincide with shot that is playing.

usefuldesign.au's picture
Re: Multiple video streams

Okay, I get it now. That holding of last value is a really handy feature of multiplexers and other patches sometimes didn't occur to me it adds overhead but I'm not so much a technical type.

Re: resolution tip George, are you saying if you feed say, a 64x36 movie to a sprite and upsize it to say W=H=1 which might be 512x288 on a 20" display that it will render slower than a 64x32 (W=H=0.125)? Never guessed that either ;)

Sort of moving off-topic, have you noticed how fast the native blur used for sprites and buildboards renderers have when upsizing low res images to the rendering destination? If you tried to do a similar effect with the various CI blur patches out there you'd never get that kind of speed IME. Wonder what kind of algorithm it is?

gtoledo3's picture
Re: Multiple video streams

You're right, in some cases that behavior with the multiplexer is really desired. When you load something with the multiplexer once, it's not even just the "image" being held exactly, it's the whole set of instructions that the renderer needs to call upon, I believe. For instance, I've had times where I've had many large scenes inside of render in image patches, and when I trigger it the first time, I may actually see a lag when loading. Every time thereafter, the chain loads quickly. So that can actually be a case for the multiplexer... it's a bit of a catch-22. With movies, it seems like I've had the best performance from storing the file paths in composition (not via directory scanner), and then loading as needed. That might not be viable, but if it is, it's an idea.

As far as the resolution thing goes, all I was saying is that smaller images run quicker and that a smaller source image, upsized by the renderer to run at destination size, typically runs quicker than a big source image running at the same size as the render destination. That's kind of a lame solution for getting something running faster in some ways, but in certain situations it's a viable option. I've never done any kind of test on that particular scenario you outline though.

Checkout the resize patch for blur as well! It's actually kind of cool in certain situations.... just tie the sprite to rendering destinations and keep dropping the resolution of the resize.

franz's picture
Re: Multiple video streams

Hi Steve, here's an example patch to multiplex 12 video files. Feed in your own files, and eventually try the 2 different options ( multiplexing strings / or images). You'll need an external timebase to drive everything, that should come from your audio playing software ---- to be sure that audio and video stays in sync, even if you drop video frames.

Hope this helps.

PreviewAttachmentSize
12movieMultiplexer.qtz11.65 KB

stuart's picture
Re: Multiple video streams

I like the multiplexing method. Very slick.

Here's another way that comes to mind that might have it perks as well.

Make a large movie that has all the angles combined in a 3x4 or 4x4 grid. Similar to the Brady Bunch intro or security camera multiviewer.

Then one can crop out the region (or feed) on demand. All the regions will play in perfect time together and the computer is only playing 1 movie file, not 12. Although it's a larger resolution, it is still easier on the Mac and quicktime in some ways.

my 2 cents

gtoledo3's picture
Re: Multiple video streams

...and they are a proven two cents :)

That's an extremely good method with QC.

steve holmes's picture
Re: Multiple video streams

Thanks for all the help, this has given me a great starting point for testing. I have tested Franz's setup with the addition of a V002 movie player as well to do some A B testing between the build in movie patch and vade's. So far my results have been mixed but nothing useable yet. The best results have been using vade's v002 movie player with AIC encoded file (1024x576) with these feed into a image multiplexor and on to a billboard. Generally this works okay but still every second there is a slight stutter in playback. This seems to match activity monitors HDD peaks so it may be a limit in HDD bandwidth so I have ordered a OZC SSD to see if that helps.

In general it seem incredible difficult to get multiple movies to playback in sync on the same machine (with triplehead output) or across multiple machines using network broadcaster or OSC to keep things in sync. I have a setup that triggers 3 files (for use with a triplehead2go) with the same input splitter trigger and yet the centre file plays fine with the other 2 files 2 and 4 frames behind, this seems consistent but still annoying as it means I may have to render the files together and prevents me image processing each output of the triplehead2go.

Oh well thanks for the help I will let you know how it goes with the SSD. Cheers Steve

franz's picture
Re: Multiple video streams

if you're using a triplehead to play 3 files in synch, i assume you have 1 file per screen. Why not pre-comp all the 3 files in 1 then ?

Meanwhile, if you see a glitch every 3 sec. on your HDD monitor, and in case you've got plenty of RAM, you should try with a RAMDISK, like esperanceDV, to see if it helps.