Choppy Video in QC

Caelshadowhunter's picture

Hi, just a quick one.

Has anyone noticed that video playback in quartz can drop frames during video playback?

I've been having major problems. They seem to be worst when using the standard QC "movie importer" but still persists when using the same player in asynchronous mode.

I've also tried the V002 movie player, which helps again but it's still not reliable enough for what I'm trying to do.

Has anyone got any theories as to why this is happening?

gtoledo3's picture
Re: Choppy Video in QC

If you exceed the performance capability of your computer, you'll see frames drop. If you're using an effect that your computer is incapable of rendering at the same frame rate as the source, you'll see frames drop.

If you grab the viewer or editor window and shake it vigorously while watching your program, you'll probably see frames drop. If do something really heinous with another app... you'll probably see frames drop.

The usually alternative would be for frames to queue (err, not talking about the QC queue... talking about the alternative to skipping those frames), and then you would see a kinda "whirrrr" timestep up when the computer recovers. You would see frames from the past, instead of it at least catching up... actually, the initial version of KinectTools did that (in a self contained kind of way), and it was fairly obnoxious. In a really generalized sense, I could see it being beneficial to have control over that kind of behavior, but I don't believe that it really fits into the QC pipeline.

itsthejayj's picture
Re: Choppy Video in QC

Have you tried kinemes own video tools plugin i find the ffmeg Kineme Video Player player to be the best way to play videos within QC, check the supported video formats though. And the cache n history settings.

gtoledo3's picture
Re: Choppy Video in QC

Oh yeah, that's a really good point. Agree. It's not going to help all scenarios, but it's pretty solid.

vade's picture
Re: Choppy Video in QC

Unsure if it will help, but try the beta 2 of the v002 Movie player: http://v002.info/downloads/beta/

Caelshadowhunter's picture
Re: Choppy Video in QC

Well, I' fairly sure i'ts not the hardware were using (It's not mine and it's fairly beefy stuff... Xeon prosessor; 6GB of decent quality ram; and really great video cards).

We have tried the video tools plugin, it's definitely much better than the standard movie importer. Your player offers much better performance again Vade, and it's awesome, really great stuff, but we're trying create a setup which involves fairly large scale projection of video files and, because it's so big, any imperfections, however minuscule frames are more obvious...

There's been a suggestion that the problem's might be down to QC's 'patch time' being an unreliable, I don't think it's down to this, as the problem persists (to a much lesser extent) in asynchronous mode,and with V002. I would really love to hear thoughts on this theory though...

gtoledo3's picture
Re: Choppy Video in QC

No, it's not patch time.

Ok, are you talking about straight up video with no processing at all? Are you talking about video with processing?

Do you get the earlier concept? That in realtime rendering, if something is "too much" to render (I'm putting this super simply) at a given time, that your basic options are for either frames to drop, or to have the visual info "slip" and keep drifting further and further. In trying to synch something with live occurrences, it's more productive for your frames to drop. I don't think there's any need to theorize.

You need to examine what codecs you're using as well, because some are simply going to be better at demanding playback scenarios.

harrisonpault's picture
Re: Choppy Video in QC

I would definitely look at your codec. Anything with b-frames and interframe compression has got work to do for every frame. If you have the resources to convert your source video to jpeg, AIC, ProRes, or such you could compare flaky-ness between codecs. Depending on the resolution you are running, the compression algorithm, and the frame rate you can require substantial processing power just to render the frames.

vade's picture
Re: Choppy Video in QC

This times 1000x. Try playing your movies in Quicktime, with "Play all frames" and present full screen. That is one of the more optimal paths hardware can take to decode.

If you are using just Quartz composer, and not a special app, you also have hiccups on the main thread. It is highly suggested that you use a CVDisplayLink from Core Video to drive a multithreaded renderer.

cwright's picture
Re: Choppy Video in QC

before forming a hypothesis, you should define 1) the size of the video 2) the codec used 3) the power of the machine 4) other things the machine is doing at the time 5) other things the composition is doing at the time 6) where the video is located (on a local harddrive, on an external harddrive, on an SSD, on a USB flash drive, on a network drive, streamed from a remote host in siberia, etc).

without 1-6, it's difficult to make any informed guesses.

cwright's picture
Re: Choppy Video in QC

this is extremely important -- rendering on the main thread is not a good idea if you won't want to drop frames.

EDIT: And in case indentation isn't restored as a policy, I'll clarify to not look retarded: "this" above is in relation to @vade's comment about rendering off the main thread. #LoL #anarchronism #disorganized #twitterSyntaxDoesn'tBelongAnywhereOtherThanTwitter

jersmi's picture
Re: Choppy Video in QC

vade wrote:
It is highly suggested that you use a CVDisplayLink from Core Video to drive a multithreaded renderer.
What does this mean in this context?

gtoledo3's picture
Re: Choppy Video in QC

In simple terms, this means that Core Video is driving the drawing loop... that will tend to help things stay at a consistent rate.

In QC, the events from the editor and viewer run on the same thread, which makes stuff happening in the editor impact the viewer.

I'm sure vade might have more clarification/details...

Caelshadowhunter's picture
Re: Choppy Video in QC

Hi guys, if it helps... the video is 720dp; the machine is just running quartz; whilst we need a composition to do basic transformations to the video (1024 perspective transform / 1024 bezier surface), it seems that the problem persists even just when rendering onto a quad; video is located on local hard drive (which isnt solid state). We're still experimenting with codecs so any recommendations would be great.

Vade, if you have the tome could you please elaborate on the "CVDisplayLink from Core Video" theme?

(PS sorry I've been away for a while, I'm juggling a lot of balls at the moment, [I have AS level {British college} exams in just under a month])

cwright's picture
Re: Choppy Video in QC

Caelshadowhunter wrote:
Hi guys, if it helps... the video is 720dp; the machine is just running quartz; whilst we need a composition to do basic transformations to the video (1024 perspective transform / 1024 bezier surface), it seems that the problem persists even just when rendering onto a quad; video is located on local hard drive (which isnt solid state). We're still experimenting with codecs so any recommendations would be great.

This is good to know. What happens if you drop the movie resolution (i.e. SD, less than SD, etc)? I'd also recommend experimenting with a few codecs, just to see what effect this has on performance in your particular scenario.

Can you post a "bare bones" composition (i.e., the "video to quad" portion only, no other stuff) and a snippet of the video file you're using, so people can check it out and do some experimentation for you/point out mistakes that might be present?

Caelshadowhunter wrote:
Vade, if you have the tome could you please elaborate on the "CVDisplayLink from Core Video" theme?

This isn't possible inside QC itself -- you have to write a custom app. It's pretty straightforward, and there are likely some skeleton apps laying around that do this for you already. QuartzComposer editor doesn't make use of this feature.

http://developer.apple.com/library/mac/#documentation/QuartzCore/Referen...

voxdeserti's picture
Re: Choppy Video in QC

we just met the same problems making our installations. A movie that is easily played by Quicktime has frame drops in QC. This issues become more visible in resolutions approaching to HD.

The main question is: what's the difference between quicktime movie playback and QC movie playback that we get so different unpleasant results?

vade's picture
Re: Choppy Video in QC

Which player are you talking about? If you are talking about Apples, its due to not using Async mode I suspect.

Care to elaborate?