iPhone, Quartz, Quartz Composer

Quartz2D and QuartzComposer are two very different technologies. Quartz2D is simply a pretty API on top of OpenGL and other drawing stuff, while QC is a collection of high-performance technologies (OpenGL, CoreImage, QuickTime, etc).

QuartzComposer is not on the iPhone (not even in the private API's, as far as we've found) -- building it would be very impractical due to the iPhone's severe limitations on what's allowed for SDK apps. There are also some stiff limitations on graphical features (no GLSL shaders, no HW-accelerated CoreImage), which nullifies much of the QC feature-space.

One dead end avoided. Now I can concentrate on OpenGL for the dome - I saw one thread here, will follow up. More questions to come.

C4C Center for Community Weblog: The Daily Glyph

If, using a new Kinecore feature (iPhone restricted list), we could blacklist some patches in QC and limit to an iPhone compatible list. Is it possible for you to create a kind of "iPhone re-compiler" that could translate QC's patches into iPhone basics Apps ? Image a new Kineme app (like QuartzCrystal !) I will pay for that !!! (Like for most of your products...)

http://stakeventures.com/articles/2007/06/28/games-on-iphone-via-quartz-...

Here's the list of restricted patches that don't work on the iPhone:

• All Of Them

Here's a list of core classes/technologies that will make this infeasible:

• CoreImage
• OpenGL (iphone has OpenGL ES, which is a subset of gl -- this might not be a problem, but it's difficult to tell)

Additionally, here's a list of incompatible technologies that will make development impractical:

• UIView vs. NSView
• All touch-related stuff (vs. the current mouse/keyboard stuff). (this will possibly change with snowleopard, but I don't know the details).

While you're willing to pay for such a tool, unless we can get convincing support for 4-8 man-years of development time to recreate the QC environment on the iphone (it would require about that much development time/tuning), we're not going to touch it. (for some numbers 4-8 man-years of our time would be around 1/3-1/2 million USD)

[note that the article that was linked to is completely bogus, and it doesn't work at all on a real iphone as far as I know.]

[update: the test page doesn't work with the 2.0 nor 2.2.1 iPhone OS, the latter being the latest]

1. Low CPU speed. Look at how much resources QC uses.. how much memory + cpu time. On a 400mhz mobile device with 64mb, it's just not going to work.

2. Slow GPU. QC with a slow GPU just isn't going to be fun.

3. No shaders at all! That's right, no GLSL, no core image, nothing. If you want to use any filters at all, they have to be done on that slow CPU.

For now, I'd say the only slightly practical way to port QC stuff to iphone would be to re-write the composition in cocoa/opengl es. That would mean writing an app to do the job, and QC would become just a fast prototyping tool.

Maybe things will change though, there's some expectation that iphone os 3.0 is due this year, mobile GPUs suitable for iphone with pixel shader support are available (infact the successor to the iphone's imagination mbx, the sgx has good shader support) and faster CPUs are around (there's a new phone with a 1ghz CPU, and ARM are demonstrating dual core mobiles soon). So perhaps we'll see something good soon.

Thanks for the details on gl-es vs desktop gl -- I wasn't very clear on what exactly gl-es didn't provide (shaders being a significant one); a pretty crucial omission for anything remotely interesting in QC.

I remember being all excited at one point, thinking that I would be able to wrap up qc comps in a little widget type things and then put them on iPhone.... and then finding out otherwise :o( Such a disappointing moment.

OpenGL ES supports shaders, but only in version 2.0. I'm not totally certain, but I think the iPhone only supports 1.x in software.

But regardless, the current MBX gpu in it has no shader support at all, which makes it a moot point. The newer SGX chips are full 2.0 parts with shader support, and phones are starting to come out with that chip now. Apple have invested a large chunk of cash in Imagination (the company making the PowerVR chips) so everything points towards the next iphone having shader support (not that it's too helpful for a couple of years, unless you want to target a minority or the market..)

I remember a few wwdc sessions covering exactly these things. Sorry for repeatedly omitting details like version numbers :)

you're correct in that iphone (even as of 2.2.1) only supports GL ES 1.x, which doesn't have shader support -- it could be faked in software (like anything, ultimately), but performance would be abysmal :/

Well, i wasn't expect such a passionate reaction ! When i said, i will pay for a kind of QC2iPhone app, i was speaking as a NewB ! I could'nt imagine that this "convertion" was such impossible. Sorry for that... I was just dreaming to a QuartzCrystal Like application (Simple and efficient) to create iPhone apps with QC... Even if 3D and Shader doesn't works, a simple 2D application that use the "so friendly" interface "à la Quartz Composer" would be very useful. Just imagine using Math, boolean, images, sounds and movies could create fun apps for iPhone. I don't know a lot about the iPhone SDK, but it seems to be dedicted for C programmers. In the begining QC was a kind of "tools" for Xcode, so i hope for that kind of tool for iPhone: Nodal-Friendly-easy interface ! A lot of us are not developpers (and that's why we are so naïve sometime !) but we find, here, at Kineme, a place for imagination (with the creativity and the ability of developpers).

By the way, an auto-compiling option, with "auto selected and embedded" media and files could be possible in QuartzCrystal ? I also dreaming in a "export to App" as a new feature ! For shure it would be for Mac OSX only, but it could be great and solve lots of problems (like "embedded files", "special Xcode template"...). I hope not to be "too disappointing" with my other question, but if it seems possible to add an export2app (for Mac only !) option in QuartzCrystal, i could start a new thread in the feature request (cheaper than 1/3-1/2 million USD !) Thanx for all.

I've been thinking about a QTZ->App tool as well for a little bit -- the QCFullScreen skeleton in the wiki was a very primitive start to that; Obviously a bit more work would be necessary, but not too much. -- to discuss this more, another thread would be appropriate (to separate it from iphone stuff, which is very unrelated)

Compiling for an iphone would be considerably more tricky.

I don't know how much more friendly the iPhone SDK will get -- it's an embedded device, after all, and there are developer-esq things that are essential to know (like memory usage and algorithms etc). It'll be interesting to see where apple takes it over the next few years.

Sorry if I came off as angry -- it wasn't my intention. It's just that QC->iPhone is a Really Really bad mix, and people bring it up from time to time, getting their hopes up, when it's really very impractical currently :(

I think we've all dreamed of QC for iPhone at some point, and I'm sure I asked about it ages back (I definitely looked into it anyway).

Dunno how much more friendly it'll get from this side though, apple need to do a fair bit of work with interface builder first I'd say. Although they presumably have multiple teams working on different things..

Chris: I think the "export as app" is a good idea, and can easily be done in a plugin (perhaps as part of a future kineme core?)

You'd just need 2 very basic apps, one with a window + a drawer with a standard QCParameters view, the other fullscreen with QCRenderer (perhaps a small setup dialog with screen res, cursor hidden or not etc. would be good). Then just save the .app bundle to the desktop or wherever, and add the .qtz file to it with some generic filename that's set in the code.

Plugins would be a bit more troublesome, but I guess there's some way to find the path to the plugin bundle? And you already have the code to check for plugins in use in the composition..

I'm sure that would be a very popular (and possibly sellable?) one for the non-programming QC users (which I guess is the majority :)

It would be cool... I find using non-standard API plug-ins troublesome in contexts (still haven't figured out what the frick is going on with that QC multiscreen app...visualizer? performer? can't remember.... it sometimes craps out when I go to full screen with non standard API stuff... it is so aberrant and hit or miss.)

any more news on this in light of 3.0?

news is: still no QC.

Note the lack of a Quartz.framework:

cwright@phendrana:/Developer>ls /Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.0.sdk/System/Library/Frameworks/
AVFoundation.framework/        CoreData.framework/            IOKit.framework/               QuartzCore.framework/
AddressBook.framework/         CoreFoundation.framework/      MapKit.framework/              Security.framework/
AddressBookUI.framework/       CoreGraphics.framework/        MediaPlayer.framework/         StoreKit.framework/
AudioToolbox.framework/        CoreLocation.framework/        MessageUI.framework/           SystemConfiguration.framework/
AudioUnit.framework/           ExternalAccessory.framework/   MobileCoreServices.framework/  UIKit.framework/
CFNetwork.framework/           Foundation.framework/          OpenAL.framework/              
CoreAudio.framework/           GameKit.framework/             OpenGLES.framework/ 

And before this turns into a "should-exist fest", keep in mind that hardware shaders were /just/ added to the iPhone 3GS. So technically CoreImage might be possible. However 64bit, OpenCL, GCD, and other Snow Leopard technologies aren't available on the iPhone (yet?), so the environment still won't make much sense on the iPhone come Snow Leopard. Sure, compatibility mode would/might work, but how many of you honestly used that to make Tiger compatible compositions? I think I did like once, just to see if it actually worked. :)