A New Direction

We're making a new multimedia visual programming environment called Vuo. Check out the new website, and read on to learn how we decided to take this path.



It's a little-known fact, but "Kineme" hasn't always been synonymous with "Quartz Composer Stuff."



The late 1990s


Photo from Pragmatic Illusion's 2000.03.15 performance. Left to right: Ashley Mokris, Bradley Mellen, Chris Wright, Robert Vance, Tina Kemp, Steve Mokris, Steve Burke. You can almost make out SPURTG's circular oscilloscope on the projection screen behind us.

Way back when I was in high school in the late 1990s, I went with a few friends to an ambient electronic music concert — a rare event for Columbus, Ohio at that time. There were 2 people sitting on stage, curled up among a pile of synthesizers. Nearby, they ran a slide projector that auto-advanced through a deck of blurry macro photographs. I enjoyed the music, but was confused by the slide projector — what purpose did that serve? To distract us, the audience, from the fact that we weren't able to discern what the performers were doing on-stage, since they were making minute adjustments to their synthesizers, without any dramatic rock star gestures, or even any kind of obvious connection between movements and sounds?

Inspired by Winamp's visualizations and the European demo scene, I decided I wanted to try to bring the same to the live performances of the band I played in. To establish a connection between the aural and the visual.

I worked with Chris Wright (saxophonist and fellow synthesizer-player in our band) to write software to take live audio input, analyze it, and draw graphics that looked kinda like Winamp. Linux, framebuffer, C and assembly. We used it in a few performances, like the picture on the right. We gave it the name "SPURTG" since I was a fan of impenetrable acronyms at the time, and published the source code a few years later.



The early 2000s

Fast-forward to 2003 — SPURTG was showing its age, since it only did Winamp-style flying oscilloscope spaghetti in fog, so development began on a second-generation realtime graphics performance environment. It focused on realtime control and video blending, and debuted for the dance/image/sound collaborative performance.

Chris Wright eventually stepped in again, and together we expanded it for use on my senior thesis project and a performance the following year, with a nascent GUI for patching together controls and graphics components.

We put together a website for the software — the original kineme.net — and were just about ready to publish it, when...



2005: Quartz Composer


The Kineme Brick, designed and contributed by Kineme user yanomano.

Another Chris, who had acted as a mentor for my senior thesis project, said, "Hey, have you checked out Quartz Composer?" I hadn't, seeing as I had been in full-on hermit mode for the last few months to get ready for that last show.

Quartz Composer was really exciting to me, as it seemed that a lot of the ideas we'd been working on had finally gone mainstream — Quartz Composer was built into the operating system and used by a bunch of applications. So I decided to start hacking on Quartz Composer to see if I could coax it into doing what the software I developed with Chris Wright could do. This resulted in, among other things, the first Kineme Plugin, published over here (not on kineme.net!).

This generated a ton of feedback and feature requests — and I was enamored with Quartz Composer — so Chris Wright, Matt Radcliffe, and I decided to start building a Quartz Composer community on kineme.net.

Over the next couple years Chris and I worked hard to produce a ton of Quartz Composer add-ons — Kineme3D, QuartzCrystal, QuartzBuilder, GLTools, and AudioTools.

And a bunch of awesome Kineme users commissioned work and/or donated to support our efforts: franz, stuart, coremelt, chriskeath, achim and florian, pbourke, niclas, gtoledo3, muqeem, sare, magnetmus, tempest, sebastian, nilsporrmann, chwaege, brettm, peterknu, benoitlahoz, itsthejayj, benjaminryle, usefuldesign.au, pantakk, volkerk, sheridanis, m1, cbn, gabemott, shaun, and balaban. Thanks, also, to everyone else who purchased our Kineme plugins and apps!



2010: Quartz Composer begins to wane

Chris Wright was then hired by Apple in February 2010, initially to work on Quartz Composer, but soon after was stolen away to work on Core Graphics. Kineme continued to produce Quartz Composer add-ons — releasing Kineme2D, ChartTools, VideoTools, and NetworkTools over the following year, and providing support and improvements to the existing software.

But it soon became clear that Quartz Composer was not a priority for Apple. The release of Mac OS 10.6 in mid-2009 introduced several minor new features, but also introduced a ton of bugs, many of which to this day remain unfixed. And the Quartz Composer platform still has several fundamental flaws that bug us: not supporting any OS besides Mac OS X, not supporting modern OpenGL graphics, and slow performance.

So in 2010, after dusting off my notes from 5 years earlier, we began research and discussion, and eventually development, of a new visual programming environment. We're moving on.

...but we're not abandoning Kineme. We do not plan to add major new features to Kineme products, but we are planning to continue fixing bugs and providing support though at least mid-2013. And the kineme.net website isn't going anywhere, since it houses a ton of valuable knowledge you, the Kineme community, have contributed.



You've been working on this for 2 years and you're just now telling me!?

I hate vaporware. And I don't like making empty promises. Our ambitious design has presented a lot of technical challenges — we're making a new environment, not just a clone of Quartz Composer — which I wasn't sure if we'd be able to solve. In the last year, Kosada has hired 3 new employees to help get this project off the ground. So far, Kosada has been funding this project with money saved up from other consulting projects.

And I think we're finally close enough now that it's safe to announce it — we've built numerous working prototypes, and now have the solid beginning of an actual functioning language.



2013: Vuo

We're calling it "Vuo" — the Finnish language word for "flow," as in data flowing through cables. We just launched the website:

vuo.org

Sign up for our mailing list if you're interested in keeping up-to-date on our progress. And feel free to ask us questions. We're planning to publish the first beta in early 2013.

We hope that, as a Kineme user, you'll see the potential in Vuo — and that you'll consider signing up for a subscription to help kickstart future development once we begin accepting subscriptions in January 2013.

Thanks for reading all this, and I look forward to embarking on this new era with you.

— Steve

and Team Vuo:

Jean Marie Cackowski-Campbell
Melissa Egan
Karl Henkel
Bradley Mellen
Matthew Radcliffe
Jaymie Strecker

vade's picture
Re: A New Direction

Congrats everyone on the stealth mode work. I am very curious about how Vuo "feels" when working in the environment.

Will Vuo

a) be loadable in existing applications in a safe manner (ie, have Obj-C or C support for loading as a bundle)?

b) Will Vuo support 3rd party plugins, or since Vuo code is compiled at runtime (so it seems), will it be just use "macros" from a library? I am thinking OpenCV, porting my plugins, etc.

c) What data types are supported? How do you deal with image processing, ie: images on the CPU vs GPU.

d) Licenses. You say open. What does that mean?

Very very curious, and again, congrats on all the hard work.

Edit: Nevermind, I see some of this is mentioned in the FAQ.

vade's picture
Re: A New Direction

Ok, let me come back with a few more specific questions:

What is the node mechanism for execution? Is this still QC-esque pull metaphor? Judging from the screenshot, it looks "push", with a Max MSP style metro object that initiates processing (at least in this example).

gtoledo3's picture
Re: A New Direction

It's push.

mradcliffe's picture
Re: A New Direction

Yes, it's event-based or push.

This was a topic of intense discussion. Ideally push & pull would be great (press keyboard and vertical refresh execution at the same time), but I think that is an idea for Vuo 2.0. There is a lot to worry about when you have both models.

I hope jstrecker or smokris can write a blog post about that some time.

vade's picture
Re: A New Direction

Yeah, just as a topic of abstracted / programming pro's and con's, I'd love to know more about that decision making process, but Im in no rush.

mradcliffe's picture
Re: A New Direction

It would be a cool topic at a conference for anyone to present on too.

Udart's picture
Re: A New Direction

It will be interesting to see how Vuo compares with the other node based environment you posted a pic of recently, Vade. I can only say - the more the merrier. I wish the best of luck to both projects.

.lov.'s picture
Re: A New Direction

This is sounds very-very interesting and awesome, congratulation guys, and can't wait to try Vuo! ;)

dust's picture
Re: A New Direction

cool a new media visual programing environment with api and standalone export for mac and ios sounds pretty handy. i think it would be cool if you could also export to webgl or maybe have some kind of web player or plugin type of thing ?... this new direction sounds promising. i think its a great idea, with kineme community support vuo should have a good start. having a qc patch/plugin etc... for vuo could ease peoples transition into a different environment ? having a data type preview node for the editor would be awesome. like a qc splitter but a splitter with a little billboard preview window ? so you could visualize the data flow inside the editor while building and debugging your patch ?

pixelnoizz's picture
Re: A New Direction

congrats!

gtoledo3's picture
Re: A New Direction

Congratulations! It will be cool to see it working.

franz's picture
Re: A New Direction

kewl looking forward to it !

cybero's picture
Re: A New Direction

Exciting news.

Like the potted history of how you came to Vuo - shall look forward to the Vuo updates from vuo.org's mailing list.

tobyspark's picture
Re: A New Direction

Great! Best of luck. You'll have a subscription from me.

After years of using QC, beyond the above these would be the two things I'd really like to see addressed.

i. Really paying attention to everything parallel. Reference the interface and state change efficiency of spreads as per VVVV, or the crazy making of duplicating macro patches and having to perform the same edit on all duplicates.

ii. XPC or similar so you can use it as a rendering engine for your own GUI app and -- critically -- edit the composition while the app is running.

Toby

epochapex's picture
Re: A New Direction

Congrats! I'm looking forward to learning this one from the beginning.

harrisonpault's picture
Re: A New Direction

Sounds very interesting. Can you clarify for us less technical types, who don't live C-variant programming. I tend to embed QC in existing performance applications/ecosystems rather than build applications.

  1. I gather that IFF VDMX, CoGe, Module8, Resolume, v002/Syphon choose to, it will be possible for them to deliver versions of their software that can integrate VUO compositions as plugins/media?
  2. That sometime in the future, if they so choose, Windows/Linux-based performance applications could provide a plugin capability for VUO also?
  3. Kineme is not building a performance application, but rather a toolkit application in the same general space as Quartz Composer, OpenFrameworks, VVVV, Processing?
  4. The idea is that noodlers like myself should feel at home with this new environment, even if I find GLSL shaders and OpenCL kernels somewhat opaque and, er, challenging?
I added an extra break line before the ordered list so Markdown would format it properly.

mradcliffe's picture
Re: A New Direction

  1. Yes, that should be possible.
  2. Ditto.
  3. Vuo will contain the compiler, node sets (like patches/plugins), and an editor. It can be a performance application that is usable in interactive performances, installations, etc... We use QC for that now too. :-)
  4. Vuo should also be intuitive to use for noodlers. That's one of our criteria.

Edit: Didn't realize there was a 4.

gtoledo3's picture
Re: A New Direction

  1. Is there still a wall object planned? Will there be modes to drop/queue/coalesce data?

  2. Will the editor still be able to read/translate qtz's of a subset of patches?

  3. If so, will the editor take qtz's, traverse through the graph to find leafs, and add pushes?

  4. Is there going to be GLSL shader support?

  5. Is there still a notion of "types"? Can we make our own?

  6. Are all values node objects that may or may not walls in between them and the patch?

  7. Will it be possible to create an object in VUO that loads something like a CI filter?

  8. Will there be tool for building interfaces in the kit, or would Xcode be used?

mattgolsen's picture
Re: A New Direction

I am so damn excited to hear you all announce this. I've had a sneaking suspicion for awhile that you were working on something new. I look forward to supporting your effort in any way humanly possible. If you're ever in Louisville, KY, look me up and I'll buy you beers.

Are you all planning on matching the current patch set that QC has with Vuo? Has there been any talk about the possibility of importing of .qtz files? What about Javascript support?

Something that would be really cool is if you have a bounty list on the Vuo site of feature suggestions, and users could crowdfund them; ie. $200 dollars to add a node-set with "X functionality", or $50 dollars to add an option in the preferences to do "Y functionality". You could even game-ify it a little bit and battle the features, "would you rather have X" or "would you rather have Y", and after voting you can Paypal (or whatever) money to your team.

Ooh and one more thing. Once you get to a 1.0 release version, you need to get Vuo submitted to Steam since they're selling software now. Money!

usefuldesign.au's picture
Re: A New Direction

Congrats on the announcement. Be interesting to see how the innovative business model works. Will people get involved in an open source project that has a King (or one ring to bind them ;-])? I certainly hope so, I've already got my new patch contributions penciled in ;-) Better get my C in order.

yanomano's picture
Re: A New Direction

Looks like an awesome project. Your guys are always there to write the story !

shakinda's picture
Re: A New Direction

Great stuff lads, congrats

I'd just like to assure everyone that iloveqc.org will be continuing to develop and expand on Quartz Composer community and development for many years to come. Although some QC ninjas may have found it's limits there is a whole lot of power in there for many others and unleashing that to the wider creative public is something we are excited to continue to develop :)

Udart's picture
Re: A New Direction

Congrats - or should I wait till the project is actually out? In any case I'm very excited to see something new happening in this corner of the world. Even if it means further headaches when finding out what to choose for every new project.

One question: Will Vuo integrate Apple Scene Kit? Or some similar high level approach for 3D that provides a scene graph with lights, cameras etc.

cwilms's picture
Re: A New Direction

This looks very exciting! I am looking forward to seeing where you take it and what wonderful things people make with it.

A few questions come to mind:

1) How much do you expect the subscriptions to cost? I haven't seen an answer for that anywhere yet.

2) Will the viewer window be as scalable as the one in QC? I have one QC project that runs at 2560x720 (double-wide 720p) full-screen on a secondary display and was wondering if Vuo will be as versatile?

3) When making a native OSX app will it be possible to make an app with a gui and a seperate viewer window and select what monitor each window will open in (ie GUI on primary and viewer one secondary display)? What would be VERY cool is a feature to have the GUI run on an ios device running Vuo and the viewer window and all the processing happening on a Mac or Windows machine.

4) Will it have a Javascript node?

5) Will it be possible to import QC compositions and have them translated to Vuo projects?

mradcliffe's picture
Re: A New Direction

I can only answer a couple of your questions:

1) We're not quite ready to publish the subscription model yet, but it should be up soon. That's probably the #1 question at the moment. :-)

5) We tossed this around, but it probably won't be there for the first release.

jrs's picture
Re: A New Direction

My two cents are that your time would be much better spent on implementing new features than the functionality to import old QC files - I'd much prefer to see the ability to add patches and make connections programmatically via an API.

Apart from allowing people to write their own converters it would make many tedious noodling situations much more bearable - whilst clips are great for re-usability and modular coding I often find I have a number of identical macros where I need to go in and change an aspect in all of them. It would be great if there were a script-able way to go through the graph and do things like find and replace, publish/un-publish ports etc.

dust's picture
Re: A New Direction

i agree that maybe the ability to import old qc files might not be as necessary and implementing vuo features and requests may be more beneficial. if one of the final goals is to push vuo out to ios then integrating qc files wouldn't really help in those regardes. also given there will be some sort of framework or api available qc could easily integrate in that context. although im glad vuo is considering a qc vuo patch and vise versa. there really isn't any need to edit a qc comp inside the vuo editor, so im glad you guys don't plan on it. that really sounds like major redundancy and waste of time. however rendering a qc comp in vuo as an image port or something does seem extremely useful.

a recent vuo post suggests that the vuo team is a bit overwhelmed, which is understandable. in the road map it suggests there will be an api that will let you embed vuo into your apps, which is great. is there any plan on a plugin type of framework that will let 3rd party developers create custom nodes ? or is this covered in vuo 0.2 all ready. the road map states

Quote:
• Preliminary visual composition editor — create nodes, connect them with cables, edit values, run the composition
when it says create nodes does this mean the ability to create custom nodes or just the ability to add existing nodes to the editor ? obviously custom node creation would be very beneficial and also help offset some of the work load. the road map isn't entirely clear in these regardes ?

im sure this has been a topic of discussion internally. it may not be a needed feature for many people but it would make vuo a go to tool for me if this was possible for vuo one point zero. i was thinking more along the lines of a pre compiled bundle loader type of plugin like qc has but even a runtime java script evaluator or f-script type of node would cool as well.

GL's picture
Re: A New Direction

I once started with QC because I had no money for decent VJ-software. . . . I'm actually a social worker and for the rest of the time a filmer / photographer / producer. QC caught me really escpecially in combi with QCrystal and QBuilder. So I didn't have to buy FC as well. I'm very thankful for the Kineme community for waking up the geek inside of me and giving me the freedom to visualize my ideas on the spot.

But since I had to buy a new computer and I'm on 10.8 . . . well . . . sigh

So I'm very very happy with this news and wish you guys a lot of energy, concentration and coffee.

bless=more, Geerard

harrisonpault's picture
Re: A New Direction

I'm back on SL still, but it seems to me from discussions here that QC is pretty much fine on ML. Does your "sigh" mean that you are currently experiencing problems? What are they? And have you followed the 10.8 discussions here?

GL's picture
Re: A New Direction

yes, my sigh is about having problems with Vade's movieplayer. I made nice things with that plug-inn, all BPM synced and connected to sliders for nice wopwop effects on dubstep. But on 10.8 QC is crashing with opening old compositions (10.5) containing Vade's movieplayer. And for new compositions there's no signal.

With the normal movieplayer of QC itself it's not possible to build simulair things. I tried async different stategies but it's not working or has a framerate of -1 ( yes . . minus 1).

Half of my VJ stuff is based on long slomotion movies (read Vade's movieplayer). So my sigh was about missing this essential part.

vade's picture
Re: A New Direction

http://v002.info/forums/topic/v002-movie-player-beta-4/

If you have issues with our software, be sure to complain in the right forum :P

thomasvanta's picture
Re: A New Direction

Congrats!!! Really looking forward to it, and meanwhile refreshing C and derivatives...

dust's picture
Re: A New Direction

man i wish i had money for vuo right now. i got almost all of the kineme products and im sure it is awesome ;) looks like a bunch of work has been done.