version controlled compositions?

randall's picture

I tried opening a .qtz file in a text editor and found that while it opened, and I could recognize some of the stuff... most of it is totally unreadable. Has anyone dealt with the idea of version controlling compositions in places like Git, etc?

Would love to know how you guys do it.

cybero's picture
Re: version controlled compositions?

Perhaps you are over thinking version control.

After all version control can be achieved quite simply - file 1 of an earlier date than file 2 of a later date type version controlling. KISS.

What is usually 'unreadable' to the human eye in a .qtz file are the binary encoded data incorporated into port settings and so on.

GitHub for Mac is available as a download http://mac.github.com/ - it is very easy to use.

Download it, run it, log into your GitHub account and use whatever alpha numeric version control you want - suffix, prefix, whatever.

gtoledo3's picture
Re: version controlled compositions?

The qtz's are plists... but I think that issue is a sidestep (looking at the text, or anything other than the qtz form, to attain version control).

On the note of "dates" on files - this isn't dependable. If a system time gets changed, or files get backed up to externals, etc., every so often files can get completely wrong time stamps on them. I'm not sure the exact reasons why, but for instance, I have a cache of qtz's all with a 1979 timestamp on them, that were on an external HD at one point. Go figure. Qtz's can also store incorrect filename/path, depending on if it was renamed... It makes me feel suspect about using only info in the file.

As far as naming convention goes, I'd recommend something like this...

TIME_DAY_MONTH_YEAR_PROJECT_PROJECTPHASE_COMPOSITIONNAME_VERSION.qtz

Please note that I'm obviously not talking about a totally manual route, and about organization/archival naming schema, not doing difs on the compositions, etc. It really depends on your needs, of course, and how you see yourself searching for stuff. I'd suggest a folder hierarchy of something like Client/Project/ProjectPhase, and then something like the naming convention above.

This works well for me, because I can go to my main folder of QC stuff, and I then see a hierarchy of clients. Inside each folder would then be a given project. Inside of those would be the phases. When I open that folder, I can sort by name, and immediately get the succession of compositions, as they happened, regardless of mac os x time stamp functionality. I also don't have to look at the end of the composition name for something like "Version 20B"...though if I'm working fast, that does happen quite a bit, and it sucks. It creates more organization work for myself later, if I ever even do get to it!

smokris's picture
Re: version controlled compositions?

From the commandline, you can do plutil -convert xml1 yourComposition.qtz, and it'll convert your Binary Property List (the default encoding) into an XML Property List (which still opens just fine inside the Quartz Composer editor and elsewhere).

If you add these XML files to version control, you'll get semi-coherent diffs between revisions. (However, QC injects a lot of metadata that seems to change fairly indiscriminately, so there's a lot of cruft to sift through. And once you start inserting images stored as part of the composition, it quickly becomes futile.)

% diff -u composition.qtz compositionWithAddedCable.qtz
--- composition.qtz   2011-10-11 20:36:55.000000000 -0400
+++ compositionWithAddedCable.qtz   2011-10-11 20:37:16.000000000 -0400
@@ -20,6 +20,20 @@
       <string>QCPatch</string>
       <key>state</key>
       <dict>
+         <key>connections</key>
+         <dict>
+            <key>connection_1</key>
+            <dict>
+               <key>destinationNode</key>
+               <string>Clear_1</string>
+               <key>destinationPort</key>
+               <string>inputColor</string>
+               <key>sourceNode</key>
+               <string>LFO_1</string>
+               <key>sourcePort</key>
+               <string>outputValue</string>
+            </dict>
+         </dict>
          <key>nodes</key>
          <array>
             <dict>

cybero's picture
Re: version controlled compositions?

I found an interesting blog site that deals with timestamp problems http://blog.sudosu.net/2009/file-created-date-under-os-x/ that mentions using http://developer.apple.com/documentation/Darwin/Reference/Manpages/man1/... mdis tool. I was wondering if that could be used with a batch script to retrieve any remaining correct date information to then restamp that file's time date?

mattgolsen's picture
Re: version controlled compositions?

This brings up an interesting question. Would it be possible to add a patch search functionality to KinemeCore, searching through thee various patch names? There have been so many times where I have a QB app crash, and it gives the specific patch name, but 99% of the time it's "Javascript_12" or something along those lines. It's maddening for debugging.

gtoledo3's picture
Re: version controlled compositions?

That's the patch's key, as opposed to it's real name... every time you make a new javascript patch (or any patch), it will add that number onto it. You can search for they key info in the Editor by looking for tooltip info on it as you hover (I'm not suggesting this is as awesome as typing in a name somewhere, and it zooming to that area on the editor).

Each patch has an x/y position stored (I think in the User Info field of the plist, along with the "name" you can manually write in), to let QC know where it's placed on the editor. This is unreadable in form.

It seems like kinemecore could have something where you have your app (or qtz crash), you get the thing "javascript_12" blah, and then you reopen, type in "javascript_12", and it zooms to that place in the Editor.

I think right now, Lion has something where it lets you search for a name, but I'm pretty sure you can only type in the "real name", at which point you get a list of all of the same patch, with no kind of key info (that would be useful for letting one know which one they're actually selecting). You can keep selecting them until you find the right one though.

cybero's picture
Re: version controlled compositions?

Good point. A way of picking up named JavaScript patches would be useful indeed. It would also be useful with other coding patches, like OpenCL, CoreImage and so on.

I hasten to add, I can understand why the XML internals reduce the naming down.

mattgolsen's picture
Re: version controlled compositions?

Yeah the mouseover stuff works for small stuff. but it quickly becomes unmanageable.

I haven't had much of a chance to use the Lion version of QC, as I do most of my development at work, but I'll have to take a look at that.

A search function though would be phenomenal though, how hard do you think something like that would be for a first attempt at a QC plugin?

gtoledo3's picture
Re: version controlled compositions?

I think it would somewhat hard, because you'd have to use the skankySDK or GFPlugin, or whatever one wants to call it, if you want to have something weld itself onto the Editor, and there aren't any sample code projects of anything that does something like that, that I'm aware of. I don't think Apple documents the UserInfo string either (which is where I think the x/y locale of the patches is stored, but I may be wrong).

randall's picture
Re: version controlled compositions?

@smokeiris: Perfect. Exactly what I was looking for.

usefuldesign.au's picture
Re: version controlled compositions?

One thing I have resorted to when I have more than a few Javascript patches operating in a composition is to rename each to it's internal name so for Javascript_n I type "Sort Objects for hits (JS %n)".

eg Choose Obj with Cats (JS 11)

This is really good when you Log(); statements in your JS and multiple instances of similar JS patch feeding comments to the console where it just list the Javascript_n name.

One feature I have been thinking about for QC.next is an internal file structure that lets each patch (or node as they're called in most visual programming systems) be operable on from within the Editor. The inspiration came from seeing a demo of Tinderbox. TB uses text notes as it's primitive data element in a kind of mind-mapping/excel-for-text/dbase application with many varied use scenarios.

In addition to outline modes, notes in TB can be viewed in a couple of visual layout modes where they become much like nodes in a graph. In TB notes (nodes) can be assigned prototypes which govern their appearance and behaviour. So you can kind of program your text to self organise to some extent.

It would be cool to be able to write a quick 'prototyping' script in QC.next that says, take all Math Expression patches in this composition and rename them to be the Expression text, as is default in QC3. Or convert all Math patches to Math Expression patches and put splitters on all the value inputs. Or draw a rectangular note marquee around all the patches that satisfy this test expression and enter each Patch name followed by a NL character in the note text. Sort of trivial examples but I hope I am conveying the picture of QC.next application that is hugely extensible by the user community. Even within the rigid QC parameters there's lots of ideas for KinemeCore which would be nice but if each of us could create and share a couple of scripts that enhanced QC usability we'd have a much better QC.

Developers have to much to do and have to prioritise the features of an app as complex as QC (or QC.nxt). It makes sense to leave the bonnet unlocked so users can tune the engine, change the oil and even add their own superchargers or catalytic converters. Any mods that prove widely empowering/popular can be rolled into future version of the App. Given QC is for the developer/designer community it's the right audience to give a free set of mod tools too. Quark and InDesign went nuts with scripting in production houses and they're not even developer tools.