Onscreen Keyboard (Composition by mattgolsen)

Author: mattgolsen
License: Public Domain
Date: 2011.06.14
Compatibility: 10.6
Categories:
Required plugins:
(none)

An onscreen keyboard for use in touch compositions.

Created by mattgolsen & swiftlikeninja.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

idlefon's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Cool!! I think the text is limited to twenty something characters though.

yanomano's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Wow.Great stuff !

Swiftlikeninja's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Thanks, The keyboard patch is limited to 30 something i think, but this has 32 per screen (num lock, caps, lowercase) and uses only hit test, javascript, and mouse input for a virtual keyboard but could easily be converted to use with the tuio bridge.

dust's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

sweet. lower case

jrs's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Awesome - says a lot about QC though that something this simple only runs at 38fps on my recent mbp

mattgolsen's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Yeah, not totally thrilled about that. We're open to any suggestions for improving the performance of it.

cybero's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Apart from the aforementioned 38.6 fps performance hit, there's little else to say than brilliant, a really useful patch. I actually get between 34 to 60 fps, BTW.

gtoledo3's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

@jrs, no, it says something about the performance of this, as coded. Nothing else. I know that reads a bit testy, but I think the comment is far off base in logic. I've had keyboards running that don't even give an fps reading that is measurable, from being so quick and only evaluating on key down.

mattgolsen's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Yah, the framerate thig is definitely a problem, and we'll probably be looking into ways to streamline it. But we figured there was a pretty glaring omission in the community for this sort of thing, so we thought we'd contribute. Feel free to check it out and offer any suggestions to improve it.

Swiftlikeninja's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

I took the main iterator for the key detections and threw it into a render in image macro. as a result the fps ranges from negligible to 600 fps...This is without disabling the rendering of the patch. (Ive attached the qtz)

On that note, could anyone enlighten me as to why this had such a drastic affect?

PreviewAttachmentSize
keyboard2.qtz105.56 KB

gtoledo3's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

The RII in 10.6 has a method:

  • (BOOL)supportsOptimizedExecutionForIdentifier:(id)fp8;

Which may be a factor, but that may be the method that makes stuff that's offscreen (or that QC thinks is offscreen) not evaluate. I'm unsure.

If you have provider patches, it also actually changes the execution mode, so that the whole macro actually becomes a provider. When the billboard, or whatever you're rendering to, goes up the chain to the RII, it probably doesn't evaluate any changes, after a check or two, unless do something that triggers your providers inside. It's a nifty trick... I believe I used this in one of my bezier demos. As long as nothing driven by time (like an lfo or interpolate) is in the macro, it should be "all good".

BTW, my comment about performance, etc., was in no way a knock on the composition. I opened it up for 30 seconds, and don't have an opinion on it one way or the other. I just wouldn't want someone judging what QC can/can't do by my compositions either. I'm definitely a bit defensive of QC, because it tends to draw people in that are new to programming, and I hear slags about performance or graphics sometimes that I think are more related to that, than the tech itself.

Swiftlikeninja's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Thanks for the great Info. I can understand the desire to defend Quartz Composer, Negative comments tend to stick in the minds of the uninformed. I am in no way a distinguished programmer and for the most part trudge through things until it works. I take from this community so much, that me and mattgolsen wanted to try to give back. We saw a severe lacking of onscreen keyboard comps and (mainly due to need)created the example above. We would welcome any and all criticism to make it better.

Swiftlikeninja's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

to idlefon: I just realized i misread your post. Yes there is a char limiter built in out of the desire to limit character count for the final output but can easily be changed by adjusting the conditional plugged into the active input of the javascript.

dust's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

i think its great you guys did this. one of the first patches i made was a keyboard patch in qc. taking input from the keyboard. oh man it was an ugly mess of noodles. that was before i knew how to make plugins. fortunately someone else noticed the void or missing keyboard and made a typing plugin, soon after the freeboard patch was born. this however did not address multi-touch. which ironically is irrelevant when you think about typing. i mean yes you use more than one finger to type, but you never use both fingers at the same time while typing.

once i figured out that i didn't have to program in ten finger ids per key to make a multi-touch keyboard. i opened up my macs onscreen keyboard in the language prefs and took a screen shot as a reference. i then decided how can improve this form of input by utilizing multi-touch. i decided it was best to build the keyboard in two parts. that way i could split the keyboard and rotate each side making it more ergonomically friendly. this worked out great using some basic multi-touch gestures i was able to split and rotate each side of the keyboard.

i felt like i made a big step forward in creating my multi-touch toolkit as all toolkits need to have some sort of keyboard input. i got to the point that i was going to program in system wide input, so i could use the keyboard to type into windows other than qc. at this point i discovered a tuio windows7 hid software emulator that turned windows 7 into a touch certified system. this was very interesting as like the macs onscreen keyboard windows 7 has one and its touch. after some serious calibration i was able to touch type both on mac and windows.

feeling accomplished i decided to start building a polyphonic touch keyboard (music) this was a bit more complicated as i needed to compensate for a polyphonic chord, and had to implement touches on all keys. things where going great, i re factored some code into unity and had both touch typing and musical keyboards that could be placed ergonomically on the the screen, i went so far as to build them in 3d.

after all that work disaster struck. i knocked over my time machine while copying a file to it. i have done this so many times that i wasn't worried until i noticed this time the transfer cable became loose. for what ever my entire back up went down. and went down at the wrong time. i had just backed everything up in order to make room for windows 7 so my computer was fresh and clean. ironically at the same time my computers internal hard-drive started reporting smart status failure and a day later my logic board went down. i pretty much lost everything.

i was just getting to the point where i wanted to share the keyboard... i even made a little screenshot video and put it on vimeo demonstrating the musical keyboard in 3d. i must say the only reason i am ranting on like this.... is that building an on screen keyboard in quartz composer wasn't fun, or easy, it was actually painful in comparison to doing the same thing in 3d with unity.

so i really wanted to say regardless of the performance issues i applaud your share. i have been cringing at the thought of building one of these things again. honestly it was a particular chore to map out the whole keyboard and concatenate the strings into sentences etc... particularly doing it alone and more than one time and in multiple languages. so i really wanted to say thanks. good looking out, you guys noticed a void and are preparing for the future and for that i commend both of you.

mattgolsen's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

Thanks for the compliments Dust. Both Swiftlikeninja and I have learned so much from this site, we wanted to wait til we had something with real value until we pushed it out for everyone. We definitely want to keep publishing stuff on here for other people to use.

I definitely get what you're saying George, about judging QC based on the compositions written for it. I don't think either of us took any offense to it. I also want to thank you personally because we've learned a ton of stuff from compositions that you've posted. That's one of the many things that I love so much about the QC community, the sharing of knowledge.

Thanks again for all the great feedback, I look forward to more!

usefuldesign.au's picture
Re: Onscreen Keyboard (Composition by mattgolsen)

I don't own any touch devices so I'm wondering what use cases people have for a touch keyboard and what physical interface people are using (seeing as iOS is a QC free zone)?

On the question of hit-testing and fps, I developed a comp that had more than 30 onscreen buttons. I ended going in the direction whereby I would take the mouse cords on a mouse-down and run them through a JS patch that was loaded with the region data for every button on screen. The JS patch output basic quad data for the visual regions of active buttons. It was kind of like a sequencer interface so many buttons may have active at one time. Also some buttons were grouped (radio button type logic) so the JS patch handled that and all the other state requirements of the larger composition too.

The JS patch output of quads went to a Kineme GL QuadStructure patch which generated a B&W matte/key (Via RII patch) which enabled various regions of an image of buttons in their active states to be supperimposed over the bkgd image of the inactive states and front panel artwork.

Anyhow the JS parsing of mouse data was impressively fast. I would avoid iterators if you can, although I realise in SL they can be somewhat optimised if you're careful.

George and I near came to blows over the most optimal optimisation route to go on this problem of mine! His QCButtons are plenty fast too, although for my needs it was better to handle it inside the JS patch because it enable me to handle a bunch of state logic at the same time.

I'd love to open source the whole comp but never got it to a point of release it was actually part of a much bigger vision, to have patchbank style control over QC comps running in the editor (or else where I guess). This was inspired by the fact that I'd often get all these great different looks out of the one composition but by the next day I had forgotten the combination of input values that created many of the results. Being an old synth head, a patchbank seemed like a logical and achievable goal!

I'm punting that global are in the next major QC release (Lion?) so that will help simplify my project which currently relies on Kineme Spooky, which as we all know (I didn't when I started PATCHbANK) can be flaky, but I've never had issues with it because I only pulse data through it on patch changes, not every frame.

If you want the JS patch to study PM me: usefuldesign.au (art) gmail (dot) com