Mouse Interaction 'The Hard Way'

toneburst's picture

Little demo of mouse-interaction without using the new QC4 Interaction patch. I need a few extra options the Interaction patch didn't allow, so ended-up attempting to do the whole thing in JS, with a simple Iterator to render the billboards.

Here's the JS code, if anyone's interested:

/*
   toneburst 2010
*/
 
///////////////
// Functions //
///////////////
 
// Init points array
// In: number of points, scale on x, scale on y, offset x, offset y
// Out: array of point objects
function initPoints(numpnts, scalex, scaley, offsetx, offsety) {
   // Init vars
   pnts = new Array(numpnts);
   var spacingx = scalex / (numpnts - 1);
   var spacingy = scaley / (numpnts - 1);
   // Loop through creating points objects and adding to array
   for(i = 0; i < numpnts; i++) {
      var p = new Object();
      p.x = spacingx * i + offsetx;
      p.y = spacingy * i + offsety;
      p.focus = false;
      pnts[i] = p;
   }
   return pnts;
}
 
// Distance between two 2D points
// In: points objects in form p.x, p.y
// Out: number
// For mouse-click hit-testing
function distance(p0, p1) {
   xd = p1.x - p0.x; // x-distance
   yd = p1.y - p0.y; // y-distance
   return Math.sqrt(xd*xd + yd*yd);
}
 
// Function to clamp input value to cmin > cmax range
function clamp(val, cmin, cmax) {
   return Math.max(cmin, Math.min(cmax, val));
}
 
// Function to handle dragging of points, with constraints
// In: points array (point x y z focus), mouse-position, point index, clamp-limit
// Out: point object
function dragPoint(pnts, mspos, pindex, clmplimit, dminx, dmaxx, dminy, dmaxy) {
   // Get working point
   p = pnts[pindex];
   // Get previous point x-position
   if(pindex == 0) {
   // If first point
      prevx = dminx - clmplimit;
   } else {
      prevx = pnts[pindex - 1].x;
   }
   // Get next point x-value
   if (pindex == pnts.length - 1) {
   // If last point
      nxtx = dmaxx + clmplimit;
   } else {
      nxtx = pnts[pindex + 1].x
   }
   // Clamp x with minimum distance
   p.x = clamp(mspos.x, prevx + clmplimit, nxtx - clmplimit);
   // Clamp y
   p.y = clamp(mspos.y, dminy, dmaxy);
   // Return point object
   return p;
}
 
///////////////
// Main Loop //
///////////////
 
// Vars for main program loop
var clickCoord = new Array(0, 0);
var oldMouseButton = false;
var points = new Array();
var dragIndex = -1;
 
// The Loop
function (__structure PointsOut)
         main
      (__number MouseX, __number MouseY, __boolean MouseButton,
       __index NumPoints, __boolean InitPoints, __number InitScaleX, __number InitOffsetX, __number InitScaleY, __number InitOffsetY,
       __number DragMinX, __number DragMaxX, __number DragMinY, __number DragMaxY, __number DragMinDistanceX,
       __boolean PinStart, __boolean PinEnd,
       __structure StartColor, __structure EndColor,
       __boolean LoadPoints, __structure PointsIn)
{
   if(!_testMode) {
 
      mousePos = new Object();
      mousePos.x = MouseX; mousePos.y = MouseY;
 
      // Set initial X and Y positions
      if(!points[0] || InitPoints) {
         points = initPoints(NumPoints, InitScaleX, InitScaleY, InitOffsetX, InitOffsetY);
      }
 
      // Load points
      if(LoadPoints) {
         if(PointsIn[0]) {
            for(h = 0; h < PointsIn.length; h++) {
               p = new Object();
               p.x = PointsIn[h].x; p.y = PointsIn[h].y; p.focus = false;
               points[h] = p;
            }
         }
      }
 
      // Start drag on mousedown
      if(MouseButton > oldMouseButton) {
         clickCoord = Array(MouseX, MouseY);
         for(i = 0; i < points.length; i++) {
            if(distance(mousePos, points[i]) < 0.01) {
               dragIndex = i;
               if(dragIndex == 0 && PinStart) { dragIndex = -1; break; }
               else if(dragIndex == points.length - 1 && PinEnd) { dragIndex = -1; break; }
               else { points[dragIndex].focus = true; break; }
            }
         }
 
      // End drag on mouseup
      } else if(MouseButton < oldMouseButton && dragIndex >= 0) {
         points[dragIndex].focus = false;
         dragIndex = -1;
      }
 
      // Do drag
      if(dragIndex >= 0) {
         // Call function to set point position with constraints
         points[dragIndex] = dragPoint(points, mousePos, dragIndex, DragMinDistanceX, DragMinX, DragMaxX, DragMinY, DragMaxY);
      }
 
      // Update mousebutton state for next frame
      oldMouseButton = MouseButton;
 
      // Init output object
      var result = new Object();
 
      // Set output values
      result.PointsOut = points;
 
      // Return result object
      return result;
   }
}

Dunno if this is going to be of any direct use to anyone.

a|x

PreviewAttachmentSize
tb dragpoints test 040610.qtz20.73 KB

gtoledo3's picture
Re: Mouse Interaction 'The Hard Way'

That is killer. Interesting patch.

I think my personal "this would be cool as heck" twist on this would be a macro like this where there circles would be able to bounce against one another.

toneburst's picture
Re: Mouse Interaction 'The Hard Way'

And inertia would be good, too, of course, and multitouch operation. Would probably be better to implement those in a custom Obj-C patch though, I'd have thought.

a|x

gtoledo3's picture
Re: Mouse Interaction 'The Hard Way'

I agree, but I suspect it can be done with stock patches (to some extent, pain in the butt or no). It's easy enough to create boundaries and have something bounce against the boundary in QC. I've just never tried to make the collision point a moving target. It would be fairly simple with two objects (and if only one had to bounce, the other being the movable collider).

... and it would be a real pain to have it work in 3D as well. Certainly, one couldn't use the standard Interaction patch, as it is a 2D thing, and the optional extra control outputs aren't a help for doing anything to affect Z placement.

toneburst's picture
Re: Mouse Interaction 'The Hard Way'

I'm sure you could do the whole thing using stock patches too, but I prefer to code it in JS rather than creating a rat's nest of noodles (if that isn't too mixed a metaphor).

My original purpose for doing this was to create a series of points for a transfer function for raycast volume-rendering, so I really want the points to stay where I put them. On the other hand, there's no reason you couldn't extend the concept to emulate something like some of the controls I've seen for the Lemur, where you can add inertia, springs, gravity and other physical properties to a point.

a|x

gtoledo3's picture
Re: Mouse Interaction 'The Hard Way'

Oh, when I say stock patches, I'm totally including javascript.

In my mind, your patch is actually more complex than doing a simple slider type control that had spring/gravity/whatever, because your points can actually move in X, instead of being parked, and just moving in Y.

Hmm, it will be interesting to see how you use this with your raycast volume-rendering experiments.

dust's picture
Re: Mouse Interaction 'The Hard Way'

the mesh renderer is interactable so that leads me to believe that behind the scenes of apples interactions they are either ray casting to a 2d plane to check for a hit or checking vert by vert. so i suppose in a 2d world making sprites collide would be a good start seeing that there are not that many verts to hit test.

i would be interested in seeing how to hit test with a mask via js.

in reference to hit testing the hard way with java script. check out this method for hit testing moving and adding points.

  1. // GLOBAL VARIABLES
  2. // SegmentAray: Points, Centre, Pivot, Anchor, Rotation, Zoom, Time, Type
  3. // SegmentArray: [{Points: [{X:0,Y:0,U:0,V:0(RGBA))},{X...], Centre: {X:0,Y:0,U:0,V:0}, Zoom: 1, Rotation: 0, Time: 0, Type: "string"}, {Points...]
  4.  
  5.  
  6. if (DistinctQuadStrip == undefined) var DistinctQuadStrip = [];
  7.  
  8. //------------------------------------------------------------
  9. // STRUCTURE FUNCTIONS
  10.  
  11. function cloneObject(theObject) {
  12. var NewObject = new Object();
  13. for (i in theObject) {
  14. if (typeof theObject[i] == 'object') { NewObject[i] = cloneObject(theObject[i]); // recursive
  15. }
  16. else NewObject[i] = theObject[i];
  17. }
  18. return NewObject;
  19. }
  20.  
  21.  
  22. function elementsInObject(theObject) {
  23. var Count = 0;
  24. for (e in theObject) Count++;
  25. return Count;
  26. }
  27.  
  28.  
  29. // This restores the index of an Array mutulated by __Structure
  30. function arrayFromObject(theObject){
  31. var SortedArray = [];
  32. var Count = elementsInObject(theObject);
  33.  
  34. for (i=0; i<Count; i++) {
  35. SortedArray.push(theObject[i]);
  36. }
  37. return SortedArray;
  38. }
  39.  
  40.  
  41. //------------------------------------------------------------
  42. // MAIN
  43.  
  44. function (__structure segmentStructure)
  45.  
  46. main ( __boolean Edit,
  47. __structure DistinctQuadStripFromFile, __boolean LoadDistinctQuadStrip,
  48. __structure SegmentStructureFromFile, __boolean LoadSegmentStructure)
  49. {
  50. var result = new Object();
  51.  
  52. // LOAD SEGMENTS AND SORT SEGMENTS & POINTS
  53.  
  54. if (LoadDistinctQuadStrip == 1 && DistinctQuadStripFromFile) {
  55. DistinctQuadStrip = arrayFromObject(cloneObject(DistinctQuadStripFromFile));
  56. // for (s in DistinctQuadStrip) SegmentArray[s].Points = arrayFromObject(DistinctQuadStrip[s].Points);
  57. }
  58.  
  59. // LOAD SEGMENTS AND SORT SEGMENTS & POINTS
  60.  
  61. if (LoadSegmentStructure == 1 && SegmentStructureFromFile && SegmentStructureFromFile.Segments){ var SegmentArray = arrayFromObject(cloneObject(SegmentStructureFromFile.Segments));
  62. for (s in SegmentArray) SegmentArray[s].Points = arrayFromObject(SegmentArray[s].Points);
  63.  
  64. // CONVERT FROM TRIANGLESTRIP TO DISTINCTQUADSTRIP
  65. DistinctQuadStrip = []
  66.  
  67. for (s in SegmentArray) {
  68. // find centre
  69. var Centre = new Object(); Centre.X = 0; Centre.Y = 0;
  70. for (p=0; p<SegmentArray[s].Points.length; p++) { var Point = SegmentArray[s].Points[p];
  71. Centre.X += Point.X
  72. Centre.Y += Point.Y
  73. }
  74. Centre.X /= p;
  75. Centre.Y /= p;
  76.  
  77. // find rotation
  78. Angles = [];
  79. for (p in SegmentArray[s].Points) { var Point = SegmentArray[s].Points[p];
  80. X = Point.X - Centre.X;
  81. Y = Point.Y - Centre.Y;
  82.  
  83. var Angle = Math.atan2(Y, X) //- Math.PI / 1800;
  84.  
  85. Angles.push(Angle);
  86. }
  87.  
  88. // find order
  89. Order = []
  90. var first = 10; var second = 10; var third = 10; var fourth = 10;
  91. Index = 99;
  92. for (a=0; a<Angles.length; a++) {if (Angles[a] < first){ first = Angles[a]; Index = a; }} Order.push(Index);
  93. for (a=0; a<Angles.length; a++) {if (Angles[a] < second && Angles[a] > first){ second = Angles[a]; Index = a; }} Order.push(Index);
  94. for (a=0; a<Angles.length; a++) {if (Angles[a] < third && Angles[a] > second){ third = Angles[a]; Index = a; }} Order.push(Index);
  95. for (a=0; a<Angles.length; a++) {if (Angles[a] < fourth && Angles[a] > third){ fourth = Angles[a]; Index = a; }} Order.push(Index);
  96.  
  97. // set order to 1, 2, 3, 0
  98. // set UV to 1,0, 1,1 0,1 0,0
  99. var P0 = new Object();
  100. P0.X = SegmentArray[s].Points[Order[1]].X
  101. P0.Y = SegmentArray[s].Points[Order[1]].Y
  102. P0.U = 1;
  103. P0.V = 0;
  104. DistinctQuadStrip.push(P0)
  105. var P1 = new Object();
  106. P1.X = SegmentArray[s].Points[Order[2]].X
  107. P1.Y = SegmentArray[s].Points[Order[2]].Y
  108. P1.U = 1;
  109. P1.V = 1;
  110. DistinctQuadStrip.push(P1)
  111. var P2 = new Object();
  112. P2.X = SegmentArray[s].Points[Order[3]].X
  113. P2.Y = SegmentArray[s].Points[Order[3]].Y
  114. P2.U = 0;
  115. P2.V = 1;
  116. DistinctQuadStrip.push(P2)
  117. var P3 = new Object();
  118. P3.X = SegmentArray[s].Points[Order[0]].X
  119. P3.Y = SegmentArray[s].Points[Order[0]].Y
  120. P3.U = 0;
  121. P3.V = 0;
  122. DistinctQuadStrip.push(P3)
  123. }
  124.  
  125. var sObject = new Object();
  126. sObject.Segments = DistinctQuadStrip;
  127. sObject.ActivePoint = -1;
  128.  
  129. result.segmentStructure = sObject;
  130.  
  131. return result;
  132. }
  133. }

usefuldesign.au's picture
Re: Mouse Interaction 'The Hard Way'

Hey dust this code looks interesting, do you have a QC patch showing how to use it?

Hit testing with mask could be achieved in CI easily couldn't it? Just multiply the masks and somehow test the resultant image for any non-black pixels which would indicate location(s) of a hit. That's assuming you have the objects as images not as structured info obviously, so would requite RII of the data which would be slow if your talking 3D objects. Sorry if I'm missing the point here.

usefuldesign.au's picture
Re: Mouse Interaction 'The Hard Way'

Thanks for the post toneburst.

The JS code is interesting to me and it works on Leopard in the comp you posted — when I copy and paste the JS code from your patches into new JS patches though QC it rejects the code.

I'm wondering why I get a "Main function missing error in the parsing window". Also how do you get those Inputs Outputs up/down selectors in the settings inspector. This is crazy, man!

dust's picture
Re: Mouse Interaction 'The Hard Way'

i don't have an example but i know it works ;) this code is from a friend who doesn't really participate here on kineme. i have been studying it today. it took my friend the better part of a year to develop a quad mapping system in quartz. thought tb or someone would know how to use it. i'm still trying to figure it out. the recursion and cloning functions are pretty straight forward. basically you input your segment data via a .plist (kineme file structure) and push that data into a js array because you cannot straight copy structure in js without losing the order. once you plot points you can edit or move them around (interaction) or add more. its kind of like how bezier's work with control points or handles but for quads that can be textured. its to help you project onto surfaces that are not flat.

toneburst's picture
Re: Mouse Interaction 'The Hard Way'

Sounds like you're using the old Tiger version of QC there, usefuldesign. They changed the way JS patches work between the Tiger and Leopard versions of Quartz Composer. Dunno why you're seeing the old JS patch editor in Leopard though...

a|x

toneburst's picture
Re: Mouse Interaction 'The Hard Way'

Looking forward to giving this a go... :)

If you use arrays internally in the JS code, instead of objects, you can ensure things remain correctly sorted. I don't know if you noticed, but I used a mixture of objects and arrays in my code above- objects for each point, so I could use named properties, and an array for the points collectively, so they would stay (and be output) correctly-sorted.

a|x

cybero's picture
Re: Mouse Interaction 'The Hard Way'

The following is copied and pasted from the code in my QC.

The only thing I've done is to get rid of some spaces within {}.

Don't know if that made any real difference. This code should work

// GLOBAL VARIABLES
// SegmentAray: Points, Centre, Pivot, Anchor, Rotation, Zoom, Time, Type
// SegmentArray: [{Points: [{X:0,Y:0,U:0,V:0(RGBA))},{X...], Centre: {X:0,Y:0,U:0,V:0}, Zoom: 1, Rotation: 0, Time: 0, Type: "string"}, {Points...]
 
 
if (DistinctQuadStrip == undefined) var DistinctQuadStrip = [];
 
//------------------------------------------------------------
// STRUCTURE FUNCTIONS
 
function cloneObject(theObject) {
 var NewObject = new Object();
     for (i in theObject) {
     if (typeof theObject[i] == 'object') {NewObject[i] = cloneObject(theObject[i]); // recursive
      }
     else NewObject[i] = theObject[i];
    }
    return NewObject;
}
 
 
function elementsInObject(theObject) {
  var Count = 0;
    for (e in theObject) Count++; 
  return Count;
}
 
 
// This restores the index of an Array mutulated by __Structure 
function arrayFromObject(theObject){
 var SortedArray = [];
   var Count = elementsInObject(theObject);
 
  for (i=0; i<Count; i++) {
        SortedArray.push(theObject[i]);
    }
 return SortedArray;
}
 
 
//------------------------------------------------------------
// MAIN
 
function (__structure segmentStructure)
 
    main ( __boolean Edit,
            __structure DistinctQuadStripFromFile, __boolean LoadDistinctQuadStrip,
           __structure SegmentStructureFromFile, __boolean LoadSegmentStructure)
{  
    var result = new Object(); 
 
  // LOAD SEGMENTS AND SORT SEGMENTS & POINTS
 
  if (LoadDistinctQuadStrip == 1 && DistinctQuadStripFromFile) {      
        DistinctQuadStrip = arrayFromObject(cloneObject(DistinctQuadStripFromFile));
   //    for (s in DistinctQuadStrip) SegmentArray[s].Points = arrayFromObject(DistinctQuadStrip[s].Points);
  }
 
   // LOAD SEGMENTS AND SORT SEGMENTS & POINTS
 
  if (LoadSegmentStructure == 1 && SegmentStructureFromFile && SegmentStructureFromFile.Segments){var SegmentArray = arrayFromObject(cloneObject(SegmentStructureFromFile.Segments));
      for (s in SegmentArray) SegmentArray[s].Points = arrayFromObject(SegmentArray[s].Points);
 
    // CONVERT FROM TRIANGLESTRIP TO DISTINCTQUADSTRIP
     DistinctQuadStrip = []
 
     for (s in SegmentArray) {
         // find centre
         var Centre = new Object(); Centre.X = 0; Centre.Y = 0;
         for (p=0; p<SegmentArray[s].Points.length; p++) {                    var Point = SegmentArray[s].Points[p];
               Centre.X += Point.X
              Centre.Y += Point.Y
          }
         Centre.X /= p;
            Centre.Y /= p;
 
      // find rotation
           Angles = [];
         for (p in SegmentArray[s].Points) {                    var Point = SegmentArray[s].Points[p];
               X = Point.X - Centre.X;
             Y = Point.Y - Centre.Y;
 
               var Angle = Math.atan2(Y, X) //- Math.PI / 1800;
 
               Angles.push(Angle);          
            } 
 
      // find order
          Order = []
           var first = 10; var second = 10; var third = 10; var fourth = 10;
           Index = 99;
           for (a=0; a<Angles.length; a++) {if (Angles[a] < first){ first = Angles[a]; Index = a; }} Order.push(Index);
          for (a=0; a<Angles.length; a++) {if (Angles[a] < second && Angles[a] > first){ second = Angles[a]; Index = a; }} Order.push(Index);
            for (a=0; a<Angles.length; a++) {if (Angles[a] < third && Angles[a] > second){ third = Angles[a]; Index = a; }} Order.push(Index);
         for (a=0; a<Angles.length; a++) {if (Angles[a] < fourth && Angles[a] > third){ fourth = Angles[a]; Index = a; }} Order.push(Index);
 
      // set order to 1, 2, 3, 0
     // set UV to 1,0, 1,1 0,1 0,0
          var P0 = new Object();
         P0.X = SegmentArray[s].Points[Order[1]].X
            P0.Y = SegmentArray[s].Points[Order[1]].Y
            P0.U = 1;
            P0.V = 0;
         DistinctQuadStrip.push(P0)
           var P1 = new Object();
         P1.X = SegmentArray[s].Points[Order[2]].X
            P1.Y = SegmentArray[s].Points[Order[2]].Y
            P1.U = 1;
            P1.V = 1;
            DistinctQuadStrip.push(P1)
           var P2 = new Object();
         P2.X = SegmentArray[s].Points[Order[3]].X
            P2.Y = SegmentArray[s].Points[Order[3]].Y
            P2.U = 0;
         P2.V = 1;
            DistinctQuadStrip.push(P2)
           var P3 = new Object();
         P3.X = SegmentArray[s].Points[Order[0]].X
         P3.Y = SegmentArray[s].Points[Order[0]].Y
         P3.U = 0;
         P3.V = 0;
         DistinctQuadStrip.push(P3)
       }
 
       var sObject = new Object();
        sObject.Segments = DistinctQuadStrip;     
        sObject.ActivePoint = -1;
 
     result.segmentStructure = sObject;
 
      return result;
 }
}

& this what the patch should look like - attached image file

PreviewAttachmentSize
JSPatchStructure.png
JSPatchStructure.png53.2 KB

usefuldesign.au's picture
Re: Mouse Interaction 'The Hard Way'

Sorry I was looking at another other comp entirely while reading the thread and I though it was yours — this dragable button comp posted by yanomano on 2007-11-28.

Your JS is okay on my Leopard although it took me 16 attempts (stylus-downs) to move five points which is approx 30% hit rate :/.

I know this is completely OT but do you happen to know about this variant of the Javascript settings inspector and how to invoke it. Like I said the code won't copy and parse into stock JS patch for me on 10.5.8.

PreviewAttachmentSize
Picture 4.png
Picture 4.png106.95 KB
Picture 7.png
Picture 7.png115.55 KB
Drag_able sprite button.qtz17 KB

smokris's picture
Re: Mouse Interaction 'The Hard Way'

usefuldesign.au wrote:
I know this is completely OT but do you happen to know about this variant of the Javascript settings inspector and how to invoke it. Like I said the code won't copy and parse into stock JS patch for me on 10.5.8.

As @toneburst mentioned, the version of the Javascript patch with the inputs/outputs counters on top of the Settings panel is from Tiger.

QC Leopard and later automatically detect that a Javascript patch instance was created in Tiger, and use the appropriate old version of the patch (probably since it would be nontrivial to automatically update the code to use the new format).

You can create new instances of the Tiger version of the Javascript patch by copying and pasting it. (I wouldn't recommend this, though, as there's no guarantee how long the Tiger version will still be around.)

gtoledo3's picture
Re: Mouse Interaction 'The Hard Way'

Also, just because the old style Inspector pulls up doesn't mean the patch works (but maybe it does?). Note the "incorrect or missing main function" note. I've definitely opened up some old patches that no longer work with javascript code that worked in Tiger.

usefuldesign.au's picture
Re: Mouse Interaction 'The Hard Way'

Okay now I get what tb was saying thanks to you smokris, cheers. So much confusion my end this month!!

usefuldesign.au's picture
Re: Mouse Interaction 'The Hard Way'

gtoledo3 wrote:
Also, just because the old style Inspector pulls up doesn't mean the patch works (but maybe it does?). Note the "incorrect or missing main function" note. I've definitely opened up some old patches that no longer work with javascript code that worked in Tiger.
In this case the code works in the old patch but not in the Leopard patch. Yah — I'll stay well clear of Tiger JS patch though, hey it was just a case of (unknown) feature envy on my part.

gtoledo3's picture
Re: Mouse Interaction 'The Hard Way'

OT: In the case I'm talking about, it was a javascript demultiplexer that no longer worked (actually, I've done a javascript demultiplexer before as a test as well)... but it left me really wondering WHY, as a demultiplexer has been around since Pixelshox.

toneburst's picture
Re: Mouse Interaction 'The Hard Way'

There's been a builtin Demultiplexer patch since QC 2.x though, so a JS version may no longer be needed.

a|x

gtoledo3's picture
Re: Mouse Interaction 'The Hard Way'

Yeah, that's my point... I saw it, and it had me scratching my head, because it was a Tiger era qtz, and I thought "certainly the demultiplexer was around", though I was told by someone it wasn't (I do think it was). Someone went through a crapload of trouble to do the javascript demultiplexer in Tiger, and then it didn't work because of js updates between Tiger and Leopard.

It was in the Pixelshox patch list... I don't know if it was in "QC1" though. I never used the demultiplexer Tiger, as my activities were more like video->filter->Billboard at that point. I would be surprised if it hadn't been around in QC2 (or even QC1).

A JS version is definitely not needed, as far as I can see (though there is a pretty good example of how to make one that works in QC still somewhere in the developer examples...).