Kineme 3D Alpha 9

Thanks for the alpha. However, there seems to be a slightly annoying bug with the plane generator: it crunches UVs or something, hence the upper right part of the image texture missing. See example comp for full understanding.

Good demonstration, thanks -- I'll take a look at this

Thanks Chris.

I am going to be working with this ALL weekend.

I've noted that it is impossible to multiplexe 3D object (of the KnM datatype) This is quite a normal behaviour, since you are using a custom type. Could you provide a solution to multiplexe objects (esp. multiplexing 3D transformations like noize, twist...) any suggestions welcome.

That's interesting to note -- apparently Virtual Multiplexers output something weird that I'm not checking for -- I'll take a look at that, thanks for pointing it out :)

In the mean time, a Virtual Splitter between the Mux and the renderer seems to do the trick.

sorry, that composition uses a patch from the upcoming GLTools 1.0 release, so it's useless to you ...

Here's what the inside looks like though:

Preview	Attachment	Size
	3DMultiplexer.png	263.05 KB

Thanks....

I ended up giving GL Ortho a workout, and didn't even hardly tap into this. The example comps are pretty exciting.

The one thing that pops up in my mind, as I work with this, and as I look at a lot of 3D renderings.... There really needs to be some way to create shadows that interact with the lighting patch. Maybe just a lighting/shadow patch that is entirely new? I would think that would make this whole setup that you have going very visually arresting. I remember already posting about this in a different context, and Chris saying that it is pretty impossible.

I was making a really simple composition with sprites, with varying positions on the z-axis.... nothing to do with Kineme-3D exactly, but it started being very striking to me that within QC, a sprite can't even create shadow on another sprite... It seems like it would be reasonable to calculate what the shape and intensity of the shadow would be, doesn't it? Maybe with a clever macro using math patches, filters to angle/deform and "darken" the original object or shape so that it "looks" like a shadow, and sending the image output to two renderers. It seems like this could all be derived somehow.... ie, the shadow from a circle would look "oval" according to the angle/distance from the light source.

So, I have been messing around with the GL Tools, and 3D transforms, the "bend box", etcetera, to stretch out images/objects, darken them, and then fake a shadow. I haven't got that far, and it seems like I am probably barking up the wrong tree!

Of course, I am sure that isn't the way you guys would approach it, lol. That is all my limited intellect on this subject could fathom as a method of achieving this.

....

I think that this is really a great set of patches, that I know I'm still not even touching the surface on, because of the way it can potentially interact with the GL Tools, and the ability to have these "shader program" inputs... which I don't even know what that quite entails yet!

I'm a little confused about the smooth on the object loader, versus the actual smooth patch. I'm also struggling with figuring out a rhyme or reason of how to setup the 3ds and obj type files that seem to have multiple "shapes" within them. I can figure out that I can sort by using structure tools, and then manually place each of the pieces around in space, but I'm never quite sure if I am reassembling it correctly.

The smooth is a great function, and would be "the bomb" if it could somehow run faster. As is, I basically only use it when rendering.

Is it "wrong" to put the smooth after a bunch of bend box and deformation type of stuff? Obviously, it will render and look cool, but am I doing something stupid here? There is a such a fine line between 30fps and n/a~0.5fps. ;oP

All in all, I think this is amazing and a genius effort. It is obvious that Apple is planning 3D support considering the QC mail list leak, and I hope that doesn't steal your thunder.

Is that running man visual from the spoiler clip a md2 with gl polygon, or something like that?

When I look at that, and consider the experiments you guys have done with OpenCV... I keep thinking that there is somehow a way of mixing the two...

My father is a retired graphic artist, and has received many awards, etcetera... when I showed him that he was saying "what these guys need to do is create a graphic where the running wireframe man turns to flesh". This is a guy that is old school, and just rolls his eyes when I explain how it doesn't quite work that way. But he does have a keen eye, and the suggestions of somehow creating shadows and mixing this wireframe and openCV stuff are really his.

Sorry for such an unstructured, barrage of an email!

Also, is this something that would be expected to load?

I don't seem to get a structure from it.... This is google collada file.

Incidentally, I have some friends who are Architects, and would they have the ability to make any files that could be used with this... they are using Autodesk and Revit(?), and something else called Microstation... I'm actually going to just do some more investigation on this myself, but I thought one of you might know off hand.

Regarding shadows:

To calculate them, you need to know light positions, and all object positions (so that you can do the depth buffer stuff necessary to find where the shadow lies) -- an alternative method is using the accumulation buffer with some counters (read up on Shadow Volumes), but QC doesn't create any accumulation buffers, so this cannot work (except in external apps that happen to provide one for us -- no such apps exist today to my knowledge, because no QC patches use them, so they're a waste of ram). The former solution is possible if we make a Scene object that has all the rendered objects in it, so that it can re-render the scene itself multiple times to work out where shadows go, etc. But then it starts looking like Particle Tools ("WTF are all these inputs?! And why doesn't anything work automatically!?"), which I want to avoid. The math for shadows has long been known, calculated, and documented, it's just doing it quickly that requires these hacks, which require special design considerations that QC isn't very helpful with.

As long as you don't Normalize the objects in a multi-object file, they'll be presented in the same space as they're saved in. Normalizing currently works per-object, not per-file, so normalizing it scales each piece differently. Centering requires normalization (though now that I think about it, that's not really necessary, just the way I coded it...), which breaks it too.

Regarding Smooth: Smooth on the loader performs the smooth step at load time, and that's it -- if you're just rendering the object, it never has to re-smooth it, so it looks pretty, and goes very fast. the separate smooth patch re-smooths, and is required after deformations (since those break the normals, most of the time). So if you're deforming the object, don't smooth on load (it's thrown out anyway), smooth afterwards -- Caveat: in Alpha 10, smooths are much faster (in alpha 9 and earlier, they take (Number_Of_Points ^ 2) time, so it gets exponentially slower (each additional point makes it take 2x as long) as the models have vertices. In Alpha 10, I've modified the algorithm to take (Number_Of_Points) time, so doubling the number of points doubles the amount of time it takes. This works by by a Smooth Cache (internal, nothing outside changes), which is created by the Smooth input on the loader, but not the Smooth patch -- so, if Smooth is disabled on the loader, smooths later on have to fall back to the Points^2 method, because the cache doesn't exist. Still working out the details for this, to make it painless for everyone.

So, for the "is it wrong" part, in Alpha 9, sort-of, because it kills performance. There's no way around it though. In alpha 10, that's the proper way to do it, so just leave it like that I guess :)

The running man (provided by Yanomano) is a 3DS file, with no smoothing (each polygon has 1 normal, instead of 3). Putting it in GLTools' LogicOp patch makes for some cool rendering feedback along the edges :)

Mixing CV, GL, and 3D will be a kind of heroic effort that will be amazing :) It's possible now, but there's still a lot of ground to cover to make it all simple and easy. Snow Leopard's OpenCL will likely do a lot to simplify this and integrate it. It's a rather logical combination of technologies, just dissimilar enough to make integration rather tricky :) (And OpenCV's braindamaged API largely foils any attempt to simplify it)

Turning the running man from poly to flesh is pretty simple: Just use shaders and clever textures. Sub-surface scattering shaders (necessary for convincing-looking flesh) are way beyond the scope of this reply though ;)

I saw this yesterday...

The part of the clip with the trees and the waving floor really caught my eye... reminded me instantly of the example with the waving plane and sphere, as well as some stuff I was working on with waving plants.

I think it wouldn't be totally impossible to do some stuff bordering on this with the Kineme3D. But the realism on this is pretty awesome.

I have never worked in Blender, Maya, whatever... I am guessing that is the kind of stuff that this guy is using. Mixing with real stuff? I have no clue.

The lens focus effects and textures/shaders on this stuff is great.

Wow that video is quite interesting!

I am not very clever when it comes to 3D, but for 10 years Ive dreamed of interesting realtime 3D, and have used a variety of VJ programs on Windows and more recently Mac. The Kineme 3D stuff has made this year very interesting to me, but I have to be realistic about what can be achieved in realtime. All these years I have seen impressive stuff in games, and wanted the same for realtime visuals. I always imagine that more is possible than it really is, in terms of visual quality, lighting, shadows etc.

Hopefully I am finally working on a sane approach to this, which is to use a specific style for realtime 3D that gives good performance, and covers up some of the weaknesses in what we can easily achieve in realtime with present tools. In other words I have to stop imagining that I can get that lovely raytraced 3D look in realtime stuff. I can still hope that this will happen one day, but for now if I want that look I will have to render in non-realtime using other apps, and then use that video in quartz composer. By prerendering various different 3d animations, camera moves etc, and then triggering those different videos using quartz composer or VDMX, and combining with actual realtime 3D, the best of both worlds is almost possible. Cheat :) Blender is attractive because it is free, but I was intimidated by its UI, so I went with Cheetah3D instead, which has some interesting rendering options but isnt too complex, and doesnt cost too much. Plus I can use it to make models for realtime use in qc as well. I also went crazy and bought poser and vue for 2 specific types of 3d creation & rrendering (humans and landscapes) but Ive failed to achieve anything good with them yet, Im all out of breath trying to jog up the learning curves. Good 3D aint easy, so many different aspects to get right, but Im giving myself 20 years to get there.

Oh I forgot to say that this website shows a variety of primitive things I was experimenting with using Kineme 3D earlier this year:

http://www.mutantquartz.com/

Unfortunately I got frustrated with my own lack of ability and time, and sidetracked by making some farily naff itunes visualizers using the standard qc cubes and cylinders (as itunes didnt like qc plugins), then sidetracked further by poser and vue. Since I made those videos the performance of VDMX has improved, I got a Mac Pro, and ATI 3870 graphics card just turned up for me today, so I will be returning to this stuff soon. My ultimate aim is supposed to be using the jazzmutant lemur to control 3d puppets in realtime. I just have to hope my skills improve, that my imagination falls into line with what I can actually achieve, and that the economic gloom does not cause me lose my job. If I lost my job then Id actualy have time to follow these dreams, but all the shiny equipment that was bought on credit would get taken away. Either way, I am extremely greatful to Kineme for breathing new life into my 3D dreams this year. I noticed that Apple email leak too, interesting times ahead eh, shame their NDA's leave people like me clueless about where this technology that I pin so many hopes to, is going in future.

I like the effects + design here. Some good ideas.

I can say though that this is 100% possible in realtime. Perhaps even with QC with some pretty hardcore plugin making, but you'd need real dedication.

If you really want to do stuff like this, I recommend downloading a few decent 3d apps (no idea which are best these days, it's been over 5 years since I did any serious non-realtime 3d) and go through the tutorials. Many seem to offer a 'learning edition' or similar for free.

They'll look pretty daunting at first, but when you get the basics learning a 3d app is good fun. It's worth trying a few though - depending on how you like to work, some apps will be much more suitable than others.

If you're serious about doing stuff like this in realtime, I suggest following the demoscene. Most demo stuff is windows only, but some groups are doing work of this quality in realtime.

That's what I was thinking as well -- simple global illumination (for the realistic soft shadows) have been possible for a couple years, if you're resourceful and clever (in opengl and glsl), and the rest of the stuff is simple vertex distortion/animation/skinning, which has been possible for decades (And now really fast, with vertex shaders).

I don't know that QC would be the best place to recreate this though, since, as psonice said, you'd have to do some hardcore plugins (and by hardcore, I mean you'd have to basically throw out all of QC's pipeline, and manage tons of opengl stuff + models yourself). It'd be a lot of work, but possible none the less.

Hmmm, so we aren't on Alpha9 anymore, but I didn't feel like starting a new thread :o) I had the idea of adding a panorama background to the Kineme3D taunt scene, and threw in some particles, so that they look like they are sparks flying up from a valley. The start of an interesting technique I think, as far as using the panorama for the landscape... low overhead way of getting a realistic scene.

Are there any actual "panorama" movies, that actually translate like a panoramic shot does? Not talking about the scrollable things that Apple makes. This would have to look like an actual panoramic shot does in "2D", so that I could attach a movie loader to the sphere, and get a realistic moving background? I don't know if I am talkin' greek here...