jwatte:

Only way something like that would work virtually is through a body suit or the Matrix.

I said "overcoming the design limitation," not "solving the unified field theory of gravity." Although that would be cool, too :-)
You work around this limitation by designing around it. Kind-of like my design above (which btw can be used on non-motion-controlled games, too, when you want forward kinematically animated characters to have physics act on them).

From walking the floor at E3, I can tell that this is the year of the input peripheral. Shotguns, skateboards, turntables, cowbells... MORE COWBELL!

I think Rock Band/Guitar Hero has opened up that space and shown that with compelling gameplay people will buy external's. You can almost count on any gamer having a guitar or Drum Kit...(plug)

I think to overcome the force feedback they should have some Chuck Jones type arm with a mallet pop out of the device and whack the user back.

Ok...  FPS controls in the post-Natal age.  Your body trunk becomes the analog joystick.  You lean forward to move forward, you lean left or right to strafe left or right.  You twist your trunk to rotate.  Hold your fists like a rifle, and the game can use that line to infer what you are aiming at on the screen(this will need autoaim... or analog type motion, it wouldn't be sensitive enough that you would just point).  Shake your body and arms like you are recoiling from rifle fire, to shoot your gun.  Weapon selection would be moves like...  Hold your hands out in front of you like you are holding a pistol.  Hold your arms like you are holding a rifle, or a rocket launcher.  Make a throwing motion to throw a grenade.  Jump to jump.  Go down on one knee to hop behind cover.

Once again, the players body and gravity, and imagination will provide a ton of what it is you want to provide with force feedback in this scenario.  I don't think you need to go for a Matrix like simulation to make a game like that a blast.

Sounds like a plan! But what about the .00001% of the universe that plays games other than shooters? Following the above setup:

RTS controls in the post-Natal age: say goodbye to clumsy mice and inefficient keyboard. You now use a combination of voice and hand gesturs to dish out commands at a lightning pace. (But not to the point of being the only purpose for existing, *cough cough* EndWar *cough*). You can draw rectangles around units to select them, and either use voice commands (attack the front wall) or pointing gestures to show the way. Since it would be more of a touch screen type interface, there will be a mouse cursor type icon on screen that moves when you move your hand, and turns color when you're pointing (as in holding the mouse button down). Suddenly, instead of doing a spider crawl accross the keyboard and franticly waving the mouse around on the table, you're pointing and thrusting your hand while shouting orders, just like a real commander!

GPG unveiled a new Supreme Commander 2 trailer at E3 (see here), and it has a vague "2010" release date. What perfect timeing - Chris (CEO of Gas Powered Games, for you non-Commanders), are you reading this? Good - now go bug your Registered Developer contact!

UberGeekGames:

Barkers Crest:

Many moons ago I had a CompSci professor tell me a computer will never need more than 512mb of RAM.

A friend likes to tell a story that when shopping for a new computer, the salesman said "This baby has a whopping 16mb hard drive! You'll never be able to fill it up, ever!". You gotta love the pace of technology advances.

Nick Gravelyn:

At PAX last year a company was demoing a vest that could be worn that was full of little rumble packs. They demoed it with a FPS such that when you were shot you could feel where the character was hit.

I remember that! I think it was advertised in GameInformer magazine - if I can find a link I'll post it here.

The time line sounds reasonable, but I'm hoping it'll be 1 year accelerated. Of course, maybe it's better for Microsoft to force us to pace ourselves when dishing out cool new toys - my head might explode from all the awesomeness if we get Avatars and then Natal right afterwords! And then there's the ZuneHD...

[edit]: @GregA: That's hilarious, and you'd probably be right! How about modifying their Avatar based on how well they rank, and make them fatter and fatter as they fall to the bottom of the scoreboard for added realism?

I just thought of something that might throw a wrench into all of our plans: the average gamer is probably barely fit enough to get up and swap discs in the Xbox. We'll need to very gradually ramp up the physical activities, since we don't want to give all our good customers heart attacks.

I was reading about it in an article in an older mag, "but 16 - and even 24 mb are fast becoming standard issue[talking about RAM for Win95" "The most muscular microprocessor is 166 MHz" back in June 1996, 26 MHz was fast.
On topic: I'm sure MS if Natal is finished, will have stuff for it in XNA, this technology detects movement in an area. It would probably be hard to program...

i wonder how well project natal would work as a motion capture device for game assets.

SmileDog:

On topic: I'm sure MS if Natal is finished, will have stuff for it in XNA, this technology detects movement in an area. It would probably be hard to program...

This is the thing people don't seem to realise. Even detecting a simple circular hand movement would involve complex mathematics, curve recognition etc. It's seems the device will need to do most of that work for you, but then how inflexible will that make it for the programmer in terms of recognising variations in templated movements.

A poster above mentioned something about karate movements etc. Recognising the extremely complex and subtle movements of the human body performing these types of activites would be enormously complex and is probably something fulltime researchers have been working on for like 20 years.

There has to be a compromise somewhere. If the device just dumps raw movement data then it would be no use for community games programmers because it would be too complex to make anything from. So the device would need to return some type of packaged movements which inherently removes flexibility.

Craig,

Actually, I have beens studying it pretty intensly since the announce, and finding a pattern like that should be pretty doable with a neural net trained to recognize patterns.  The only issue I am concerned about now, is making the thing run fast enough in XNA to be worthwhile (the xna software I am using right now takes a minimum 3 seconds to recognize face patterns), and being able to get at the data stream that Natal generates so that I can train the neural nets.

This morning I was hoping to be able to make a boolean test, and with a bunch more reading, tinkering with the available software and a day later, I know I can classify types of motion.  The math behind it all is transparent to the technician...

There are a bunch of huge massive overwhelming assumptions in my thinking.  They are mostly about microsofts willingness to let outside developers tinker with Natal.... 

I am all hopped up on on hyperbole right now, but Natal and similar technologies will be as big a change in computing as the internet and the gui were.  I kinda think Microsoft will want to lock down the IP and techniques for as many years as they can get away with.

But anyhow, welcome to the beginning of the computer vision age...  I had no idea how far it had progressed as a science.

Very cool Greg - I wish I could do cool things with NNs like that! While you're at it, maybe you could whip up a SkyNet, and use a P2P server scheme so all of us could help advance it with the power of our computers and Xboxes? ;-)

On topic, I would hope that Microsoft at least gives us a starter set. I think they will probably give us a bone structure of the person in front of the camera - how else could they get the Avatar working like that in the live demos? And if we also get the raw data then we could apply our own NN-powered prediction with it.

Since your structure depends so much on processing power and the Natal unit has it's own processor, maybe we can offload Natal-specific commands to it? That would be all kinds of cool (well, more so than it already is!).

That is cool. I wasn't aware NN's had such application for something like this. Sounds good :)

UberGeekGames,

You jest, but look at the tiny amount of information you can find about DARPAs Petavision.  Darpa wants to give all those UAV's they have been flying around an AI visual processing system.

I found one project, where they track all the people who walk through an area, and they can classify ~what~ it is the people are doing while they are in the FOV of the camera.

Autonomous robots are programmed with NN's and apparently there are some fairly clever game agents, for chess and backgammon made with neural nets.

I found one paper where they were using neural nets to classify the little randomly generated genetic assembly code structures a genetic AI made...  That is, they were using one AI agent, to train another AI agent, that wrote software...

I was a huge skeptic of Kerzweils singularity until yesterday...

*cough* bring this back on topic or the thread will get locked *cough*

In the demo I saw, the guy has his avatar on the screen. He then says "ever wondered what the bottom of an avatar's shoe looks like? Well... WHABAM" and then the avatar's torso turns around 360 degrees, his right arm goes awkwardly behind his back, and his left arm shoots through his chest. Also, we get to see the bottom of the shoe.

looks like there is some work to do for microsoft :P

Nick Gravelyn:

*cough* bring this back on topic or the thread will get locked *cough*

Please don't, Nick - there's a lot of really good creative discussion going on. You don't want to stifle the creativity of the community, do you?? ;-) (Ok, the SkyNet/NN subtopic may have been pushing it a just a little...)

<On Topic>
I had another idea for a use of Natal: a mouse. Slightly expanding on my RTS control idea, you simply point at where you want the cursor to be. In addition to RTSs, this would open up a ton of room on the Xbox for all different types of genres - I'd bet there'd be a huge market for a solitaire game that utilizes it! Or maybe you could have a PIM app, where you can slide information, photos and music around (like in CSI but more realistic)?

[edit]: @Daniel: This is only a beta version. Of course there will be some odd bugs like that. I was truly impressed with how well it worked, even with the occasional spazziness of the Avatar. Also, remember that this was on a big stage. I'm sure the dark lighting + lots of really bright, moving LEDs in the background didn't help the camera. And from what I can tell, the live demos weren't the programmers, who would know exactly how to move to avoid doing anything bad. (You know, like the first time you give your game to someone else and they manage to crash it in 5 seconds? :P )

Boy you lot like making things complex!

All we're talking about is detecting relative motion!  It's pretty simple provided you've got enough processor power.  My per theory is:

Part 1 - capture engine
1) The stereoscopic camera input is used to produce a set of positional vectors at time t=0
2) At time t=1, another set of vectors is produced, and a set of 3 Taylor series functions derived to describe the delta between t=0 and t=1
3) Time t=2 and another set is produced of vectors is produced.  These are used to verify that the functions provide the correct positions
        if not, then the "gesture" is complete.  Pass the starting vector, Taylor functions and delta t for the gesture
        if yes, the "gesture" is ongoing
4) Go back to capturing frames and generating functions to describe the motion

Part 2 - Gesture -> Action engine
1) A set of vectors and a function are delivered by the capture engine.  These are compared to a pre-defined series which describes the relative motion of a given vector (say the one for the right hand) and decides which preset is the closest match to the data delivered by the capture engine
2) The closest matching preset gesture triggers a framework event which is passed to the game
3) Go back and start on the next available dataset

Part 3 - Game Initialization
1) A set of gestures is captured via the Natal SDK.  These are "compiled" into positional vectors and a Taylor series
2) These are passed to the Natal unit, each with an ID used in event triggers
3) The game subscibes to Natal events which will be raised by the framework

Part 4 - In-game execution
1) The game tells Natal which of the precompiled gestures to watch for
2) The game waits for events
3) The game acts on triggered events

This means you can define "left foot back", "eyebrows up" and other simple gestures, as well as more complex gestures like "stamp on enemy", "reload gun", "pick up glass" or whatever, giving you overall avatar control as well as complex movements unique to an individual game.  All we're interested in is the rate of change of the components of a set of position vectors.

The one question I do have - can we have network games which let you have 5 xboxes linked - forward, back, left, right and up views all available so I can go on a drunken Jedi-style rampage?

Regards
Mike

That's how I'd do it anyway ;0)

Does the XBox already have a camera and can we access it in XNA?  Can you hook up two of them?

Zenfar:

Does the XBox already have a camera and can we access it in XNA?  Can you hook up two of them?

Yes, nope, and nope.

Zenfar:

Does the XBox already have a camera and can we access it in XNA?  Can you hook up two of them?

Yes. But we don't have any access to it in XNA. :-(

And I'm pretty sure you can't hook > 1 up at a time.

[edit]: Whoops, beaten!

I would say if they gave us access to the web camera then we could prototype some Natal-like games, but I also fear somebody would make My Perfect Video Massage...

It'd be near impossible to prototype anything given that two normal webcams are still paltry compared to the two IR cameras that Natal boasts. Not to mention Natal runs a custom processor to likely do analysis or transformation of the data by itself.

That said, you could easily hook a couple webcams up to a PC, find a generic .NET webcam library, and get to work that way.

It's neat and all, but still strikes me more as a gimic to get causual players interested in games rather than something "better". I get a feeling that I'll be able to pile this next to my Virtua Boy, 32X, and NES R.O.B.

Kris Steele:

It's neat and all, but still strikes me more as a gimic to get causual players interested in games rather than something "better".

In games for which Natal will be the only input, I agree it's a more casual move. I'm sure I'll love the first Scene It to come out using it as well as some of the other games. However I see this as a big way to improve upon current games by adding more depth to their controls in a more natural way.

My brother tossed this one out the other day. Imagine a flight game on the 360. Currently your options are game pad joysticks or buy the big expensive one. Why not use Natal to figure out the orientation of the controller in my hand? Boom, instant easy to use flight controls while still giving me a controller for all the weapons and stuff.

You could apply that to turning a wheel in a racing game, melee attacks (just swing the controller to smack some enemy in the face during a match of Halo), or other things like that. I see a lot of room for Natal both in casual games and traditional games.

I could see that being useful for supplimenting current controls (and was not aware that was their intent). I've tried some of the driving stuff with my Wii but still prefer to have a controller in my hand. I like the idea of being able to tilt one way or another as a sort of quick jump out of the way movement.

Going back over all the peripherials made for the systems over the year, the number that succeed that weren't initial packins with the system is very small (Wii Fit is the only one that comes to mind).  Community Games in some respects could help that, as it's likely easier (assuming we're given the tools to do so) for a small developer to create a game that uses the technology than to justify implimenting this in a larger game when there may not be a large enough audience to make it financially worthwhile. I think far too often the size of the audience is what prevents more games, and without more games, more people don't buy into these things.

Kris Steele:

I could see that being useful for supplimenting current controls (and was not aware that was their intent).

I don't think it entirely is or isn't their intent. I think Natal is largely, as advertised, aimed at changing the way games are played and, more importantly to Microsoft, bringing casual and non-gamers to their console. However I'm sure they've also considered using it as a supplement to current controls. I know that was the first thing I thought of when I saw it.

Nick Gravelyn:

Kris Steele:

I could see that being useful for supplimenting current controls (and was not aware that was their intent).

I don't think it entirely is or isn't their intent. I think Natal is largely, as advertised, aimed at changing the way games are played and, more importantly to Microsoft, bringing casual and non-gamers to their console. However I'm sure they've also considered using it as a supplement to current controls. I know that was the first thing I thought of when I saw it.

Still the remote would make a great gun to hold in one hand.  It even looks like one of the weapons from Halo if held on its' side.

Danny Tuppeny:

I'm wondering if MS might loan me a Natal kit so I can start work on "Jungle Blocks 2: REvenge of the Blue Block". It's got hit written all over it :-)

Seriously, it'd be wicked if it was added to XNA. I just hope if it happens it's not like Avatars and Xbox only :( It'd be cool for PC games/Media Center/etc. to make use of this too! :D

Honestly I think when it comes to Live feature support, Games for Windows - Live is lagging behind and Microsoft don't appear to really be doing anything about it.
But... I won't go on about how there is an Avatar library for GFWL that is completely pointless as they don't provide any support via any PC-based SDK. Even Xbox Live support is kept very secretive when it comes to Windows unlike the whole promise they had originally made that the Xbox 360 and Vista would be a huge interactive space that you could swap between.

As far as Natal goes, I wouldn't hold your breath for XNA support.
Right now we're waiting on support for Video Media Files, and Xbox Live Vision Camera.

You want to know the frustrating thing about the lack of support for features like the Vision Cam is that in the XDK there is not only full support but actually there is an entire sub-api designed for allowing you to utilise it as a controller without coming up with your own control system. Problem is though the only game I've seen using the Live Vision as a controller is Totemball, which after talking to one of the developers I discovered they actually made their own.

So no idea how good the Microsoft one is, but from what I understand 3DV did the back-end code (same guys who did the EyeToy and who are the main people behind Natals' technology), so I would figure that it's got to be quite decent.

Would be nice to know why this functionality hasn't been made available for Windows as well, because I've had to write my own from scratch with far less than enthusiastic results.
The shame of it all is though that while I have the XDK, I don't have a DevKit and so while I could write a library for XNA a good question would be without it being signed code would it work? Probably not has been my general thoughts... but still if it doesn't I there is no way to add this support ourselves (so to speak), so we're at the mercy of when the XNA team feel it is "worthwhile enough"

More over is that the library is PPC-64 Code, which mean that even if I could get an XNA version working the base library would be in the wrong Assembler Code to desktop PCs well cept older MacOSX machines like the G5, which isn't supported by XNA. It's a headache (but potencially possible) for us, but for the XNA team they could get direct access to the code and team who wrote it. Not saying it's a simple task but one I think that would be appreciated.

Alright so Natal is a bit more advanced than Motion Detection and Facial Recognition, there is also Voice Recognition and Text-to-Speech functionality; but to be honest Microsoft also has libraries for these too as well as staff still working on updating them. I dunno but perhaps the XNA development team should discuss with the Community Developers, and as a community we help push forward XNA in a direction we believe will benefit everyone.

Shorter spans between XNA releases as well, say every 3 months we have a new update (just like DirectX and XDK developers get)

Just a thought