Hello!

I'm new to these forums, and relatively new to XNA, although I have previously done a fair bit of DirectX coding with C#.

I have read and searched as much as I can about the Viewport.Unproject() method in XNA 2.0. It seems like a handy function to convert screen coords into world coords.

Unfortunately, it doesn't work properly. I had been persisting with trying to get it to return the right values for the last 3 nights. I discovered that, it returns the correct result if the camera's inverse view matrix is given instead of the normal view matrix, but only when the camera is at 0,0,0 and is facing up the z axis. Going by this, I tried to transform the resulting points to the correct camera position and rotation, but for some reason I had absolutely no luck (I think I was doing something wrong with the rotation.. I don't think the conversion from direction vector to matrix was working properly). Anyway, I shouldn't have to perform more transformations after doing an unproject.

With the regular camera view matrix given to the Unproject() method, it behaves quite strangely. It may seem to be returning correct results in some situations (close to the origin, camera pointing towards the origin). If picking (as in the picking examples, I have seen people complaining about its "inaccuracy" away from the center of the screen), the generated mouse ray seems to always pass through the origin. On further investigation, I discovered that Unproject() was returning CLOSE to the right results for the near plane (z=0). Infact I could get it to return the exact correct position by specifying a translation matrix of the camera's position, into the world parameter of the method. But in both cases (more so with the camera translation matrix specified instead of identity for the world param), the point generated at the far clip plane (z=1) was way off. This resulted in incorrect mouse rays.

Just tonight I decided, well, Viewport.Unproject isn't worth the hassle. I understand that XNA is aimed at XBOX, where Unproject() has little use, so I can see how this problem has gone unnoticed. So, I went and wrote my own Unproject() method. I must say that even though mine still doesn't work properly (can anyone help?), it still returns a much more accurate point on the near plane than the Viewport.Unproject().

Here is my method:

        public static Vector3 UnprojectD(Camera camera, Vector3 screenSpace)
        {

            Matrix scale = new Matrix();
            Matrix invScale;
            Viewport port = camera.Viewport;
            scale.M11 = port.Width / 2f;
            scale.M22 = -port.Height / 2f;
            scale.M33 = port.MaxDepth - port.MinDepth;
            scale.M41 = port.X + port.Width / 2f;
            scale.M42 = port.Y + port.Height / 2f;
            scale.M43 = port.MinDepth;
            scale.M44 = 1f;
            invScale = Matrix.Invert(scale);

            Vector3 us = Vector3.Transform(screenSpace, invScale);
            //us.Z = screenSpace.Z;
            Vector3 up = Vector3.Transform(us, camera.ProjectionMatrixInverse);
            Vector3 uv = Vector3.Transform(up, camera.ViewMatrixInverse);

            return (uv);

        }

I'm going to put the scale inverse calculation in the camera class for performance reasons. Thanks to whoever it was in this forum that originally posted that! It seems to work well..

The problem with the code is that no matter what I specify for the screenSpace.Z, I still get the point on the near plane. Does anyone know why?? The commented line seems to have no effect.

I realise that I'm probably "crying over spilt milk" about Viewport.Unproject(), being a function that most would implement a custom version of for performance reasons anyway (myself included). But I think if someone is just wanting to get a mouse ray then they will probably try to use it. And I don't think many would persist as much as I have.

Well, thankyou for reading, and I really hope someone can help me escape this nightmare!! And sorry for an excessively long post!

-dexy

There's at least two other threads on the (presumably) broken version of Unproject in 2.0.

http://forums.xna.com/thread/36985.aspx

Leave feedback on the connect post linked in the first one and I'm sure it'll get looked at.

I posted on the other threads, as well as in the connect issue which has been closed(!)

I also just used Reflector on the XNA DLL. I guess that's legal. It wasn't obfuscated. But for reference here is the method I extracted:

        public Vector3 Unproject(Vector3 source, Matrix projection, Matrix view, Matrix world)
        {
            Vector3 position = new Vector3();
            Matrix matrix = Matrix.Invert(Matrix.Multiply(Matrix.Multiply(world, view), projection));
            position.X = (((source.X - this.X) / ((float)this.Width)) * 2f) - 1f;
            position.Y = -((((source.Y - this.Y) / ((float)this.Height)) * 2f) - 1f);
            position.Z = (source.Z - this.MinDepth) / (this.MaxDepth - this.MinDepth);
            position = Vector3.Transform(position, matrix);
            float a = (((source.X * matrix.M14) + (source.Y * matrix.M24)) + (source.Z * matrix.M34)) + matrix.M44;
            if (!WithinEpsilon(a, 1f))
            {
                position = (Vector3)(position / a);
            }
            return position;
        }

I can't see where it's going wrong. My guess is that the matrix multiplication is wrong somehow. For some reason I don't think they should be multiplying the view and proj then inverting. And I'm not really sure what the WithinEpsilon bit is all about. Probably a degenerate case. But dividing by 'a' i don't really understand. Maybe I need to do something like this in my own method to fix it? :S

-dexy

        public static Vector3 UnprojectD(Camera camera, Vector3 screenSpace)
        {
	    Matrix transform = camera.World * camera.View * camera.Projection * camera.Scale;
	    Matrix inverse = Matrix.Invert(transform);
            Vector3 result;
            Vector3.Transform(screenSpace, inverse);
            return result;
        }

dexy:

I posted on the other threads, as well as in the connect issue which has been closed(!)

"Closed" doesn't necessarily mean "fixed" or "ignored". It generally means that it's been moved to their internal bug tracking system and is being tracked there.

Thanks for the replies...

I was a little confused about the status "Closed (By Design)" on the connect issue. I guess it's just wording that I'm not used to.

Obviously I can't wait for someone to change the XNA framework Viewport.Unproject, so I will have to persist with my own method.

Kris, I'll try your suggestions tonight, but somehow I don't think the results will be any different from my current implementation...

Example: My camera can be rotated and translated using keyboard and mouse. I am unprojecting a point at the bottom right of the screen, and using screenSpace.Z=0. I then draw an object at that point in the world (my world matrix is just identity). The object is drawn in the correct position, far more accurately than if using Viewport.Unproject(). The problem is, that when I specify screenSpace.Z=1, it returns the same point, as if Z=0 (the point on the near plane).

I'll try concatenating the matrices first, then inverting. But somehow I don't think it will make any difference. I seem to remember when learning about matrices that you need to use a W value when using the projection inverse matrix..? I'm really not sure.

-dexy

"Closed (By Design)" is a different thing entirely. That means that it's working the way they intended it to and don't plan to change it.

David,

That's what I initially thought "Closed (By Design)" meant. So I posted a comment in the connect area about how I couldn't possibly understand how this behaviour is "By Design" because I can't get it to return any sort of useable result at all. If someone can show me how to get a correct mouse ray in world coords with this method I will retract my comments. Until then I won't believe the method is working properly.

Also thanks ZMan, I probably should have gone researching about that before hand. But I still don't understand how the issue got it's current status, I guess because no-one voted for it (because how many people would be using unproject in XNA!? Not many! Most would be using XNA to make little games for their XBOX, which would have no use for Unproject.)

And if this behaviour really is "By Design", then I want to know how to use the method properly!! The examples seem to be using Unproject in the same way as in XNA GSE. And given these functions return different results between 1.0 and 2.0 of XNA, I can't see how the examples are valid. Anyway the samples don't work properly, they suffer the same problem as when I use Viewport.Unproject.

Well, it only took a week of my spare time, but I fixed it. All by myself!

After implementing my old matrix class because I thought maybe the XNA matrix inversion wasnt working properly, I sighed, because it returned the same result as with the XNA invert.

So back to the drawing board.. I tried what Kris suggested, but it returned results almost identical to the Viewport.Unproject() [not much surprise because really it is very similar to Viewport.Unproject]. I fiddled more with the way the matrices were concatenated... trying with all multiplied together first and inverting, and trying with all inverses multiplied in reverse order... both returning different, incorrect, results. Really I would have expected them all to return the same thing (they look correct to me! Somehow this was not the case).

And thats when it all flashed back to me. I read my earlier post in here where I said I had to do something with the W values. OF COURSE!! I remembered reading, while I was learning about how to perform a matrix inversion (a while ago), that when unprojecting you have to use 4-component vectors...!!!!! And then after you have transformed your 4d vector, to convert it into the 3d one you simply divide by W. And that's what the "float a=..." is in the Viewport.Unproject.

AND IT WORKS PERFECTLY.

So here is my very own Unproject method!! Working much much much better than Viewport.Unproject().

        public static Vector3 UnprojectD(Camera camera, Vector3 screenSpace)
        {

            Viewport port = camera.Viewport;

            Vector3 position = new Vector3();
            position.X = (((screenSpace.X - port.X) / ((float)port.Width)) * 2f) - 1f;
            position.Y = -((((screenSpace.Y - port.Y) / ((float)port.Height)) * 2f) - 1f);
            position.Z = (screenSpace.Z - port.MinDepth) / (port.MaxDepth - port.MinDepth);

            Vector4 us4 = new Vector4(position, 1f);
            Vector4 up4 = Vector4.Transform(us4, camera.ProjectionMatrixInverse);
            Vector4 uv4 = Vector4.Transform(up4, camera.ViewMatrixInverse);

            Vector3 uv3 = new Vector3(uv4.X, uv4.Y, uv4.Z);
            uv3 = uv3 / uv4.W;

            return (uv3);
}

So there it is. The method itself doesn't do any matrix inversions so it should be much faster than the original. It does however require the view and projection matrices to be already available in inverted form. My camera class is doing this whenever the view or projection matrices are updated (respectively). I think this ends up with the minimum of processor wasteage. Note the input vector is scaled without a scaling matrix, this is probably faster than using a matrix for this particular calculation. port.MinDepth and MaxDepth are usually 0 and 1, so really the Z part of the scaling does nothing and could probably be removed.

GO ME!!! (also thanks to anyone else who put any time into this! :)

-dexy

I'm wondering if your failure to obtain a correct value from the Viewport.Unproject may be related to a too small near plane value.  Take a look at this link... http://www.sjbaker.org/steve/omniv/love_your_z_buffer.html

In my case, I was originally using a model scaled down to 1/100th the original size... so my near plane was set to 0.1...  With this setup my collision code was perfect, but due to some other changes I had to stop scaling my models.  After I stopped scaling my models, I noticed that my collision code wasn't working...  Larger models should be easyier to collide against right? wrong...  because my models where larger I had increased my rate of movement and moved my far plane out so that my models wouldn't get clipped at the back end, which meant that I would be farther away from the models when I perform a triangle intersection test. 

A simple test that I did was to first test my triangle picking code w/ a near plane of 0.01...  which was really bad... couldn't collide a against anything.  I then tried 0.1...  things worked better, but it would be off at times...  I increased it to 1 and was able to collide accurately 100% of the time.

Thankyou everyone for replying.

I've been a bit busy, bit i got back on to 3d stuff last week. I've been playing around with a generic octree, go figure.

So I implemented a method to get a list of octree nodes currently under the mouse cursor (hello Unproject()!). I just plugged in my unproject method and it seems to be working well, at least for areas close to the origin.

In reply to the posts:

Pike - The method I posted above (and the one I'm going to include below) both have the input in pixel coordinates (x and y are 0 - screen res), so the method scales this to -1 to +1. The Z input coord however is between 0 and 1 (0=near, 1=far).

Ligren - Perhaps you are running into the inaccuracy issue I found, which I think I may have a solution for - dividing by W after the projection inverse transform and before the inverse view one. This way it gets rid of the 'd approaching epsilon' problem that can happen with different view transforms... (I think...)

Shawn - Cool, thanks!

MissileControl - I did find pushing the near clip plane out can help with precision issues, I think I have it at 0.5 at present. 1 might be sensible, but I like getting close to objects. I might consider a secondary transform for close objects.

And finally here's a copy of my current method:

 And this method is very useful! It may be the source of some people's problems:

Then you can just feed in the x and y coords and there's your mouse ray! In reality, the screen Z coord for the far point unproject doesn't really matter as long as it's big. It could do with further testing, so if anyone uses this, I'd love to know!

Regards,

dexy

Thanks dexy!

I had this problem and raised the question long time ago but gave up on it when I found the same issue with the Picking Sample.

However, your code didn't work (for me!) with the Z screen coordinate thing (farClip * 10) - but changing it to 1f made it work just fine.

So far I have only tested it quickly, but when using the built in UnProject I could reproduce the bug easily - with your code I have been trying for 10-15mins without seeing the bug.

Thanks a bunch!
Now I can finally get some picking done in my test engine :)

/Ronzan

I changed the code a little, so the method is an extension of viewport with the same parameters as Viewport.Unproject(). Now you have to simply change your call from m_viewport.Unproject(...) to m_viewport.UnprojectEx(...). (...and copy the following class in your namespace of course)