Hey guys,

I have an need to read a color value off the client window, does anyone know if there is a comparable method to the old GetPixel api call?

I need to read a pixel off the display, not out of a bitmap, texture, or back buffer.

I'm porting a game project over to XNA and one of the things I do for super-fast mouse picking, is that whenever the mouse moves, I render to a 1x1 pixel render target any objects that might be under the mouse, using a pixel shader that actually just draws a solid color representing a uint32 index for the object.  I draw the rendertarget texture in the top corner of my display and use GetPixel to sample the color.  The color represents the top-most item drawn under the mouse.  So for example, if my screen update blitted out 4000 objects, I know exactly which one the pixel under my mouse pointer belongs to based on the pixel value.  Anyway, so thats me justifying why I'd like this work.  ;)  When I was working with OGRE, I figured out this trick.  I know there is an XNA call to grab pixels out of the back-buffer and textures, but pulling data backwards from the GPU pipeline will kill my framerate more than GetPixel will.

No, in general you cannot do that, because the display is asycnhronous with the CPU.
You have to render into a render target, and then copy that render target to backbuffer as well as reading the pixel out of the render target.

OK I got it to work.  :)  If anyone else is interested, here is the code.


Just needed to use the old API calls, create a device context for the screen, sample the color and release the context.

I really don't recommend using GDI. Not only is it error-prone due to various complications (layered windows, etc...) but my understanding is that getting a DC for the screen and trying to read/write can have severe performance complications when desktop compositing is enabled (Vista, Windows 7). And of course, it won't work on the 360.

I'm also not convinced that CreateDC+GetPixel will actually be faster than a single-pixel readback from the framebuffer.

Heya Kevin,

I tested it both ways and you are correct.   :)

My previous method for mouse picking : render any objects potentially under the cursor to a 1x1 render target with a special shader that draws them a solid color representing a numeric index, render the pixel on top of the display with everything else, read the pixel with a ghetto call to the win32 API, and convert the resulting RGB back into an integer.  And do this only when the mouse moves.

Result : 920 fps idle, and 820 fps with mouse moving across the playfield.

Your method : same as above, except read the pixel value directly off the render target, skip drawing the render target and skip GetPixel.

Result : 956 fps idle, and 865 fps with mouse moving across the playfield.

I actually realized an idle speed increase by skipping the state change required before planting the render target on the backbuffer.  I'll take it.  :)

Thanks for your help.  Maybe there was something eccentric about OGRE that made this method slower, I don't know.

Techniques like this can definitely have strange results. It never hurts to try alternate approaches - at least now you know for sure which technique runs better! The difference in performance is small enough that I wouldn't be surprised if it was faster in OGRE due to slight architectural differences.

Your technique is a nice solution to mouse picking, by the way; I don't think I've seen it done for 3D before. Thanks for sharing it!

I'm not sure what you mean with: "I don't think I've seen it done for 3D before.".
If you're saying that picking in 3D hasn't been done before, then you're wrong, but, and that's what I think you mean, if you're saying that it hasn't been done before like this, then you're right.
I just wanted to add the method of picking I am using in my game:

public static Ray GetPickRay(MouseState state, Matrix View, Matrix Projection, GraphicsDevice Device)
{
int mousex = state.X;
int mousey = state.Y;

Vector3 near = new Vector3((float)mousex, (float)mousey, 0.0f);
Vector3 far = new Vector3((float)mousex, (float)mousey, 1.0f);

Vector3 nearpoint = Device.Viewport.Unproject(near, Projection, View, Matrix.Identity);
Vector3 farpoint = Device.Viewport.Unproject(far, Projection, View, Matrix.Identity);

Vector3 direction = farpoint - nearpoint;
direction.Normalize();
return new Ray(nearpoint, direction);
}

Then use the Ray.Intersects() method to check for collision.

Hi Valandur,

For my purposes I needed to know which object a specific pixel belonged to, not just whether a mouse ray shot through the bounding areas of the quad object.

I think your method of ray intersection is an efficient solution for a 3D game that uses actual 3D models and complex geometry.  My method is meant for picking irregular shaped sprites with significant areas of transparency in a 2D game design.  I tried using the ray method initially when I was working with OGRE but discovered I was just detecting an intersection with the sprite quad, which is unacceptable when you might have dozens of items, units and terrain objects layered in one spot.

My game project is tile based isometric, and I use an optimized method for blitting my tiles downscreen.  At startup ( or on window resize ) I precalculate and sort a list of pasting offsets and map offsets for every isometric tile visible on the display.  This prevents me from doing a ton of fancy xy calculations in my "map draw" function, solves some overlap issues and simplifies my code, but it also means every tile location has an index number in the structure/list.   When I move the mouse, I render any object under my cursor to a 1x1 render target using a shader that colors them in solid with this index value.  I can then sample the color from the resulting texture and I know exactly which object, out of however many objects were layered on that spot, was the very last to modify that pixel, and therefore the topmost object which I am clicking on.  The pixel color = the list index of the thing I want, irrespective of how many transparent areas I'm shooting through that belong to other sprites.  The great thing about it is that I can include other objects, like user interfaces, in the pass and not worry about whether I clicked "through" a dialog box and so forth.  If your need is to determine whether you clicked on the geometric face of an object, then yes your method is preferable.  If you want to use transparency then you probably ought to use a system like mine.

You might be able to get a similar effect by modifying the 2D Per-Pixel Collision example. Since you'd be testing for collisions between a point (the mouse location) and your object list, rather than between an entire object and your object list, it should perform rather quickly.