When I fill an entire texture with one color, then draw it onto the back buffer with SpriteBatch using a matrix the rotates it, the edges become ugly and jagged.  If I take that same texture and fill it with one color except for a transparent 1 pixel border and draw it using the same method, it blends nicely and the jaggies dissapear.  Anyone know why this is happening or how I can fix it?

When you give it 1 pixel it hides the fact you have not turned on multisampling.

Turn on multisampling to fix it.

You are seeing the difference between triangle edges and texture filtering.

If your texture is entirely solid, the whole triangle will be drawn. The edge of your shape will be the same thing as the edge of the triangle. Triangle edges are hard: they either cover a pixel, or they don't cover a pixel. So you get jaggies. You can reduce this by enabling multisampling, which averages the result of several pixels to compute a single final output pixel color.

If your texture has a zero alpha band along the edge, the edge of the triangle is still hard, but this doesn't matter because it is being blended with zero alpha in any case. The determining factor that makes up the edge here is not whether the pixel is inside the triangle (which is a boolean thing) but the alpha value produced by your texture lookup (which is a smoothly varying fractional thing). If you have texture filtering turned on, this will produce fractional alpha values as the texture is transformed, giving you a nice looking antialiased edge.

From that you might conclude that relying on texture filtering is always better, but this isn't neccessarily the case: having alpha blending with fractional values causes a bunch of other problems, like zbuffer depth sorting no longer working properly. The best approach really depends on the situation. There is no single magic technique for avoiding aliasing in computer graphics: this is one of the hard problems and most experienced developers have a big bag of tricks they can dig into as required.

Thanks, that makes more sense to me now.  I'm running into this mostly in UI elements where I added a rotation option in my framework.  I don't imagine this will come up too often and the jagged edges may very well be acceptable.  I would rather use the texture filtering than multisampling, but I don't want to allocate an extra 1 pixel border for all textures and change all my rendering methods to take that into account.

Do you think I could change the UV settings to fill with TransparentBlack where the range is outside 0.0 - 1.0 and then just draw the texture a pixel outside the texture and a pixel bigger?

Jesse:

Do you think I could change the UV settings to fill with TransparentBlack where the range is outside 0.0 - 1.0 and then just draw the texture a pixel outside the texture and a pixel bigger?

Not directly. You could emulate such a thing in your pixel shader, but it would be a pretty complex calculation.

Shawn Hargreaves:

Jesse:

Do you think I could change the UV settings to fill with TransparentBlack where the range is outside 0.0 - 1.0 and then just draw the texture a pixel outside the texture and a pixel bigger?

Not directly. You could emulate such a thing in your pixel shader, but it would be a pretty complex calculation.

It seems like this should work:

device.SamplerStates[0].AddressU = TextureAddressMode.Border;
device.SamplerStates[0].AddressV = TextureAddressMode.Border;
device.SamplerStates[0].BorderColor = Color.TransparentBlack;

and then when using SpriteBatch.Draw pass it a source rectangle of Rectangle(-1, -1, width + 2, height + 2).  I tried it, but for some reason it looks like it was still using TextureAddressMode.Clamp.  So I was wondering if it was a flaw in the concept or my implementation.

Border address mode is not supported by most graphics cards. You can check the caps to see whether it works on yours.

Shawn Hargreaves:

Border address mode is not supported by most graphics cards. You can check the caps to see whether it works on yours.

device.GraphicsDeviceCapabilities.TextureAddressCapabilities.SupportsBorder is true on my computer, but if many graphics cards don't support it, I probably shouldn't rely on it.  I guess I'll just deal with it and make sure my control skins have a transparent pixel border if I want clean rotations.