XNA 2.0 requires a "2005" Edition of C#.
Download Visual C# 2005 Express Edition here: http://www.microsoft.com/express/2005/download/default.aspx
Lower down on that same page is the link to download SP1, which you will need to install after you install Visual C# 2005 Express Edition.
If you are using Windows Vista there is a further ...