See also the 'Evil checklist of ways to fail a game' http://forums.xna.com/forums/t/19525.aspx
See also the 'Legal FAQ' for legal reasons you should and should not fail for http://forums.xna.com/forums/t/17830.aspx

This FAQ is a list of things that historically we don't fail a game for in peer review. Yes "confusing or frustrating" is vague, but if you fail for one of these things under that phrase then you will be mocked at best, possibly vilified and worse get a ban if people complain or you persist in doing it.

Many of these things are things that you SHOULD fix because it makes your game look lower quality and could annoy some users enough not to buy it. But that's your problem - don't come complaining when your sales suck. Many developers would love to see any or all of these as feedback in playtest so don't hold back there. Just don't fail for it in peer review.

DO NOT FAIL FOR

• Not having an exit option - users can always quit using the guide 
• Using "Return to Arcade" as an exit option. Prefer "Exit" though "Return to Dashboard", "Return to Game Library" or "Return to Indie Games" are OK too. 
• Leaving in the default 'back'=quit code
• Not having a pause/Not using start for pause/not pausing when Guide is pressed/not pausing when controller is disconnected.
• Legal reasons beyond those in the legal FAQ http://forums.xna.com/forums/t/17830.aspx
• Forcing a storage device to be selected
• Forcing a user to log in/play with a logged in controller (though test what happens if you sign out AFTER the game has started and ensure the game can be played from any controller)
• Code 7 crashes. These are caused by logging out the account with the Creators club membership. This crash never occurs in non-review play
• Spelling or grammar - point it out but its the developer problem if they want to release a poor quality game
• Lack of instructions or controller hints on screens and menus unless they are so convoluted the game/screen couldn't be used with a reasonable amount of guesswork e.g. you are totally stuck on the first screen because you had no clue that you had to hold down both triggers and use the right thumbstick to move.  In these cases post to the forum or look at other reviews - if everyone is having the same confusion it might be worth a fail. If others are not confused then don't fail. Note that we occasionally see a game where guessing the controls and gameplay is part of the game. Developers should call this out in advance so that they don't fail for it.
• Not using thumbstick AND dpad for navigation
• Not using 'standard' buttons or controls
• IN GENERAL - confuse and frustrate is related to technical issues. If the instructions, control scheme or the gameplay is confusing or frustrating but its as the developer intended then just ignore the game. You can't fail a game because you think it's too difficult, boring, stupid or ugly. You can't fail a game because its one of the starter kits - though starter kits are trivial to fail unless they have been fixed properly. You can't fail a game because you think it's not a game! Feel free to send the developer a comment on any of these and ignore the game but its not a fail.
• Not providing a way to upgrade to the full version of the game.
• Undefined/odd behaviour after pulling a MU. If it crashes then fail, but anything besides that is the developers decision.
• Mandatory device selection - though it is annoying to some
• Mandatory login - also annoying to some
• Vomit doesn't count as 'excretion' - poop does. We've not debated if urination is.... have fun with that one.

SPECIAL CASES:

• Some xboxes with some games under some unknown conditions exhibit very long deployment and loading times. If you find a game that appears to have very long loading times then don't fail right away. Post to or read the peer review forum and see if anyone else is seeing this. If everyone is seeing it then its OK to fail for. However if its just you then its likely that you have one of those special xboxes. See discussion here http://forums.xna.com/forums/p/27019/148171.aspx - the issue is known to Microsoft and we hope a fix will happen at some point. The issue only occurs when sending a .ccgame through connect. It doesn't happen in marketplace and it doesn't happen under the VS debugger.
• A small numer of testers report black screens at game startup when reviewing a game through GS connect. A restart of the xbox usually clears this up. It appears to be random and not related to the game or XNA framework. Its not been reported with any marketplace release. If you see this and it doesn't repro every time then check the forum to ensure its not just you.

OTHER ADVICE (especially to new reviewers):

• Read through the previous review comments and/or playtest comments (click the number under the developers name to find all their threads including the playtest one). You may find that your fail reason has already been discussed.
• If in doubt post first - other reviewers will help out
• If really in doubt then spend some time playtesting only. Note in playtest what you would fail for and people will correct you and you can learn the ways of the peer review process.

Please reply with other suggestions as you see them in the forum. If its already on the evil checklist as a fail then it will NOT be reconsidered here. If you can't provide a link to where its been discussed or happened then I'm not going to preemptivly include random things people can think of as the list will get too big.

That's a good list ZMan. If one of these things popped up in my game I would most likely *want* it to be failed so I can fix it. Sometimes these things don't get picked up until towards the end of the review process and by the time the developer sees the comment the game has completed review and has gone live. They can ofcourse do an update but by the time the update has gone through the game would have had its moment in the spotlight (new releases), and a bunch of those potential customers would be lost. Ofcourse most of it should already have been identified in playtest so I suppose most people who put their game in review with these things are already aware of them and just don't want to fix them.

Some of these points are deemed as acceptable even in XBLA, for example some game erquire that you play with a logged in controller. If you log out the game will pause until you log back in again. So these items would be more likey a concious design decision made by the developer and not just a flaw that they're too lazy to fix.

I should point out that Code 7 crashes can (and for me, often do) occur during non-review play. I have some sketchy network equipment that tends to kick the bucket whenever it's least convenient, and for several community games that I've purchased this instantly exits. It royally stinks near the end of a level.