Silver accounts can do everything gold accounts can EXCEPT play online via Xbox LIVE.

Not Signed in controllers can't use any of the GamerServices except Guide.IsTrialMode. They can't buy games, can't display the marketplace, can't access gamertag or icons, can't do anything else.

Patrick

That is what I needed to know, mostly. The biggest question for me is which calls with generate an exception and need to be wrapped in a try/catch and which ones I can call and just won't give a value.

For example if I call


Under what circumstances with that generate an exception (if any) or how Guide.IsTrialMode or Guide.IsVisible?

Also, if I am not logged into any account I can't start my game at all. Is that because it is still an uploaded game directly from Visual Studio or because of something I have IN my game that I can add in a more dynamic way than I am now?

hotshot 10101:

Under what circumstances with that generate an exception (if any) or how Guide.IsTrialMode or Guide.IsVisible?

Those will never cause an exception. Only things like Guide.ShowMarketplace() and such. Properties such as those are always safe, function calls may cause an exception. Adding the component is something you always do, it makes no difference what kind of account they have or will use. Basically if it asks for a PlayerIndex or a Gamer it can cause an exception. If it doesn't ask for those then it is safe.

for example, SignedInGamer.SignedInGamer[PlayerIndex.One].GamerTag will crash if the controller is not signed in as SignedInGamer.SignedInGamer[PlayerIndex.One] will equal null.

hotshot 10101:

Also, if I am not logged into any account I can't start my game at all. Is that because it is still an uploaded game directly from Visual Studio or because of something I have IN my game that I can add in a more dynamic way than I am now?

In Game Studio builds you can start the game with your deveoper GamerTag and just leave the second controller signed out or signed in with a silver and test with that controller. One controller must be signed into the GamerTag with a developer account at all times otherwise you will get a code 5 or code 7.

Patrick

Star Gaming Network:

Those will never cause an exception. Only things like Guide.ShowMarketplace() and such. Properties such as those are always safe, function calls may cause an exception. Adding the component is something you always do, it makes no difference what kind of account they have or will use. Basically if it asks for a PlayerIndex or a Gamer it can cause an exception. If it doesn't ask for those then it is safe.

That's not true at all. Properties most certainly can throw exceptions.The Guide.IsVisible property, for instance, will throw an exception if you have not added the GamerServicesComponent to a game. Now I realize that he is already adding the component and therefore won't see that exception, but that doesn't mean properties are always safe and will never throw exceptions.

Star Gaming Network:

for example, SignedInGamer.SignedInGamer[PlayerIndex.One].GamerTag will crash if the controller is not signed in as SignedInGamer.SignedInGamer[PlayerIndex.One] will equal null.

That's technically not the property throwing an exception; that's the runtime throwing a NullReferenceException since you tried to invoke a property on a null reference. I'm guessing that's what you meant, but I wanted to clarify for anyone else reading this thread.

Thanks Nick, I did simplify since really that is where the crashes usually come from, checking permission on a null signed in player. and I did mean that since he already added the component the properties would be fine.

Really the best way is to try it with a controller not signed in and again signed in with a silver. That way you see exactly what happens. Otherwise someone is going to do that in Peer Review and fail the game for it anyways.

Patrick

Thanks for the answers. This clears it all up for me.

Can I infer from the above comments that the easiest way to check to see if a controller is logged in (in order to determine whether I should show my Buy Now button or not) is to simply check if SignedInGamer.SignedInGamer[PlayerIndex.x] is null where x is the controller number the player is using? is there a better way? Will this take care of whether they are signed in with an account that I can safely show the MarketPlace without getting an exception? After your previous comments I put a try / catch around my showing of MarketPlace, but this is not a great user experience because my Buy Now button just doesn't do anything if you are not logged in. I would rather not show it at all if showing the marketplace will cause an exception.

Currently I just check to see if Guide.IsTrialmode is true and if it is I add Buy Now to the menu.

hotshot 10101:

Can I infer from the above comments that the easiest way to check to see if a controller is logged in (in order to determine whether I should show my Buy Now button or not) is to simply check if SignedInGamer.SignedInGamer[PlayerIndex.x] is null where x is the controller number the player is using?

Yes. That works.

hotshot 10101:

Will this take care of whether they are signed in with an account that I can safely show the MarketPlace without getting an exception?

No, being signed in is not the only requirement. See here: http://nickgravelyn.com/2008/12/how-to-test-if-a-player-can-purchase-your-game/.

Thanks for the answer and code link and thanks to Nick for his code.

Arcane Labs:

I am failing game after game on these issues so I think its worth to list them again.

Test case1:
- Controller1: Off
- Controller2: On, signed in profile, playing on this controller.
- Verify the game works (from launch)

Test case2
- Controller1: On, no profile signed in.
- Controller2: On, signed in profile, playing on this controller.
- Verify the game works (from launch)

I failed six games on this from todays review list alone and if the guy who seems to have misposted to peer-review instead of play test doesnt pull his game it will be seven.

Y'know I really, really like how you've outlined test cases like that.  As I go through testing I'll see if I can come up with something similar and share it here.  Thanks for Spyn Doctor for starting this and everyone else posting suggestions.  This sort of info can help me get a lot of things right the first time as a developer. 

I was planning on going through the list in its entirety on a game today (so that I am familiar with all the different aspects of it and maybe to streamline it so I can do it effeciently for every game I review, hopefully one a week) and I'm stuck on the controllers issue.  The game is responding in a manner that I assume means it's all good (I get Code 7 errors, something the end users won't see)...but I'd like to double check.  Would it be alright if I posted about this here in this thread or should I make a new one (perhaps devoted to controller/Gamertag checking)?

nrXic:

Would it be alright if I posted about this here in this thread or should I make a new one (perhaps devoted to controller/Gamertag checking)?

Please start a new thread if you want to discuss specifics of your game and controller issues.

Oh no it's not my game, it's something I don't understand about the controller issues we should be looking for in games in general. 

Delete this post if you think it's inappropriate for this thread:  I'm testing one now and I get Code 7s in the first few test cases...I'm assuming this is the desired result?

I can post a new thread with specifics, but now that I think about it, discussion on the desired results of test cases outlined in the "evil" checklist should probably also include intended results.

I don't mind re-writing these test cases in a more traditional format, ie.:

Steps to reproduce:
1. Launch Application
2. Select "File" menu
File menu pulls down
3. Choose "Open"
"Open" dialog box appears
4. Select a file to open
5. Click OK
Result: File should open

^^^ Pasted from some random QA website I googled. 

Right now I'm assuming the intended result is a Code 7 error.  Obviously the result that will result in a FAIL in my review is a crash, freeze, or no-response from the controller (looks like a freeze).  Would my assumption be correct? 

None of the test cases should result in a code 7 error (if done right ;-).

"Code 7" means that you signed out the gamertag which owns the Creators Club subscription. The reason for this is, that you can not start a CC-game without first signing in the gamertag which owns your CC-subscription. And if you sign out this gamertag while the game is running, you get the "Code 7" error. This is a precaution by the framework to make sure that only CC-members can run CC-games.

So when you do the various controller and profile sign-in/out tests from the "evil" list, you need to do them with at least two controllers:

Your CC-membership gamertag must be signed in on one of the controllers (must be signed in before the game is started and must remain signed in the whole time the game is running) and you can then use a second (third/fourth) controller to do the actual testing, for example signing in another profile on that controller, then play the game with this second controller/profile, then sign out in the middle of the game, etc. Or deliberatly do not sign in a profile on the second controller, then try playing with this controller, etc. Or vice versa, i.e. have your CC-gamertag signed in on the second controller, and play with a different profile (or none at all) on the first (third/fourth) controller, etc.

One thing to remember: When signing in a profile to test with on one of the additional controllers, do not select the CC-membership gamertag which is already signed in on the first controller (the one that must stay signed in the whole time)! This would "transfer" the signed in profile from one controller to the other. Which means: Sign out of this profile on first controller, then sign in on second controller. Only because of the "Sign out on first controller" you already get a "Code 7" and the game ends, before the Xbox even has a chance to do the "sign in on second controller" part of the transfer.
So whenever signing in a profile to one of the additional controllers you test with, do not sign in the CC-gamertag that is already signed on the first controller!

EDIT: And as to the expected outcome of the tests, that's more or less self explanatory: None of the tests must cause the game to crash with a "code 4" (if it does, then that's a fail - but do not fail for "code 7", because that was then a mistake on your part during testing, see above). Also, the game must not exhibit other faulty behavior, like hanging, freezing, bugs that render it unaplayable, etc. - see the review guidelines and the "evil" list for stuff that a game must not do. (And see the "not so evil" list for stuff that may irritate some people but which is still not a reason for failure.)
So whenver not mentioned explicitely, the "expected outcome" of each of the test cases is: "The game must not exhibit any fail-worthy behavior".

Doc

Thank you for the detailed response Doc.  From my memory I do think I followed your instructions correctly but it's quite possible I've done something wrong, I'll do them over with a printout this time (last time I was running between 2 different rooms in the house, heh).  Also, I had an arcade fight stick as my first controller, maybe that complicated things so I'll stick with the normal 360 controllers.

Which reminds me, as I do the formalized Test Case writeups, I should include "requirements" (there probably exists a better word) for testing out the test cases, ie: "3 controllers, Gold CC Live Account, Silver Live Account, Local non-Live Account" or "Memory Card, Hard Drive, 1 controller, Gold CC Live Account", etc.  That way people can see what tests they can and can't perform at a glance.  I will try out your suggestions tonight Spyn Doctor, thanks again.

The test cases in that list are not so very formalized by design. If they were too formalized/rigid, they could too easily be misinterpreted as "This is the ultimate and one and only list of test cases which you need to test and if a game passes them all, the game is error free an will pass review". Such an interpretation would be entirely wrong though. Similarly, if the test cases were too formal, the danger would be to great that people would test only these test case variations and leave out other variations that are not mentioned. Also, a formalized test list could give the impression that this is a somehow "official" list of test cases given out by MS or the XNA team. Which would also be an incorrect interpretation
It's simply a list of stuff I came up with during my own (peer review) testing, augmented with stuff that other people mentioned. It is in no way complete or official, but, as is also mentioned in the "evil" list itself:

Note, that these test cases are meant as a guideline and inspiration only, as a basis for your own testing procedure. They are not meant to be followed slavishly and are not meant as a complete set of tests. A game may very well fail review even if it passes all tests mentioned here, if the reviewer finds some other problem! So be creative when testing and don't rely solely on this list!

Describing the test cases in a more informal manner is meant to emphasize this, so that a reader feels less like its an "absolute" list to follow slavishly, but more like its an inspiration for your own testing. Also, by having informally described test cases, a reader is more forced to actually read through them and think about them, and thus hopefully understand what a certain test case means, and why it is tested, and what problems it is meant to expose, etc. so that he can not only execute the test more intelligently, but maybe even come up with variations or other tests of his own. If the tests cases were in a tabularized format like:
- Requirements: A, B, N and Q
- Steps to execute: 1. Do this, 2. Do that, 3. Click here, 4. Finish that...
- Excepted results: Bla bla bla...
then you could go through the list with your brain switched off and not understand what you are actually testing - which I do not view as desirable for peer review.

Doc

Thanks again, I did right some formal stuff mainly because I can have other people here try them out and know exactly what to do.   I totally agree with the idea that if I posted them here people would take them the wrong way. 

Because of my lack of focus sometimes, I really needed a tabulated list for me to cover the basics, and then I cover other test cases I never did but thought of (I'm finding I have a notepad where I'm writing a lot...a bit too much...most of it suitable for playtesting feedback rather than review feedback).  So I'm able to focus and make it through a list quickly, as well as doing additional testing specifically for that game.  Again, I agree that posting it here could be a very bad idea...so hopefully I can benefit the community via a thorough playtest.  For me I do want to robotize the process a bit to be honest, especially when I'm popping in mics, drumsets, fightsticks, etc. into different controller ports and then doing different profile configurations.  If I can get that part of testing sped up it would be more convenient for myself.  But I can see what you mean, I might fall into a trap where I'm no longer thinking of creative new ways to test the games.  There are perhaps hundreds of configurations really, thinking of the most applicible ones is probably the best thing to do (for me in terms of time, and for the game in terms of testing). 

Regardless I hope to be an asset when it comes to testing games.  I've been blessed with many peripherals and the like so I might as well help others out who don't have them (I hear memory card testing is one that most people skip out on).

I didn't see this in the Evil list or here in the discussion thread, but I think it should be added. Here are the repro steps:

1. Controller 1 signed in with CC membership, controller 2 not signed in.
2. Start game with controller #2
3. When prompted to sign in, select Guest account
4. Prompt for storage device, using controller #2's PlayerIndex
5. Code 4 crash

This thread has more detail.

UberGeekGames:

I didn't see this in the Evil list or here in the discussion thread, but I think it should be added. Here are the repro steps:

1. Controller 1 signed in with CC membership, controller 2 not signed in.
2. Start game with controller #2
3. When prompted to sign in, select Guest account
4. Prompt for storage device, using controller #2's PlayerIndex
5. Code 4 crash

This thread has more detail.

Thanks for posting this. I thought about it, but wasn't sure it happened often enough to warrant being put on this list.

This seems to happen most often with games whose Marketplace code prompts for a LIVE-only account when the active controller is not signed in.

I think we should ask someone to make a translation of evil checklist into Japanese, then it would be easier to point out what's wrong with their game. I think a easy way would be to just make a google translate and then ask some nice japanese to spell check.

TG Viking:

I think we should ask someone to make a translation of evil checklist into Japanese

Already been done and in the works.

link..?

TG Viking:

link..?

Here.
http://forums.xna.com/forums/t/35833.aspx

But, this thread only excerpts from the original checklist.
In the first line of the thread, the Japanese-speaking Microsoft employee states "Following thread has the checklist for peer-review which was agreed in the community."

The third line reads "Following is the translation of a part of the checklist. Please use it when peer-reviewing."

EDIT:
In a different thread, another Team XNA member said that Microsoft was going to make Japanese only-as-a-guide translation of guidelines made by volunteers
and post them in the Japanese forum within days. The evil checklist will be included.

Thank you!  I have another wish, if you give the lines symbols that matches both languages like "(#1)", then it is easier for me point out the right line to my japanese friends..

TG Viking:

Thank you!  I have another wish, if you give the lines symbols that matches both languages like "(#1)", then it is easier for me point out the right line to my japanese friends..

From the top to the bottom, each line of the Japanese translation matches with each line of the Most Common Review Failure Reasons portion of the original checklist.
However, some details seems to have been simplified through the translation.

If you are going to use the translation to write fail reasons for Japanese creators, it would be safer to add the original English line.

As I editted my previous post, sooner or later, the Team XNA guy will make the full translation of the checklist.

UberGeekGames:

I didn't see this in the Evil list or here in the discussion thread, but I think it should be added. Here are the repro steps:

1. Controller 1 signed in with CC membership, controller 2 not signed in.
2. Start game with controller #2
3. When prompted to sign in, select Guest account
4. Prompt for storage device, using controller #2's PlayerIndex
5. Code 4 crash

This thread has more detail.

I guess that this should indeed be included in the evil list (I can see that several games have now already been failed for this), my problem is, I don't have enough details to describe this correctly.
The first problem I have is, that I don't even know how to sign in as a Guest account. Could it be that this is only possible if the main account (on controller #1) is a Gold account? Because mine is only Silver. And I can't see an option for logging in as Guest anywhere.

And what is actually the problem with a Guest account?

• Seems that opening the storage device selector for a PlayerIndex which is only a Guest causes an exception. Is this so?
• What about opening the marketplace? Does this also fail for the Guest? Does it fail in any other way than the fail we are already aware of if no profile or a local profile is signed in?

If someone who is able to login as Guest (owner of a Gold account?) could please test this and give me the details, I'll add it to the evil list...

Doc

Spyn Doctor:

The first problem I have is, that I don't even know how to sign in as a Guest account. Could it be that this is only possible if the main account (on controller #1) is a Gold account? Because mine is only Silver. And I can't see an option for logging in as Guest anywhere.

The only way I've been able to sign in as Guest is when attempting to use a purchase function on a local profile/not signed in controller and the game responds by using the overload of Guide.ShowSignIn() that takes the second argument (onlineOnly) set to true.

Spyn Doctor:

And what is actually the problem with a Guest account?

• Seems that opening the storage device selector for a PlayerIndex which is only a Guest causes an exception. Is this so?
• What about opening the marketplace? Does this also fail for the Guest? Does it fail in any other way than the fail we are already aware of if no profile or a local profile is signed in?

It's not opening the storage device selector that causes the problem, it's specifically the call to StorageDevice.OpenContainer() that throws a System.InvalidOperationException.

Marketplace (amazingly) does not fail for Guest.