XNA Creators Club Online
Page 2 of 16 (395 items) < Previous 1 2 3 4 5 Next > ... Last »
Sort Posts: Previous Next

Evil Checklist Discussion thread.

Last post 11/17/2009 4:56 PM by Jim Perry. 394 replies.
  • 1/3/2009 2:03 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Spyn Doctor:

    Test case 8: If the game has some sort of text input through the player (for example to enter your player name for a highscore list), try entering a non-standard symbol and/or foreign character. This can cause a crash if the font chosen by the developer does not support this symbol/character.



    Any help on this one? I'm not sure how to go about solving it, and it's the one thing (that I know of) holding my game up from release. Thanks in advance!!

    EDIT: I think I figured this out -- here's my code if anyone else needs to fix this:

    //name is the string obtained from user  
    //validName is a boolean value that represents whether or not "name" contains illegal characters  
     
    validName = true;  
     
    foreach (char c in name)  
        {  
            if (!font.Characters.Contains(c))  
            validName = false;  
        } 
    Funkmasonry Industries
    Purveyors of fine funk since 2007.
  • 2/12/2009 3:03 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Here is another one to watch for:  Does the game maintain a network session when it is paused?  When the guide is open?  Its easy to pause your game by not calling Update on certain components, but you have to remember to update your network session or it will time out!
  • 2/19/2009 6:37 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Since this is the official unofficial TRC of XBOX Community Live, I would like to request that the only people that have shipped a game can reject or pass another person's game. People that have done the due diligence.
  • 2/19/2009 6:51 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    You make a sort of chicken and egg situation there. Reducing the number of voluntarily reviewers just overloads the ones who are left even more. And I've seen a lot of recommendations that you peer review a few games before you release as well.

    What's your concern? That games are being rejected that shouldn't, or games are being accepted that shouldn't?

    It's worth noting that MS have said that different reviewers will be weighted differently. Having released a game would be one of the things you'd expect to push someone's weighting higher (although MS haven't disclosed how people are weighted, and probably never will).
  • 2/19/2009 6:59 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Tesla Eisenheim:
    Since this is the official unofficial TRC of XBOX Community Live, I would like to request that the only people that have shipped a game can reject or pass another person's game. People that have done the due diligence.


    If this was the case then nobody would ever get a game passed.... 75% of the reviewers disappear as soon as their game is passed. Many other developers, in a vain attempt to get you to review their game, will give you a half assed review that took them all of 5 minutes. Take note of the folk who pass your game quickly with an ebay style comment (A++++ great game, would review again) and then one of the regulars fails it for a trivial reason.

    For consistency and quality you want people who have been reviewing games for a long time... you might not like that they fail your game but they are the ones who make your game better for the consumers.

    I've not had a game passed but I would bet that the developers who are looking for quality would rather have me playtest and review their game than most other people... Developers looking for a fast pass through review do not and I'm quite happy with that reputation.

    Its quite fascinating to me that people with <50 posts who just happen to be on their first game are always the experts in how the system should work and their opinion is almost always biased towards getting their games through, 2 weeks later when they are on XBLCG we never hear from them again thus proving that they really didn't care about the system but just about their own personal gain.... remember what the C stands for everyone...
    Play Kissy Poo - a game for 4 year olds on Xbox and windows
    The ZBuffer
    News and information for XNA
      Follow The Zman on twitter, Email me
        Please read the forum FAQs - Bug/Feature reporting
          Don't forget to mark good answers and good playtest feedback when you see it!!!
  • 2/19/2009 12:13 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    This is a good list of tests. I was looking for something like this and was going to write my own but you've covered most things.
    Only thing i would like is it to be made a sticky. And having it as a page on www.xnawiki.com wouldnt be a bad thing either :)
  • 2/19/2009 1:10 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    The ZMan:
    Many other developers, in a vain attempt to get you to review their game, will give you a half assed review that took them all of 5 minutes. Take note of the folk who pass your game quickly with an ebay style comment (A++++ great game, would review again) and then one of the regulars fails it for a trivial reason.
    Oh god yes, this makes me want to slap someone.
  • 2/24/2009 9:21 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    I am failing game after game on these issues so I think its worth to list them again.

    Test case1:
    - Controller1: Off
    - Controller2: On, signed in profile, playing on this controller.
    - Verify the game works (from launch)

    Test case2
    - Controller1: On, no profile signed in.
    - Controller2: On, signed in profile, playing on this controller.
    - Verify the game works (from launch)

    I failed six games on this from todays review list alone and if the guy who seems to have misposted to peer-review instead of play test doesnt pull his game it will be seven.


  • 2/24/2009 9:37 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    But they've ALL had "their friends playtest" their games and they are "pretty sure there are no bugs left"... amazing...

    Its not like these scenarios are new. People need to spend some time in the community learning about the process before they submit community games.
    Play Kissy Poo - a game for 4 year olds on Xbox and windows
    The ZBuffer
    News and information for XNA
      Follow The Zman on twitter, Email me
        Please read the forum FAQs - Bug/Feature reporting
          Don't forget to mark good answers and good playtest feedback when you see it!!!
  • 2/25/2009 1:06 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Actually Zman, we might have a serious XNA framework bug here.

    Easy Golf was one of the games rejected and following the steps above I could repeat this bug every single time:

    1 if (SignedInGamer.SignedInGamers.Count == 1)  
    2 {  
    3      SignedInGamer.SignedInGamers[0].Presence.PresenceMode = GamerPresenceMode.SinglePlayer;  
    4 }  
    5 else 
    6 {  
    7      foreach (SignedInGamer signedInGamer in SignedInGamer.SignedInGamers)  
    8      {  
    9           signedInGamer.Presence.PresenceMode = GamerPresenceMode.Multiplayer;  
    10      }  
    11

    Line 3 I get a null exception thrown.  How is that possible?  In debug mode I drill down into the SignedInGamers collection and at position 0 there is not a null gamer.

    Also, it happens on this code:

    1 if (SignedInGamer.SignedInGamers.Count == 0)  
    2 {  
    3       this.ExitGame();  
    4       return;  
    5 }  
    6 else if (SignedInGamer.SignedInGamers[0].Tag == null)  
    7 {  
    8      this.ExitGame();  
    9      return;  
    10

    Line 6, Bam, null exception.  How on earth can that be happening?

    I'm going to submit a connect ticket item.

    I prefer not to rip all of my code out as it is and just resubmit for peer review as is.  The game has been like this since day 1 and I am sure tons of others and we haven't heard a thing till now.

    Can anyone else confirm this behavior?
    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 1:16 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    So one of those scenarios is new then? Is it the controller being turned off one?
    Play Kissy Poo - a game for 4 year olds on Xbox and windows
    The ZBuffer
    News and information for XNA
      Follow The Zman on twitter, Email me
        Please read the forum FAQs - Bug/Feature reporting
          Don't forget to mark good answers and good playtest feedback when you see it!!!
  • 2/25/2009 1:37 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Barkers Crest:
    Actually Zman, we might have a serious XNA framework bug here.

    Easy Golf was one of the games rejected and following the steps above I could repeat this bug every single time:

    1 if (SignedInGamer.SignedInGamers.Count == 1)  
    2 {  
    3      SignedInGamer.SignedInGamers[0].Presence.PresenceMode = GamerPresenceMode.SinglePlayer;  
    4 }  
    5 else 
    6 {  
    7      foreach (SignedInGamer signedInGamer in SignedInGamer.SignedInGamers)  
    8      {  
    9           signedInGamer.Presence.PresenceMode = GamerPresenceMode.Multiplayer;  
    10      }  
    11

    Line 3 I get a null exception thrown.  How is that possible?  In debug mode I drill down into the SignedInGamers collection and at position 0 there is not a null gamer.

    Also, it happens on this code:

    1 if (SignedInGamer.SignedInGamers.Count == 0)  
    2 {  
    3       this.ExitGame();  
    4       return;  
    5 }  
    6 else if (SignedInGamer.SignedInGamers[0].Tag == null)  
    7 {  
    8      this.ExitGame();  
    9      return;  
    10

    Line 6, Bam, null exception.  How on earth can that be happening?

    I'm going to submit a connect ticket item.

    I prefer not to rip all of my code out as it is and just resubmit for peer review as is.  The game has been like this since day 1 and I am sure tons of others and we haven't heard a thing till now.

    Can anyone else confirm this behavior?


    What happens if you check index 0 for null in the code before attempting to access it?

    -Zenroth
  • 2/25/2009 1:40 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    I wouldn't know if any of those scenarios would be an old one.

    I'm basically starting up my xbox with controller 1, signing out.

    Turn on controller 2, sign in with creators club account on controller 2.

    Turn off controller 1.

    Start up game and play with controller 2.
    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 2:02 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Centurion Games:

    What happens if you check index 0 for null in the code before attempting to access it?

    -Zenroth
     

     

    if ( SignedInGamer.SignedInGamers[0] == null ) 

    The above code evaluates to true.

    I'll just put checks and dump the gamer back to the title screen if this happens.

     

     

     

    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 2:33 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Barkers Crest:
    Centurion Games:

    What happens if you check index 0 for null in the code before attempting to access it?

    -Zenroth
     

     

    if ( SignedInGamer.SignedInGamers[0] == null ) 

    The above code evaluates to true.

    I'll just put checks and dump the gamer back to the title screen if this happens.

     

     

     



    I haven't looked at the SignedInGamer structure in awhile, but is it based on a pre-allocated array like PlayerIndex, in which case you have [n] elements always existing and [0] may indeed be null, but [1] isn't?
  • 2/25/2009 2:56 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    I was in debug mode and checked each of the elements.  Element [0] was not null and contained the proper signedingamer.  Elements [1] through [3] were null.

    Edit: I was able to work around the issue with minimal code changes and am in the process of resubmitting. A work around is to just loop through the SignedInGamer collection inside a foreach statement.

    Another work around is to access the gamer like this:

      
    SignedInGamer gamer = Gamer.SignedInGamers[0]; 
    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 3:34 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    My guess would be that when you checked in debug mode some event hadn't processed yet, causing a valid but actually invalid state. Reminds me of the guide is visible exceptions, when checking guide.isvisible is clearly false at the moment.
  • 2/25/2009 10:31 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Centurion Games:
    Also, it happens on this code:

    1 if (SignedInGamer.SignedInGamers.Count == 0)  
    2 {  
    3       this.ExitGame();  
    4       return;  
    5 }  
    6 else if (SignedInGamer.SignedInGamers[0].Tag == null)  
    7 {  
    8      this.ExitGame();  
    9      return;  
    10

    Line 6, Bam, null exception.  How on earth can that be happening?


    Lets say there are 2 signed-in players with indexes 1 and 3.
    So in line 1, the if statement evaluates to false, then in line 6, you will get exception because SignedInGamers[0] is null.

    IMO the "bug" is: You assume that index of signed-in player is always 0, but it can be any other index.



  • 2/25/2009 10:39 AM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    In my experience its much safer to utilize PlayerIndex.One, etc.

    Since it utilizes a different overload. The one accessed through integer index has been buggy for a long time in my experience.

    In our games we always transfer the active player indices around in their PlayerIndex form to ensure nothing goes wrong.  If you just use the integer value there is nothing in the framework that says that the order of the entries will stay fixed when controllers are added or removed, so it has always been extremely unsafe in my oppinion.

    - René


  • 2/25/2009 2:59 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    DecapodStudios:

    Lets say there are 2 signed-in players with indexes 1 and 3.
    So in line 1, the if statement evaluates to false, then in line 6, you will get exception because SignedInGamers[0] is null.

    IMO the "bug" is: You assume that index of signed-in player is always 0, but it can be any other index.


    Instead of assuming, how about you actually test out what I said.  In debug mode the SignedInGamer.SignedInGamers[0] was not null when putting in a breakpoint and analyzing the values at line 6.  However, when you continue and execute the code it throws a null exception.

    Considering the fact that SignedInGamers inherits from IList the expected behavior would be all non null items in the list bubble up to the front of the list.  That is how collections work.

    I repeat, SignedInGamers is a collection and not an array.

    However, the interesting thing is, using reflector the SignedInGamer class inherits the SignedInGamer collection from the abstract Gamer class.  The bug happens when using SignedInGamer.SignedInGamers[0] but not when using Gamer.SignedInGamers[0].

    I wish I could say I am having brain issues so how about running the code too and telling me if you are able to repeat this behavior.

    - Matt

    Unarmed:
    In my experience its much safer to utilize PlayerIndex.One, etc.

    Since it utilizes a different overload. The one accessed through integer index has been buggy for a long time in my experience.

    In our games we always transfer the active player indices around in their PlayerIndex form to ensure nothing goes wrong.  If you just use the integer value there is nothing in the framework that says that the order of the entries will stay fixed when controllers are added or removed, so it has always been extremely unsafe in my oppinion.

    - René




    You are correct Rene, it is buggy.  I retooled my code to use Gamer.SignedInGamers and I don't have the problem anymore.  If I am reading what you are saying correctly, I wouldn't use the PlayerIndex as an indice for accessing a collection ever.  That doesn't make sense as the SignedInGamers are a Collection and not an Array where the indice is meant to infer PlayerIndex.  PlayerIndex is used to specify what controller a gamer is using.  If I am not understanding what you are saying correctly, please disregard my comments.
    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 3:42 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    When using the PlayerIndex as the lookup indice, I don't experience any problems in my tests.  It's possible that the SignedInGamerCollection adds null items to fill on creation.  In which case the collection could look something like { null, GamerTag1, null, GamerTag2 }.  So when using int indices, SignedInGamers[0] might return null, while SignedInGamers[PlayerIndex.One] returns GamerTag1.

    Just a thought, I have no idea if it's actually what happens.



    After reading the docs, my explanation is definitely not what should be happening. Gamer.SignedInGamers is what SignedInGamer.SignedInGamers is inherited from, and it pretty explicitly states that if only 2 gamers are signed in, there will be items at indices 0 and 1 and nothing more.

    So I honestly don't have a clue.
  • 2/25/2009 4:06 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Brian, are you able to create a project and recreate the bug?
    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 4:16 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Barkers Crest:
    Instead of assuming, how about you actually test out what I said.

    Instead of getting offensive, try to comprehend the fact that people do not always have access to XNA "to actually test out what you said". I was thinking about testing it when I get to my dev machine, but you seem to be doing fine without me.

  • 2/25/2009 4:37 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    DecapodStudios:

    Instead of getting offensive, try to comprehend the fact that people do not always have access to XNA "to actually test out what you said". I was thinking about testing it when I get to my dev machine, but you seem to be doing fine without me.


    I did not mean to offend you.  No worries.  Opinion easily gets accepted as fact on these forums so speculative comments can do more harm than good.

    Once again, no worries.  :)
    Avatar Golf - On the Marketplace

    Blog: www.barkerscrest.com - Easy Golf
  • 2/25/2009 6:59 PM In reply to

    Re: My little "evil" checklist for peer review stress testing...

    Barkers Crest:
    DecapodStudios:

    If I am reading what you are saying correctly, I wouldn't use the PlayerIndex as an indice for accessing a collection ever.  That doesn't make sense as the SignedInGamers are a Collection and not an Array where the indice is meant to infer PlayerIndex.  PlayerIndex is used to specify what controller a gamer is using.  If I am not understanding what you are saying correctly, please disregard my comments.



    If I understand what you are saying I would agree if I just read the documentation you linked to.

    But my visual studio says the following:

    Overload #1: SignedInGamer ReadOnlyCollection<SignedInGamer>[int index]     | Index: The zero based index of the element to get.     (This appears to be the buggy overload)
    Overload #2: SignedInGamer SignedInGamerCollection[PlayerIndex index]         | Index: The Framework.PlayerIndex of the local gamer to retrieve (this one always works).

    So there is infact a 'array/list' like version of the method.
     
Page 2 of 16 (395 items) < Previous 1 2 3 4 5 Next > ... Last » Previous Next