[Split from this peer review thread see http://forums.xna.com/forums/p/26450/152025.aspx#152025 for the issues]

This is why the seven day rule is just stupid. Both issues are minor. Neither is worth pulling it for. At EA we would have "known shipped" both of these. Previously as a Community Game the game would have been pulled and fixed. The days of high standards in Community Games are over. It just isn't worth going to Play Test for a week for two bugs that close to no one is going to find. I would leave it in review and see if anything else shows up.

If they were going to force that on us, I would have prefered an option to fail and send to Play Test on the review form so abusers of Peer Review would be the only ones affected. Oh well.

Patrick

I have to agree with Patrick, on every point except the intelligence of the seven-day rule. I think the seven-day rule is genius.

Good developers tend to want to ship with zero defects. It is a matter of professional pride. Especially on a personal project like these games. As Patrick has rightly pointed out, that is not how it is done in profitable software companies. The seven-day rule is genius because you can have crashing defects and still pass review. None of us (save maybe ZMan) know what the secret sauce is behind the review process, but it is clear that you can have a game that has critical massive failures in it, but if not many reviewers find it, it will ship. I saw a post on a game yesterday where a reviewer failed a game for a crashing defect and the result was that his review pushed the game over the edge into release!

By doing this, Micro$oft has forced the developer to wear the businessman's hat. Which we don't like to wear. You have to ask yourself the very question Patrick is asking himself now. He has a couple minor obscure bugs that will likely not knock him out of review. Are these bugs worth delaying release for another two weeks? And more to the point, is he willing to stake his reputation as a game developer on this game even with this defect? Can his customers live with it - at least for a few more weeks while he rolls out the next update?  I am guessing many of you developers are shocked and a little disgusted that there is even a question here.

I say if his game survives review despite the failure I gave him, then ship it! It would be a bad business decision to pull it to fix these minor defects - and it is awesome that Micro$oft has forced us to think about this trade off. Because I am pretty sure that without this penalty, Patrick would have ended the review to fix these bugs without a second thought.

I agree that this seven-day penalty will actually decrease the quality of games released through the community, but it may very well increase the rate at which we crank games out if we make the necessary adjustments to our attitudes. And the size of our catalog is just as important as the quality if Community Games is going to succeed.

Also, I think the system should not allow games to enter review until they have been through a week of playtest. The button should not even appear. I made the mistake of trying to launch right into review with our game given the choice, and I've seen many other threads where people have done the same (and failed and failed and failed). A cardinal rule of effective user interface design is that if you don't want the user to do something, then don't give them the option! If we don't want people to go straight to review, then why let them even see the button to do that until they have gone through playtest at least once?

Yes the 7 day rule is overkill for some bugs... in a world like EA where risk can be properly assessed and there are real paid testers looking at a game you don't need that rule.

In a world where 90% of our developers think they are some combination of Steve Jobs, Will Wright and John Carmack who can put terrible quality games into peer review 20 times in a week we needed this rule.

Sadly Patrick, most of the developers have neither your common sense, or your high quality approach to games.... we tried 2 months of being polite and asking people and for the most part the moderators got giant egos abusing them in return...

My personal 'design' for the feature was that the 1st failure was free if you spent a week in playtest, otherwise 3 days... the 2nd cost you 5 days, the 3rd and future ones cost you 10 days each. But it didn't happen that way... the world will be a different place when THe Zman is king... and no even I don't know the secret sauce.

Oh and Ross - using the $ sign when you type Microsoft is so very, very funny... man I almost peed my pants when I read it... how ever did you come up with that hilarious and ironic joke?

For the reference of others like me, what game are we talking about and what fail points?

Sorry nik, I didn't include the link to the original post when I split it... here you go http://forums.xna.com/forums/p/26450/152025.aspx#152025

Ah. I don't see how that's something you'd ship with. Basically you're saying that if you cancel the storage device selector (a completely valid choice) and then change the options (which causes it to try and save, but crash due to a lack of testing that you have a valid storage device), that's ok by you? That doesn't seem like all that farfetched of a scenario at all.

For instance, every game I play from Community Games I cancel the storage device selectors (I'm talking when I'm just downloading and playing them; not peer reviewing) because I don't want to clutter my hard drive with saves for games I likely will only play once. I know it sounds silly, but if I cancel the selector and the game lets me, it shouldn't crash later on because the developer forgot to check the device.

I'm all for this seven day rule. You shouldn't think of it as a punishment for failing peer review, but instead think of it as the motivation for why you should fully and properly test a game prior to submitting it. Anyone with a memory unit should have thought to cancel the device prompt and make sure all the points in the game where saving or loading takes place don't crash.

The ZMan:

My personal 'design' for the feature was that the 1st failure was free if you spent a week in playtest, otherwise 3 days... the 2nd cost you 5 days, the 3rd and future ones cost you 10 days each.

Yeah, that's exactly how I want it too. The 7 day rule is for people who are abusing the Review process, so the waiting time should kick when you're trying to Review for the 3rd time or something.

How I see it is that you'll go into Review first time. Someone will find a problem, so you go back and fix it. You go into Review again and the game passes. That's how it should normally be, so I reckon the 7 day rule should only apply if the game fails a second time, and for subsequent reviews.

The ZMan:

My personal 'design' for the feature was that the 1st failure was free if you spent a week in playtest, otherwise 3 days... the 2nd cost you 5 days, the 3rd and future ones cost you 10 days each. But it didn't happen that way... the world will be a different place when THe Zman is king... and no even I don't know the secret sauce.

I agree - we want the 7 day rule to punish the types of developers you mentioned, who just want to release SOMETHING as fast as possible without any polish..

However there is another scenario. My latest game, Awesome Tank, has been in Playtest for almost 2 months. Last week I pulled it from Peer Review (and it had almost passed) because there were several very small things that were not even thought of in Playtest (no music, you can infinite extra lives if you're good enough). So, I had spent a lot of time in Playtest, had 0 fails, and pulled it voluntarily to make some changes to it, that ultimately make it a better game.

It's not an issue this time thanks to Feature Creep, but if I had finished in two days, why would I then have to wait the remainder of the week to resubmit? Other developers in the same scenario might keep in in review because they don't want to wait, and a lower quality game would have been released. And then later 'patch' it, and clog up the que with updates.

I think that the 7 day wait period should be relaxed if you have no fails, have spent time in Playtest and voluntarily pull your game.

I guess I'm not seeing how waiting seven days is going to destroy anyone's business. I've been asking this from the start. What imaginary deadline are you guys all in a rush to hit? Most of you don't really have a real company with employees that aren't getting paid. And if you DO have a real company with employees waiting to get paid based on a business model that hasn't proven itself and no one has any clue what the numbers are going to be like, you're already a shaky businessman to begin with.

7 days is not a long time. You don't really have to release a game by March 3rd, December 25th or whatever crazy date you have in your head. Stop focusing on that and just relax, take a breath, realize you have another job (and if you don't, stop making XBLCG's and get one so you have what is called "guaranteed" income) and spend 7 more days fixing your game for whatever came up during peer review.

Yes Nick you're absolutely right.  The problem was that I spent better than a week in Playtest and got absolutely no feedback.  I had asked my testers to check ALL the backouts on the storage requestors and there was no indication that there was a problem.  In the absence of bug reports I went to peer review, and lo and behold some bugs were reported.  Of course I want the game to be as good as it can be, and as bug-free as I can make it, but I can only fix the bugs that are found. 

Regards,
Mike

Nick Gravelyn:

Ah. I don't see how that's something you'd ship with. Basically you're saying that if you cancel the storage device selector (a completely valid choice) and then change the options (which causes it to try and save, but crash due to a lack of testing that you have a valid storage device), that's ok by you? That doesn't seem like all that farfetched of a scenario at all.

...

I'm all for this seven day rule. You shouldn't think of it as a punishment for failing peer review, but instead think of it as the motivation for why you should fully and properly test a game prior to submitting it. Anyone with a memory unit should have thought to cancel the device prompt and make sure all the points in the game where saving or loading takes place don't crash.

I agree with you Nick.  I can't allow Carrum to ship with that bug.  But why didn't it crop up sooner?  I WANT people to tell me where the bugs are.

That's why you go testing.  Is it up to ME to decide if the game is ready to ship?  No, otherwise we wouldn't have peer review.  Now a bug has been uncovered, I have fixed it, and I am trying to work out if it's worth the time to pull the game, publish the fix in playtest and wait a week for something I KNOW has fixed the reported bug and probably won't get any playtest feedback on (as in my last week-and-a-half playtest), go to peer review, have another bug found with a 5-second fix, pull it, fix it, and start the whole cycle all over again.  Meantime it's got round to May and I'm in the middle of exam season when the game will get exactly 0% of my attention.  I WANT to fix ALL of the bugs, but alpha testing is notorious for missing bugs like this.  My own testers have very limited time and have to come to me to do the testing because they're students like me and don't have the spare cash to sign up to XNA and be able to test the game.  That leaves me at the mercy of the community, who I'm relying on to help me find the bugs so I can fix them - I thought that was what playtest was for.  If the community doesn't find bugs, I go to peer review.   Is this the wrong approach?

Why can't we just have a single pool of games for review, which have a pair of radio buttons for ALL games which says "I believe this is fit for release" or "I believe this is NOT fit for release" in terms of the current peer review guidelines?  Then it just doesn't matter about fix times, frequent updates (you want to talk about MS and frequent updates?!) or whatever.  When enough people believe a game is fit for release, using the current weighting algorythm, it gets released.  If not, the author can fix the bugs and resubmit the XNA package in short order without the arbitary delay of 7 days.

To clarify, there actually no way I could release Carrum with this bug, so I know I'm going to HAVE to pull it.  I think a few of us saw what was going on in the Exterminator review threads and were astonished by the idea of an author saying "not many people play with controllers other than PlayerIndex.One" and I'm damned if I'm going to fall in to that trap, no matter how much I want my game released.  I just get slightly frustrated when I try and play nice by the rules and then get zero feedback from playtest.  I have pressure on my time and energy, and I know everyone else does so I'm not going to criticise the community for this, I just think there could be a better way to find bugs, fix them and move on without the 7-day rule at all.  We've all seen playability feedback in peer review threads aside from showstopping bug reports, so why not streamline the review process so that we don't have this (again arbitary) line between "you could make the game better by XYZ" and "I've failed your game due to ABC".  They're both sides of the same coin - feedback from people who want ALL XNA games to be as good as they can be.  I should also say that the author of a game should have to confirm publication - what if they found a bug which had snuck through peer review and felt they HAD to protect users from that bug?

On a side note, I got a mail the other day saying "your payment is coming soon, check your details are correct" and I almost wet myself thinking it had crept through and my already limp reputation would be eviscerated because of untrapped bugs ;0)

Regards,
Mike

Now that there is a 7-day penalty for peer review failure, there is finally a distinct difference between playtesting and peer review. At one point, I decided I would create my own difference between playtesting and peer review, namely that I would only give feedback in playtesting. Now that is not necessary - I can give feedback for games under peer review and I know that if the developer decides to pull and revamp, they will be penalized. This goes back to the notion that Spyn Doctor originally proposed, that developers who submit games for peer review are essentially saying "My game has gone 'gold'." If they fail peer review, or they pull their game from peer review, then those developers were incorrect in their assertions that their games were finished. For the sanity of the reviewers (who do so on a volunteer basis), the developer should be penalized regardless of why the game was pulled.

Also, the responsibility of a game getting bug free rests squarely on the shoulders of the developer. Even if you get no feedback from playtesting, that still does not mean that you are not responsible for playtesting your own game so that it has no bugs before going to peer review.

DrMistry:

The problem was that I spent better than a week in Playtest and got absolutely no feedback.  I had asked my testers to check ALL the backouts on the storage requestors and there was no indication that there was a problem.  In the absence of bug reports I went to peer review, and lo and behold some bugs were reported.  Of course I want the game to be as good as it can be, and as bug-free as I can make it, but I can only fix the bugs that are found. 

DrMistry:

I agree with you Nick.  I can't allow Carrum to ship with that bug.  But why didn't it crop up sooner?  I WANT people to tell me where the bugs are.

And what about you? I don't mean to be mean here, but you can't only rely on playtesting. You need to test your game as well. You need to think of these issues and test them yourself. There are things you won't think of, but you already knew you wanted people to test that, so why didn't you? Perhaps you didn't have a memory card and didn't want to buy one (which I think everyone submitting to XBLCG should), but at the end of the day I still don't feel it is valid to blame playtesters for not finding your bugs. You made the game, you can test the game.

I'm glad to hear you want to make the best game you can and I'm sorry nobody found the bug. It really is annoying, I know. But people have to take responsibility for testing their own games as well.

As for dealing with the 7 day rule, sadly there's not much to be done. I doubt the team is going to start making exceptions for people or it will become a right mess. I agreed with ZMan initially that one fail shouldn't be punished because everybody fails once. It's the subsequent fails that should cause the penalties. Maybe in a future update the team will be inclined to tweak that a little bit, but ultimately it depends on what most people are failing for. You had a bug that wasn't the easiest to find, but we still fail a lot of people for basic controller handling and other things that show they clearly didn't try to test their games. Those people do need to spend some time in playtesting and work on bug fixing before review.

The weird thing is, you seem to get a lot better feedback in Peer Review, and people have a tendency to catch bugs that are missed over the course of weeks in Playtest.  Of course, you should always test your game thoroughly internally, but sometimes you miss things.  We make mistakes, we're human.

Now, personally, I would prefer it if the rule were modified a little, so that if you intentionally pull your game from Peer Review you don't have to wait the full week.  It feels like we're punishing developers for trying to make their game better.  Now, if your game is failed, then it makes sense to force developers to wait 7 days.

I'm not sure what would be better--maybe make it a three-day wait if you pull your game, and a week if it fails?

Just a thought.

DrMistry:

Why can't we just have a single pool of games for review, which have a pair of radio buttons for ALL games which says "I believe this is fit for release" or "I believe this is NOT fit for release" in terms of the current peer review guidelines?  Then it just doesn't matter about fix times, frequent updates (you want to talk about MS and frequent updates?!) or whatever.  When enough people believe a game is fit for release, using the current weighting algorythm, it gets released.  If not, the author can fix the bugs and resubmit the XNA package in short order without the arbitary delay of 7 days.

But that's exactly what Playtesting is, 'My game is not yet fit for release'

I can understand it's frustrating for you guys being forced into not progressing the submission of your game but you have to look at it on the opposite side of the conumer. I have played a number of Community Game that have crashed on the most basic of scenarios. This will directly impact your sales

If your game has been failed due to a bug such as the one discussed then take this opportunity to work out why this wasn't caught earlier and add it to the list of 'things I must check before I submit a game'. If that bug is there then there are probably others so take the time to play with it some more yourself

There are no exceptions to the 7 day rule, sorry

Anchorcast:

The weird thing is, you seem to get a lot better feedback in Peer Review, and people have a tendency to catch bugs that are missed over the course of weeks in Playtest.

I think the problem is most people don't playtest like they peer review. I think too many people think playtesting is just gameplay mechanics and fun and stuff and that peer review is for crashes. Playtesting is for all of that, so the goal should be to remind playtesters that they need to try and break the games just like they would in peer review.

Of course, you should always test your game thoroughly internally, but sometimes you miss things.  We make mistakes, we're human.

Of course. My point above is simply that you shouldn't blame playtesters for not finding your bugs. You (speaking in the non-specific sense) have to take responsibility for your game.

Now, personally, I would prefer it if the rule were modified a little, so that if you intentionally pull your game from Peer Review you don't have to wait the full week.  It feels like we're punishing developers for trying to make their game better.  Now, if your game is failed, then it makes sense to force developers to wait 7 days.

I'm not sure what would be better--maybe make it a three-day wait if you pull your game, and a week if it fails?

The problem is that it takes more than one fail to fail a game. So then everyone will just start pulling games after the first fail to avoid the 7 day wait, completely getting around it. Pulling a game and failing a game are equivalent in my mind. Both signify that your game was not ready and therefore the same rules should apply.

Nick Gravelyn:

The problem is that it takes more than one fail to fail a game. So then everyone will just start pulling games after the first fail to avoid the 7 day wait, completely getting around it. Pulling a game and failing a game are equivalent in my mind. Both signify that your game was not ready and therefore the same rules should apply.

Ah, I didn't know that, I thought one fail generally meant your game failed the process.

Anyway, I understand what you're saying about both signifying the game not being ready, however...  what if the developer is not fixing a fail-worthy bug, but one that wouldn't cause the game to fail?  Or if they get some new feedback and realize they need to clarify the instructions?  Or anything along those lines.

Perhaps if you end the Peer Review after having any fails, you would have to wait seven days, but if you end the process when you haven't had any fails, it would only invoke a 3-day wait?  Maybe it's a little overcomplicated, but I think perhaps it might remove some of the mentality where developers are unwilling to pull their game from peer review to fix minor issues they were unaware of.

Anchorcast:

Anyway, I understand what you're saying about both signifying the game not being ready, however...  what if the developer is not fixing a fail-worthy bug, but one that wouldn't cause the game to fail?  Or if they get some new feedback and realize they need to clarify the instructions?  Or anything along those lines.

All of that still indicates their game wasn't ready for peer review. They should get their feedback from playtesting or having friends and family test it or themselves. Submitting to peer review is saying "this game is absolutely done". If they have changes to make, then they clearly were not done and not ready for peer review.

Pheel:

DrMistry:

Why can't we just have a single pool of games for review, which have a pair of radio buttons for ALL games which says "I believe this is fit for release" or "I believe this is NOT fit for release" in terms of the current peer review guidelines?  Then it just doesn't matter about fix times, frequent updates (you want to talk about MS and frequent updates?!) or whatever.  When enough people believe a game is fit for release, using the current weighting algorythm, it gets released.  If not, the author can fix the bugs and resubmit the XNA package in short order without the arbitary delay of 7 days.

But that's exactly what Playtesting is, 'My game is not yet fit for release'

I can understand it's frustrating for you guys being forced into not progressing the submission of your game but you have to look at it on the opposite side of the conumer. I have played a number of Community Game that have crashed on the most basic of scenarios. This will directly impact your sales

If your game has been failed due to a bug such as the one discussed then take this opportunity to work out why this wasn't caught earlier and add it to the list of 'things I must check before I submit a game'. If that bug is there then there are probably others so take the time to play with it some more yourself

There are no exceptions to the 7 day rule, sorry

OK, again I understand what you're saying, but in this instance I was genuinely under the impression that this bug had been indentified and fixed at an earlier stage.  The Evil checklist was followed by my own testers and it came back positive (I should point out that I get the testers round to my house, give them a list of things to test, and then go out of the room to make sure I don't end up saying stupid things like "yea I know about that one, don't bother noting it down", or "the workaround is..." and wait for their feedback.)


This is getting to the point where I'm repeating myself several times in the same thread, so I'm only going to say this once more.  I am not blaming other reviewers or testers.  I am not expecting special treatment.  I am entirely responsible for my own code and the testing of that code.  But if people don't think the 7-day rule is going to lead to developers saying that won't pull a game for what they regard as minor bugs, then they're kidding themselves.  I've already pulled the game and put an updated version into playtest, as I said I would, and I fervently hope that any and all bugs are revealed as a result of this round of playtesting.  When you're developing your own apps or games, you get "bug blind" and unconsciously use workarounds to avoid crashes, which is why alpha testing isn't regarded as reliable.  Given the outcomes of my own testing program, I thought the game was market ready.  I was wrong.  End of story.

The 7-day rule, no matter how misguided I or others may believe it is, is clearly here to stay.  And that's the end of chapter two.

George Clingerman:

I guess I'm not seeing how waiting seven days is going to destroy anyone's business. I've been asking this from the start. What imaginary deadline are you guys all in a rush to hit? Most of you don't really have a real company with employees that aren't getting paid. And if you DO have a real company with employees waiting to get paid based on a business model that hasn't proven itself and no one has any clue what the numbers are going to be like, you're already a shaky businessman to begin with.

7 days is not a long time. You don't really have to release a game by March 3rd, December 25th or whatever crazy date you have in your head. Stop focusing on that and just relax, take a breath, realize you have another job (and if you don't, stop making XBLCG's and get one so you have what is called "guaranteed" income) and spend 7 more days fixing your game for whatever came up during peer review.

I couldn't possibly agree more. I personally don't have any imaginary dates 'set in stone', I will release my game when it's finished and bug-free. Unfortunately, developers with that mindset seem to be in the minority now. :-(

Anchorcast:

Ah, I didn't know that, I thought one fail generally meant your game failed the process.

Anyway, I understand what you're saying about both signifying the game not being ready, however...  what if the developer is not fixing a fail-worthy bug, but one that wouldn't cause the game to fail?  Or if they get some new feedback and realize they need to clarify the instructions?  Or anything along those lines.

Perhaps if you end the Peer Review after having any fails, you would have to wait seven days, but if you end the process when you haven't had any fails, it would only invoke a 3-day wait?  Maybe it's a little overcomplicated, but I think perhaps it might remove some of the mentality where developers are unwilling to pull their game from peer review to fix minor issues they were unaware of.

That's almost exactly what I suggested! :-)

The point I was trying to make with this scenario is this. I didn't mind waiting 7 days - I used the time to polish even more and add some features that were recommended in Playtest. But as Anchorcast said, with the "OMG I must release by April 5th at 12:00 A.M. or the world will end!!!" mentality that some developers have, they probably won't pull their game to fix some minor things, since it will delay their release by another week. By slightly modifying the 7 day wait period, I think that at least SOME developers might think twice about it, and maybe pull it, fix and resubmit. Then they won't have to do a mostly useless (since no one will know) re-release update in two weeks, clogging up the peer review que even more.

That way, we're still punishing those that treat Peer Review like Playtest, but not those of us that would want to add fix minor issues that never came up in Playtest. Hopefully it would increase the quality of CGs all around - I applaud the 7 day wait period, but I think that devs not fixing minor issues is an unintended side effect.

I think the 7 day rule is a good rule. And I totally agree with George: I can't see the problem with having to wait for 7 days. Even for the first fail.

As to the "unintended side effect" of developers not fixing their "minor" bugs: As someone wrote further above (I'm too lazy to go looking who it was), if it's a bug that's not fail-worthy, then it's simply a business decision of the developer, if he wants to release with that bug or not:

If he does not want to release, and thus pulls the game, then even though according to the rules the bug was not fail-worthy, at least for him the bug was obviously a major bug (or otherwise he wouldn't pull the game because of it). So by the developer's own definition, the game was put into peer review with a major bug. Ergo, the game wasn't ready for peer review yet -> the developer "deserves" the 7 day rest period to fix this bug (and look for others that he may have overlooked too) just as a developer who submitted with a major bug of the fail-worthy kind.

If the developer decides to release with the bug in place (i.e. classifies it as "minor"), then that's his decision! And yes, it will very probably happen that games are released with such "minor" bugs still in place, which previously would have been pulled (and the bug fixed, and the game resubmitted within hours).
However, I can't subscribe to the viewpoint that this is a negative thing for the XBLCG service as a whole. As I see it, it's only a negative thing for this game, i.e. this developer. I couldn't care less if someone else's game on the service has a minor bug, as long as my own games don't!

As I see it, peer review is not about the quality of the games at all, but solely about if they follow the rules or not. And "not being about quality" means it's blind about game play quality (peer review doesn't care if a game is boring or uninspired) but also means it's blind about the technical quality (peer review doesn't care about how polished the game is or is not, or if it has bugs of the non fail-worthy type).
Peer review exists solely for checking if games follow the rules or not. And for this, it should be as unencumbered as possible. And previously, peer review was quite often being abused by developers who did multiple submissions in quick succession, or who didn't really do quality testing themselves (figuring that peer-review would find any problems and they could then quickly fix and re-submit). This abuse encumbered the peer review process severely, IMO. The 7 day period is a good method to get rid of this encumbrance.

Doc

My first game is still in playtesting.  It's multiplayer so it doesn't get a lot of feedback.   Me and the other developer are testing ourselves but we still aren't expecting to pass review the 1st time.   There's always one last bug.   I will not like waiting 7 days to send it back to review but I do think that this seven day wait helps ensure reviewers time isn't wasted, which for me will mean a better product in the end.

Spyn Doctor:

As to the "unintended side effect" of developers not fixing their "minor" bugs: As someone wrote further above (I'm too lazy to go looking who it was), if it's a bug that's not fail-worthy, then it's simply a business decision of the developer, if he wants to release with that bug or not:

I suppose that's another way of looking at it. :-)

Spyn Doctor:

However, I can't subscribe to the viewpoint that this is a negative thing for the XBLCG service as a whole. As I see it, it's only a negative thing for this game, i.e. this developer. I couldn't care less if someone else's game on the service has a minor bug, as long as my own games don't!

I think it's a negative thing for the service as a whole, because as has been discussed before, if a new customer downloads a few trials of CGs and they're riddled with small bugs/unpolished, then he/she may write off the service altogether. I'm sure this has happened in real life many times to new, potential customers. I don't think anyone here wants that. It hurts everyone.

Steve Mulligan:

I have read a few reviews where the product wasn't sent to playtesting at all.  Why not force a week or two of playtesting before letting us send something to review the first time?

This is exactly why the 7 day rule was created in the first place: because people were treating Peer Review like Playtest. But forcing Playtest on everyone would hurt the professional studios (I mean real professionals, that have whole QA departments as Playtesters).

UberGeekGames:

Spyn Doctor:

However, I can't subscribe to the viewpoint that this is a negative thing for the XBLCG service as a whole. As I see it, it's only a negative thing for this game, i.e. this developer. I couldn't care less if someone else's game on the service has a minor bug, as long as my own games don't!

I think it's a negative thing for the service as a whole, because as has been discussed before, if a new customer downloads a few trials of CGs and they're riddled with small bugs/unpolished, then he/she may write off the service altogether. I'm sure this has happened in real life many times to new, potential customers. I don't think anyone here wants that. It hurts everyone.

And that's the logic I meant that I can't subscribe to. :-)

The argument that "bad/buggy games on the service will hurt the service, because users will write off the service alltogether" has been forwarded many times here on the forum, in many threads. It is usually accepted as true at face value, because it simply sounds so right and logical - it must be true, mustn't it?

Well, I personally have my doubts, but I'm not sure I can explain them very well. First, let me state the argument again, in a slightly extended fashion to better illustrate the meaning that is usually implied (at least as I understand it):

I'm a developer with a real good, bug-free and very polished game. I want to publish it on XBLCG. But I am afraid that users will not see my game, because they will first see so many other games which are not "real good, bug-free and very polished", so they will stop looking at more XBLCG games before they even get around to my game.

OK, just for the sake of the argument, let's assume that the assumption is true, i.e. that users indeed give up on the service after seeing a few bad examples, so they never go browsing in the XBLCG section of the Xbox marketplace again.

Now: Is this really something to be afraid of as a game developer?

You would only have to be afraid of this if the Xbox marketplace is the only channel through which players and the world will ever learn of your game. I.e., if you would have to continue the sentence of above with:

...so they will stop looking at more XBLCG games before they even get around to my game - and I am afraid of this because then they will never learn that my game even exists, because I have not undertaken any other marketing/advertising measures whatsoever to promote my game.

But if this is indeed true for a developer, then this developer has already lost anyway. After only 4 months of XBLCG, there were already 200 games on the service! If that rate keeps up, there will be around 700 titles come next holiday season (and most probably the rate will rise, as hopefully additional countries will be allowed to publish - crossing fingers here for this to happen for Germany soon ;-). On a marketplace with that many games, anyone who relies on his game being found "by accident", or "by users browsing through all available games" has already lost anyway. It doesn't matter if all 700 games are perfect games, or if there are 699 bad ones and your's is the only good one: People will simply not find games on this list by trying them all out - they will have to find them by other means: Reading about them, hearing about them, etc.

Actually, if all 700 games were perfect games, that would even be worse for your own game: If there are so many good games, what chances would your own game have to be noticed? Players would never have enough time to play all these good games anyway (and not enough money to buy them all). If however there would be 699 bad games and only 1 good game, I think this one pearl would get a lot of attention, because everyone would talk about "this one game which actually is good, when all the others are so bad"...

And the estimation of 700 is only for after the first 1+ year. The number of games won't stop there, it will just continue to rise.

The XBLCG marketplace is just the same as the games shelves at Wal*Mart or GameStop, or the listings at Amazon (at least it will be very soon, if the number of titles increases like it did so far): Like the shelves or the Amazon website, the marketplace simply contains all games that are available. Granted, some people will always go there and just "browse" to check if they can't find a new game to buy. But the vast majority of customers that go to GameStop to get a game, or order it at Amazon, already know which game(s) they want to buy before they go to the shop/website. They learned about them from friends, read about them in magazines or on the web, saw blog entries, and probably were subjected to a lot of marketing and advertising.

Similarly, at GameStop or Amazon, the sales of the good game titles is not hurt by the fact that those shops also have bad games in stock (and there are a lot of bad games - the ratio good/bad is certainly not better than for XBLCGs). That's because it is not so that customers go to the shop, take home a game they know nothing about, try it, then see that it is rubbish, so they bring it back to the shop, take home the next one [insert repeat-loop here] until they finally give up on games alltogether because there are so many bad games that they never get around to actually taking home a good game. On the contrary, the really good games get so much press/attention (sometimes also some not so good ones that are high-profile/well-hyped) especially because they stick out of the masses of boring/uninspired/bad games.

So, the best defense against the perceived problems that may or may not arise out of bad/boring/unpolished games on the XBLCG service, is first, to program a game that is not bad/boring/unpolished, and second, to make people aware that it exists by methods outside of the Xbox marketplace. If people hear good things about your game and thus learn a) that it exists and b) that it is supposed to be good, they will go to the place where they can get it (which happens to be the XBLCG channel on the marketplace) and simply bypass any bad/boring/unpolished games that may also clutter the same place/service. Just like customers home in on the boxes of [insert-favorite-blockbuster-game-here] on the GameStop shelves, ignoring all the other boring stuff that collects dust on the same shelves.
And once you have such a game that people will want to play on its own merits, it may actually be an advantage for you if there aren't too many other games of the same quality on the shelf (or the service in this case) because people can only split their money so many times. If there are too many other good games, maybe there won't be any money left to buy your game, be it is "real good, bug-free and very polished" as it may...

Doc

Spyn Doctor:

Well, I personally have my doubts, but I'm not sure I can explain them very well.