random contest scoring musings

Jeff,

These are all valid concerns. Let me first give a brief summary of how the competition was setup.

I approached Alien Menace about helping to judge the contest. He agreed. He accepted the duty of head judge and also recruited all of the other judges. All of the entries were to be submitted to AM, and then divided equally among all of the judges. Each judge was to play and review their entries, and then recommend the games that they thought were worth further review. After all games were given the initial review, then ALL of the judges would judge each of the recommended games. Then the judges were to decide the rankings for these recommended games.

So basically, all games got at least one review, and if it was a really good game, it would be given further review by all the judges, afterwhich they would provide a ranking for each game.

Now what REALLY happened is that head judge Alien Menace found himself too busy to fulfill the duties. XXOOCC took over the job, only to find that all but two of the other judges (Alien Menace and Wolf) were also too busy to even judge. Couple that with XXOCC's computer crashing, and it has been a long, hard road for the competition.

SOOOO, at this point, I just want to make sure that everyone at least gets their game reviewed. The contest was never really meant to be too competitive. With bragging rights being the only prize being offered, I don't feel that objective ranking is really necessary. Also, I want to maintain some sort of credibility in case we want to hold future contests.

It was also quite a learning experience for our first contest. Lord knows we won't be making the same mistakes the next time. I agree that using a large gaming group is ideal for judging game entries. Obviously Hippodice has already proven that this system works. I think the reliance on a handful of static judges is not the best way to go. It is just too time consuming for them, not to mention the obvious difficulty in finding commited judges in the first place.

As for posting the games for the entire community to review, well that's great, but that's not what these designers signed up for. Some of them might not want their game posted openly on the Internet for the whole world to read, and we have to respect that. At this point, I'd like to see each and every game submission reviewed and scored. Then we can bring the contest to a close and perhaps think about starting a new one. We can't start a new one without bringing this one to closure.

-Darke

I hate to be the fly in the ointment, and indeed, it doesn't really matter to me because I didn't even end up entering the competition, but have the judges changed? Or was it always the case that different entries would be judged by different judges?

Either way, it seems to me that while the categories are the same, the way this judge assigned points is different than the way that the previous judge did. As such, I wonder if there's really a legitimate basis for comparison between the different entries such that you could coronate a "winner".

Now, you might say "well, it doesn't really matter if you win, the point is just to have fun and enter". But if that's the case, drop the judging pretense altogether and just post all the games here on the board and we can judge for ourselves which were the "best".

I don't know; it seems to be a really tough aspect of having a competition that a lot of people are going to enter: how do you get judges to evaluate all of the games on the same scale, except to have them evaluate ALL of the games? Yet, how can you ask judges to commit to doing the latter?

Perhaps for later competitions you should use a "Hippodice-like" method where there's an initial "filter" stage, and then the finalists will actually be playtested by the judges. The fact that that club judges the 50 or so games that make it to the finals by their whole group playing them all Spring strongly suggests that trying to judge 25 entries by playtesting with only a couple of judges was overly ambitious. Hindsight is 20-20, of course, but it seems that there are changes that can be made that would make future contests better. (I would probably say "must be made")
Maybe also charging a small fee to enter, and giving the judges a small reward would also be appropriate?

As for this contest...if I had entered, I'd probably be suggesting that we just drop the scoring altogether at this point, since it's taken so long and since the scores may be relative anyway. But I couldn't presume to propose that for the other people, who presumably worked hard on their entries and would like them evaluated.

Just my thoughts...

-Jeff

Jeff,

When I said this:

I meant this regarding the current situation. As I said, initially the contest was to have an 'initial screening process' by the judges, and then the best games were to be completely reviewed by all judges. Why was it like this? So even though your game might not have made it to the final round, at least you had a judge look over, play and review your game. I think that counts for something! And as I also said, AT THIS POINT I AM TRYING TO SALVAGE THE SITUATION! That means we will have to cut corners! And that means we may get reviews that are less than
adequate.

I understand that designers who signed up may not be getting what they signed up for, and I apologize. Understand also, that I was only supposed to be a mediator between contestants and judges. At this point, I am basically running this contest blindly. I don't have access to the entries, I don't even have access to their names or e-mail addresses. The only thing I have access to is the judges via e-mail. So basically I can't even e-mail the contestants to let them know that there was a change in the contest process. The only thing I can do is post whatever information I have on this forum. That's it. Please forgive my harsh tone, but I am just as upset about how the competition turned out as anyone. But I can't change the past, I can only do my best with what I have now.

-Darke