playtesting and variations

Was there a question in there?

Blind testing is something I haven't really done any of yet. I tried explaining All For One to a group so they could play it without me (I was across the room), but I did a lousy, rushed job and they were playing wrong as a result. I don't actually have the rules printed out or organized (something I'm working to fix!)

I have been lucky enough to get several playtests of Terra Prime in lately: last week, Thursday, Saturday, and today. They are not blind tests at this point, and I'm still tweaking things. I'd like to send it to Johan soon for blind testing.

- Seth

I think for Blind Testing, you should probably have already tested the game a lot with yourself there, to make sure the bugs are out (as much as possible). It IS tough to get people to play different versions of a game that's not even published yet. I am thankful when I get to playtest my games, but often people are not interested.

- Seth

As for persuading people to playtest games? It's mostly a matter of building up trust with a group so that they know you well enough to give things a go. I know that some groups mix playtesting in with "real" games (this is sometimes interesting as the group then ends up discussing all the flaws in the "real" game along with yours!) I'm lucky to have a core group that has two nights, one for "real" games and one for playtesting. And there's an important stage of testing when you need to be physically present and watching a test [but not participating!] before you can really have a proper "blind test" with a group when you aren't there to answer questions! (A number of commercially available games don't seem have done that middle stage properly; but hey, the BoardGameGeek would be a dull place without that :)

Blind testing is something I haven't really done any of yet. I tried explaining All For One to a group so they could play it without me (I was across the room), but I did a lousy, rushed job and they were playing wrong as a result.

- Seth

Hey, I think I was playing that time! Hi, Seth!

Well, I just have to say that Blind tests are a test of the rules as written, not the system they embody. After a game has had enough playtesting to feel absolutely solid, try a blind test of the written rules on someone new. i suggest the following:

First, have someone just read the rules for clarity. Just to see if they feel like they understand the mechanics.

Next, If they feel they understand, have them explain the game to the members of their group, and report back to you after they play. Note: you shouldn't be there when they play the game. They must rely on the written rules, rather than ask you, and their frustrations should not be influenced by your presence. If they come to feel the rulebook sucks, that's important to know, and you shouldn't be there to go to bat when the written rules fail. See if the intent of your rules was followed.

This can be done after they feel they know the rules. Have them play a game with you present, and see for yourself if they are indeed doing things right. They may be playing an interesting, superior, and unintended variant.

If possible, after a given rules set has been played for a while, and more wrinkles in presentation are ironed out, go to another group, and have them all read the rules, and then jump in a begin playing, each with their own understanding of the rules. what you're looking for here is a consistancy of understanding.

Blind testing isn't easy, because you only get a single shot with each group with each version of th rules. But let us know how it works for you!

Oh, one way to lure testing groups is to provide pizza and coke in exchange for a night's hammering on your game. This is a good trade off for everyone.

After trying it a few times, I have decided that I don't like blind playtesting when I'm not present at the playtest itself. It's often a lot of hassle to send a prototype to someone, and the quality of the feedback you get is often low. Perhaps they can catch a flaw or omission in the rulebook, and perhaps give some generic comments such as "we liked it", "we thought it was too long / too short", "strategy X is too good", but it's hard to gauge the value of these comments when you were not around to check how the game actually went. You also don't know if they misunderstood a rule.

So, bottomline, blind testing is important, but make sure you are around to check how the game is actually going to be able to put comments in perspective.

In this way its a blindtest with elements of a normal test.
I would call it the other way around: a normal test with some elements of blindtesting. A real blind playtest can't be done by our frineds or family.

Actually, they shouldn't even know who the author is, or if the design comes from a published author or it's a first timer work. Any information of this kind (let alone the designer being present and having to report directly to him) will go against the advantadges of the playtest "blindness".

In any scientific testing, you even do a double-blind test: not only the people doing the actual testing lack that information, but also the people controlling it. E.g.: in a double-blind medicine testing, not only will you have a control group given a placebo, but you'll have the people administrating the pills in the blind too. If they know who gets the real medicine and who gets the placebo, thay can, unconsciusly, pass that information to the patients, thus turning the whole test invalid.

That said, I agree with Johan about the objective of blind playtesting, and it being only a final milestone to check if the game is actually ready for submition (or publishing).

Seo

I agree. I know I'm sometimes too precise when it comes to the exact mening of a term. Maybe it's to avoid confussions about what we are really talking, as many of us use english as a second lenguage, and sometimes fail to find the exact word. But I'm affraid it has more to do with me being a bit annoying. My family would wholeheartedly agree with this later explanation. ;-)

Anyway, you have a good point about really-blind playtesting being a lot harder to implement, and partially-blind playtesting (sounds better than visually-impaired playtesting or short-sighted playtesting, doesn't it?) being the best we can afford on most situations. I have trouble even to convince my wife and daughter to do fully-20/20-vision playtesting!

I think we all agree that the further developed a game is, the blindest the playtesting can and ideally should be. Though normal playtesting with people who has seen the game evolve has a different set of advantadges, so it never grows useless.

Seo

The Bard has words of wisdom on this topic (as on so many others):

"Marry, sir, [my friends] praise me and make an ass of me; now my foes tell me plainly I am an ass: so that by my foes, sir, I profit in the knowledge of myself, and by my friends I am abused." (Twelfth Night, Act V, Scene 1)

There's a considerable amount of truth to this. You have to take feedback from family and friends with a grain of salt -- they don't want to hurt your feelings. The difficulty is getting your enemies to play your game...