Skip to Content

Simply baffled by how difficult it is to get Blind Play testers!!!

16 replies [Last post]
genericm
genericm's picture
Offline
Joined: 08/11/2009

Perhaps its simply a series of unfortunate events or just the nature of the beast but I seem to be consistently blocked from getting my game blind tested.

In my first effort I sent out a call on BGG for volunteers to receive a high quality hand made prototype in exchange for a few honest opinions or reviews. After many hours cutting cards and laminating components I selected the two most (seemingly) enthusiastic volunteers and mailed out the boxes to the tune of $300 bucks. That was 5 months ago and I never heard a thing back.

Attempt number two for blind play testing seemed to be fool proof. I found a local game design contest hosted by an area convention and submitted my game. I figured even if I didn't win I could attend the event and observe those playing the game and their responses. The feedback would be invaluable. But the contest was a complete mess, due to a loss among the con's volunteer ranks the design contest was not well publicized and most of the games didn't even get played. I spent two days waiting there for nothing.

Now I'm resolved to not put all my eggs in one basket. I simply cannot afford to create the prototype in large enough numbers to mail out many more of them, so I have found a new medium to deliver the game... digitally!

I have just finished a complete Vassal module for the game and hope to recruit any and everyone interested to try it out and offer their opinion. I was initially overwhelmed with trying to learn the Vassal program but now I am very happy with the results and think the game play experience will translate quite well. Of course some audio communication while playing would help immeasurably but the core game is delivered. The rule are posted and everything needed is supplied to allow anyone to try it out.

http://www.vassalengine.org/wiki/Module:Exile_Sun

My question to this community is: Have you used Vassal to test your own games? And what other advice would you have for me as I try to get more testers?

I look forward to your responses

EM

Dralius
Dralius's picture
Offline
Joined: 07/26/2008
Blind in one eye cna't see out of the other

Eric

Online tools like Vassil are great for testing but i haven’t had any better luck getting testers with it. Its defiantly hit and miss whatever you do.

The main problem with blink testing is the quality of the feedback, when received, is often inadequate or non-descriptive. If you’re lucky you'll find a group that’s willing to work with you on a regular basis and is capable of giving you useful feedback.

Some of my best testing comes from conventions. I don’t know the people and I’m not a big name designer so they feel free to give a honest opinion. When I get responses that are inadequate like “ I didn’t like the turn order mechanic” I can ask why they didn’t like it. Also observing the game being played gives you context for the feedback. Did someone get ganged up on? Did one player become bored and drag the rest of the group with him? Were they playing it properly?

Good luck.

Maybe when I get a solid group for testing together again we can have a prisoner exchange.

David

Wagydan
Offline
Joined: 02/16/2010
Gamestore

Hi,

I can only tell you that there is a monstrous advantage to monitoring players play your game in front of you in stead of waiting for a digital summary of a game experience.

For me, testing Ortus was much easier since it's a 2 player game that you can teach anyone in 5 or 10 minutes, is portable and over in about 45 minutes. Exile Sun seems to be a real hardcore gamers type of game, more players and longer playtme. So you will need some dedicated players with some time on their hands.

So why not try your local gaming store (loads of geeks)? In Holland at least, there are a bunch of people hanging around these places and many will be happy to help. Plus there is room to do so. Maybe put up a poster announcing you are setting up a blind playtest in a couple of weeks. Pick a date and bring beer and pretzels. This way you won't have to wait for a convention to come along.

Good luck!

hulken
Offline
Joined: 04/18/2009
I also agree with Dralius. I

I also agree with Dralius.

I have used a difrent way to do the blind playtest. I have purpusly excluded some of my friend in my gaming groupe from playtesting. This way I can use them for blind playtesting. ^^

But I think mostly it depends on what type of game you made. I just took a sneek peak at youre screenshot on the vassal madul. And it looks like it is a rather complicated game. And then I would say use onley blind testers you met face to face. Sending out games to "random" people will probobly most often get the result you got. Namly nothing. If you do like Dralius says and sit down with a groupe at a convention. And just observ then youre more likley to find a groupe that are willing to play the game. Just sending the prototype to some one, then the ods are prity high the game wont match his mental pictur of the game, and usualy he have built up some expectations about the game also.

The other way to do it would be to get some one from the internet that you have spocen to a lot and maby even helpt with some of there playtesting. This way you have a understanding and probobly a unspocen agreement aswell. He will probobly go "he helped me so ofcors I whould help him".

The last two options I would say is the best and most likley ways to suceed with youre blind playtest. If you want you can PM me and I can take a look at the rules and give you some feedback on them if you like. Sort of a blind rules test ^^

rcjames14
rcjames14's picture
Offline
Joined: 09/17/2010
Testing Blindfolded

You do not need anonymous people. You just need interested people. And the best way to do that is to sell it to them. Give them something to look at (graphics/logo) and be able to tell them what it's about in a sentence or two and you should be able to coax enough people at a hobby game store, match up event, college gaming club or a convention to playtest it if you don't already have a group.

If you do have a group, then try them first. Just don't explain the rules to them when you hand it over to them. Let them read it, try to figure it out and then watch them play.

But blind testing is always the last stage of playtesting. You need to pretty confident that your rules are accurate and your game makes sense and plays smoothly before you blind-test. Otherwise it will very quickly break down and provide you with little information.

If the game is quite complicated, you will want to explain how the game works at least a dozen different times before you hand it over to someone who knows nothing about it and cannot ask you questions about it.

But, remember, almost everyone is taught the rules to a game by someone who already knows them. So, people are used to listening to others explain the rules and you often can count on visual, verbal and tactile elements to circumvent a lot of information that must be formally presented to someone who has never heard of the game and needs to know what can happen in every contingency.

So, before you blind-test, you need to know that your rules are super clear. And, by that point, you probably already know whether it is a game worth publishing or not. Blind-testing is there only to make sure that people can learn the game without someone to explain it.

RacNRoll Gaming
RacNRoll Gaming's picture
Offline
Joined: 05/12/2010
Playtesters suck.....

....when they never playtested before (for the most part in my experience) They are always so gung-ho to volunteer but when they get the playtest package and realize it isnt a finished product they cant get past the fact that they are playing with pieces of paper with little or no art on them.

I blind tested one of my games with a group of people I hand picked from another gaming web site because I saw the kind of games they played and the way they analyzed the game we were discussing on the forums etc. Now while I couldnt make 12 prototypes to send them i did take the time to pretty up the print and play package adding graphics to the tiles and counters.

Of the 12 people I got 1 recorded play with feedback from a guy in Australia and everyone else flaked out on me. My one bright spot was a younger player who actually took the game with him to camp where he was a counsellor and he played it with the other counsellors. They all liked the game however they weren't really gamers and so they played the game the way you would play a video game more than the way you would play a strategic board game. Also alot of their feedback was not so much about the gameplay itself but what they wanted to see in the finished product to make it "better"

The funny thing about it all is that I did offer them compensation for completing the task but none of them could do it.

Live and Learn...I am currently putting together a group of dedicated playtesters who will not have any access to the games being worked on until they are ready for blind play testing. Pre-screening is the way to go...it also gives you a good idea of where they are coming from based off of the games they already play and like.

I am more than happy to play-test any board/card game you guys have. (Not war games or RPGs though...sorry)
You can email me at racnrollgaming@gmail.com

jwarrend
Offline
Joined: 08/03/2008
Eric, My experiences have

Eric,

My experiences have been somewhat similar to your own. I've put two games up for blind testing on BGG. One game elicited a huge response (the theme is a big draw), with many groups expressing interest in either printing it out or playtesting the prototype. Of the four groups that offered to print out their own copy, only one followed through, and his feedback was excellent. Of the three or so groups that I sent the prototype, one just played it but never provided feedback, the other provided some minimal feedback but was pretty difficult about actually sending the game back, and ended up keeping it for almost a year. My other game elicited only one response but their group played through the game four times and gave solid feedback. And I entered that game in the same contest as you, and was similarly disappointed by the lack of feedback.

The question I've asked myself, and that I'd ask you, is, what role is blind testing supposed to serve? It's one of those game design rules -- "you must blind test your game!", but for what actual purpose? I've found the feedback from blind tests very difficult to interpret, because you don't know all the context -- did they get the rules right? Are they prone to like this kind of game? Are they putting effort into it or is their interest in playtesting lackluster? etc. So I'm not convinced blind testing is important at the designer level. Maybe at the developer level, where a game is ready to publish and you want a consensus of players to tell you it's fun, worth buying, and the rules and components are functional.

I would assume you'll get less response with Vassal than you had with sending protos, since there are many fewer players who use Vassal than who could play a physical board game that was sent to them. But hopefully I'll be wrong!

Good luck.

pelle
pelle's picture
Offline
Joined: 08/11/2008
why

My playtest experiences are pretty similar to what others already said. Most playtesters don't report anything or just plain results of one or two plays. But the few playtesters that do take the job seriously has so far made up for it, so I guess it is a matter of sending out copies to enough people to get a few of the good ones.

jwarrend wrote:

The question I've asked myself, and that I'd ask you, is, what role is blind testing supposed to serve? It's one of those game design rules -- "you must blind test your game!", but for what actual purpose? I've found the feedback from blind tests very difficult to interpret, because you don't know all the context -- did they get the rules right? Are they prone to like this kind of game? Are they putting effort into it or is their interest in playtesting lackluster? etc. So I'm not convinced blind testing is important at the designer level. Maybe at the developer level, where a game is ready to publish and you want a consensus of players to tell you it's fun, worth buying, and the rules and components are functional.

For me, most of the blind playtesting was indeed on the developer level. In fact the publisher, not me, went looking for playtesters (after testing it a bit internally). But with a small publisher the designer may end up doing a lot of the development work anyway (at least when it comes to looking after playtesters and evaluate their reports).

I also did some blind playtesting during the design phase, and I really think you need to do that as well, although I didn't send any copies to anyone I just grabbed some friends of a friend at a game night and asked them to sit down and (try to) play the game for a few hours. If I had spent more time with the design before going to development (a phase that on the other hand lasted for well over a year) I would have done more blind design playtesting.

Quote:
I would assume you'll get less response with Vassal than you had with sending protos, since there are many fewer players who use Vassal than who could play a physical board game that was sent to them. But hopefully I'll be wrong!

VASSAL for me has been good for playtesting, but so far not for blind playtesting.

pelle
pelle's picture
Offline
Joined: 08/11/2008
vassal

Regarding fewer people being able to play with VASSAL: The good thing about VASSAL is that chances are you will find playtesters scattered throughout the world, and using VASSAL they can team up and play each other, which may result in more and better games than if each tester tries to convince their local gamers to playtest with them. That's my theory anyway. So far, as I said, it hasn't resulted in any real blind playtesting for me, but I have playtested using VASSAL with me as one of the players (non-blind playtesting).

ReneWiersma
ReneWiersma's picture
Offline
Joined: 08/08/2008
I'll echo what everyone else

I'll echo what everyone else has said. It is difficult to get people to blind playtest your game, and if they do it is hard to assess their comments, because you can't put it into context. So, basically I don't do it anymore. I playtest a game with my friends, and if I feel it is good enough I'll expand my circle of playtesters, but I'm always around to monitor how things are going.

Pastor_Mora
Pastor_Mora's picture
Offline
Joined: 01/05/2010
My 2 cents

Well, I hear you brothers… In my opinion, the difficulty of finding playtesters varies according to several factors regarding the game itself; the main been: broad appeal theme, number of players, playing time, number of components, rules length and the overall level of completion.

This is the same as saying that you’ll have more chances of running into a volunteer tester if you have a common theme (like space exploration), that plays ok with 1, best with 2 and up to 4 players, plays in 60 or under 90 minutes, can be printed in less than 10 A4/Letter size sheets of paper (in grayscale), with 3 to 6 pages of rules, and the design shows you’ve been devoted to it for a while.

My experience with Roman Emperors (that fit all the previous standards) was that I found many people willing to play the game, but few of them committed to review it in a detailed manner. As much as isolated comments proved very valuable in many aspects, as the designer, you are forced to devote a great deal of thinking for every “what about changing this” comment you get. Since a game is a complex system of interlocking mechanics, you have to see it working as a whole before replacing a part. Casual contributors create quite a challenge concerning this. On the other hand, if you have a complete and detailed review, spiced with comments on the actual gaming session, now you can take feedback with a lot more confidence.

Disclaimer: I think rules are the most important part of the game. The rules must be as complete, concise and proofread as humanly possible for me to commit to playtest.

As for sending prototypes, I would only do that (willingly) to publishers, even if they are only interested in it as playtesters. I think someone involved in the publishing business understand what is expected by you when you send the game, can provide feedback in a concise manner, and can give you insight about the publisher perspective (taking production / costs concerns in account). I wouldn’t trade that specialized feedback for 50 kids in a FLGS saying they “liked the game”.

This could go on for a while. As a side note, I’ve been AWOL because my newborn (Luke) has arrived! I’ll be on the next version of Roman Emperors, and my promised playtesting review (sorry Daniel) as soon as possible.

Keep thinking!

genericm
genericm's picture
Offline
Joined: 08/11/2009
Thanks for your comments

I appreciate all the responses, ill summarize what I have gleaned from your perspectives:

I'm beginning to see that 'Blind' testing is a little less important than I realized. While true it is very helpful in making sure your rules are understandable and well laid out (which is necessary for submission), but all things considered, it doesn't help you improve the game design itself.

I guess my hope for Vassal is not to truly 'Blind' test the game, but rather to expose the game to more people and make notes on how they feel about:

Does the game appeal moderate gamers?
To the Pros of the game out weigh the cons?
What do you feel where the most negative aspect?
The most positive?
Are there any mechanics that you felt were thematically out of place?

Things like that.

I'm off work today of if anyone wants to browse the module with me to show them around, just email me and ill jump online. (genericmailid@gmail.com)

Thanks again for the responses.

salish99
Offline
Joined: 02/22/2010
vassal?

I never used vassal, and prefer to pnp the stuff myself, then hand-test it, usually with carcassonne meeples and other markers like stones or mussels, to avoid printing tedious markers.
Some of the results can be seen here:
http://www.boardgamegeek.com/user/salish99

Uhm, and feel free to send us a laminated copy, that will give it definite prime attention and ascendancy over the other odds and bits we regularly test. :-)
To be fair, we only once received a pre-printed game to test, and did that the same night arrived, in all other cases, we never saw the final product (to be fair, all we ask for is to be mentioned in the acknowledgments).

We;ll take a look at the exile sun later.

salish99
Offline
Joined: 02/22/2010
?

Hi Eric.

I just saw that Exile Sun (nice title, by the way) is a newer version of Crisis.
Uhm, may I kindly ask you to add us to the playtesters section? Thank you.

R1773R
R1773R's picture
Offline
Joined: 03/18/2010
Dude

Seriously

Why would you send out anything? Take your time and find testers around you. Family, friends etc...

I just watch people's mood while they play the game and work with that :)

pelle
pelle's picture
Offline
Joined: 08/11/2008
blind

At some point before the game is published it has to be tested by unknown external players that can look at it with fresh unbiased eyes like buyers of the game would. Can't use friends or family for that. Maybe you can get away with not doing that and leave that step to the publisher/developer, but I doubt many publishers will bother much with a game that has only been played by some unknown designer and his friends.

jeffinberlin
jeffinberlin's picture
Offline
Joined: 07/29/2008
Blind testing really necessary?

pelle wrote:
I doubt many publishers will bother much with a game that has only been played by some unknown designer and his friends.

Is blind testing absolutely necessary to get published? No, as long as you have playtesters who represent well your target group and who are not afraid to give you their true opinion (and who are willing to test it again and again and again...).

To be honest, I've only done true blind testing once, and that was with a friend in the U.S. who had a game group that liked games similar to the one I designed. I knew I could trust him, and he's an experienced game reviewer who gave me some very detailed, very honest feedback, which was extremely helpful. My friend, Bernd Eisenstein, also sent his self-published game Porto Carthago to the well-known German game club Westpark Gamers to blind test, and they were able to give him good feedback as well (he knew them, of course, as it is not their habit to play unsolicited prototypes).

Of course, I've also received good feedback from the blind playtesters that publishers used to test my prototype when I send them in for consideration. When you send your game to a competition like Hippodice, you also get feedback (at the very least, you find out if it it made the final round of consideration).

I would never send a prototype to someone I did not know and trust, however. It's best to find gamers in your area who like to play your game. My first design goal, after all, is always to design a game that my friends and I like to play.

And if you can't find enough people around you who are willing to try it (or they try it once but are not excited to play it again), you may have to face the hard truth that there aren't enough people anywhere else who would play it, either.

Syndicate content


forum | by Dr. Radut