Skip to Content
 

Common Playtesting Myths

13 replies [Last post]
JohnBrieger
JohnBrieger's picture
Offline
Joined: 11/04/2016

I am fanatic about testing my games, and for the past 6 months, I’ve run 12 - 25 tests in per month (not including playing solo or with co-designers on projects). My background is in design and design research and I run user studies for a major retailer. User Experience design (UX) all about analyzing behaviors and using data from studies to drive change, and techniques from UX research have been incredibly valuable in my playtesting and design process for tabletop games as well. I wanted to start my posts on board game design with a series on how to improve the quality and quantity of your playtests. Would love to discuss

 the finer points of playtesting if people are interested.

Suggestions for more myths? Add em in the thread!

Myth #1:
"Start testing with your family and friends, then strangers."
You see this often listed as a first step in posts about playtesting, and there are many successful designers who got designs published starting this way. But you’ll notice, almost every time you see the advice to start testing with friends, all of them follow up with: “but then, make sure you get your game tested by people who don’t know you.” I think most people reach for their family and friends to test because that is what is easy/familiar, not because it’s necessarily the fastest or best way to get a game tested to completion. My view is you should jump straight to testing with strangers. I’m not saying NEVER show your game to to family and friends, just that their feedback isn’t as useful.

  1. Your family and friends are unlikely to be honest with you.
    You’ve probably heard this before, but no matter how many times you tell them that you want honest feedback, they will be biased and try to avoid hurting your feelings.
    1b: “My friends aren’t like that, we’re all mean to each-other.”
    It doesn’t matter. Everyone has subconscious biases, and people that know you will have a harder time separating a critique of the game from a critique of you. Just because your friends are more honest than most (or you perceive them to be) doesn’t mean that their feedback is as unbiased as a stranger’s.
  2. Your social and family circle is too closed.
    Even if everyone is different, the fact that you are friends and family means you likely: come from similar backgrounds, are more likely to share political and social views, are likely to have had similar levels of education, are likely to have the same primary language, and many more hidden factors. In short: even if you have a diverse group of friends and family, you think more alike and have more in common than you and a stranger. Your friends and family are less likely to catch things during testing that you haven’t thought of.
  3. Testing isn’t a central part of your relationship, so you are changing the nature of your social dynamic.
    Let’s say I loved singer-songwriter music, and I have a friend who’s a really good musician. I’ll go to her shows. and if she invites me over to listen to a preview of her new album and give her feedback, I’d do it. But if EVERY time I came over, she wanted to play her album, I’d start to get annoyed.

Myth #2
“You shouldn’t make your prototype look nice”

This is very common design advice in the board game community, but is only true to a point. Spending a small amount of extra time (for me, this is typically ~2hrs per iteration, maybe a little less) to make your prototype look better is worth it. Now I’m not saying go out and commission artwork, or spend an extra 100 hours designing the perfect custom font for your card titles, but it is worth the time to go slightly beyond bare bones and make sure your prototype is clean and professional. Much of the job of game components is communicating information, and your prototype needs to do that as effectively as possible. Learning graphic design makes you able to communicate information more clearly. So do it.

  1. Having clear card, board, and other component layouts decreases teaching time and confusion during the game.
    How players feel about the experience of a game is connected tightly with it’s pacing and narrative arc, and stopping to parse or interpret information during gameplay messes with that pacing. If your players need to spend an extra 10 min per game reading your handwritten cards or parsing the confusing chart you’ve laid out on your game board, you’ve not only lengthened your testing time, but you have negatively impacted their play experience.
  2. Having a clean looking prototype attracts testers to your game.
    People want to pick up and play games that feel nice. My next post will be about recruiting play testers. This is way easier if you can show a nice looking prototype. It doesn’t need to look like a final product. It just needs to show clear effort and professionalism.
  3. The key is being able to do this quickly.
    You don’t want to get bogged down making your prototype pixel perfect, but it’s worth cleaning it up. Learning fundamental graphic design and layout principles is worth the time investment, because they are foundational skills you can apply across all of your prototypes, rulebooks, sell sheets, publisher pitches, email templates, etc.

Myth #3:
Playtesting is about making a series of small iterations.

This one is half right: because the design process is all about iteration. My issue is people often approach their game testing as though they are conducting a quantitative study, and each small change represents an experiment against a control of the previous version. I think it’s more helpful to approach playtesting using qualitative research techniques, using technique from usability studies or ethnographic research to base playtest structures on.

Realistically, you will never have enough tests to statistically finish your game using a “scientific” A/B testing method. It’s better to make lots of changes at each iteration during earlier parts of the design process to see how the core experience and feeling changes, then make smaller incremental changes toward the end of testing. 

The majority of the balancing of your game should likely be mathematical + a little bit of manual tweaking. If you just essentially use “trial and error” over and over, you don’t have enough real data to conclude your game is balanced.

That being said:

Myth #4
Your game needs to be balanced


There are many different definitions of balance but for simplicity’s sake, let’s say a game is balanced when players of equal skill stand a statistically similar chance of winning. If a game has multiple core strategies or paths to victory, balancing between those strategies means that each on of those paths stands a statistically similar chance of winning if executed well.

Your game doesn’t need to be balanced to be fun. In fact, the PERCEPTION of balance/fairness is more important than whether your game is actually balanced. Players will continue to have fun as long as they believe that they have a fair chance. For example: let’s say you have an asymmetrical game. You’ve extensively balanced the skills and abilities of the three factions so they all stand an equal chance of winning. However, the counterplay from Faction C that allows them to stand up to faction A is not intuitive, and requires a few playthroughs of the game to understand. If the first 4 times a player plays your game, faction C loses, they’ll conclude the game isn’t balanced and shelve it. Similar to this are games where there are prescribed opening moves. If people take said “standard” moves, then the endgame is balanced, but if anyone deviates, they essentially hand the victory to another player. Here, your balance is getting in the way of players having fun and meaningfully participating during the duration of the game.

Some designer/publishers don’t consider balance while learning the game a real issue, and focus their development only on expert players who can really plumb the depths of tactics. Ignacy Trzewiczek for example, tells a story in his book about an army for Neuroshima Hex that has absurd win-rates (Steel Police) until people learn to play against it, at which point he concludes that it is in fact balanced. That was expansion content, but again I think you have a problem if it requires weeks of playing constantly for players to achieve the level of knowledge to make the game balanced. In my view this represents a flawed design.

Final Thoughts
Playtesting is an incredibly important process. Hopefully, some of the lessons I've listed here help you in getting your games testing and getting the most out of those tests.

I think I'll likely be doing some more in this series:
Recruiting Playtesters
How to run Playtests like a Researcher
Getting Better Playtest Feedback
Interpreting Playtest Feedback like a Researcher

BenMora
BenMora's picture
Offline
Joined: 02/13/2012
Thanks for the post and

Thanks for the post and sharing insight!

I slightly disagree with #1. Complete strangers willing to test your game are valuable, and they will only be able to playtest it for the FIRST time ONCE. I don't want to waste that ONE time on what I am not certain is as solid of play as I can make it. I prefer to playtest with my developers (ie, friends who expressed willingness to be extensively involved in playtesting and developing) FIRST so that any huge hiccups are out of the way, and then my fresh strangers can test it for their first time.

I am SO with you on #2! I put my heart into my prototypes such that people ask me if it's the finished game. The artwork you see here is what I made for my prototype: http://signup.moragames.com
(Granted the video shows a CG render, but it is basically exactly like that.

I started to realize your point on #3 is so true as I tested my game. I started out thinking "change 1 variable at a time" and then I realized I can iterate faster and better if I allow myself to change as much as I think will synergize better.

Interesting thoughts on balance as well. I have to chew on that more.

The hardest part for me is finding third-party playtesters. I've been able to, but I definitely don't perceive them to be abundant. Back to point #1, I make sure to make every third-party playtest count.

JohnBrieger
JohnBrieger's picture
Offline
Joined: 11/04/2016
Friends as developers probably works decent...

The thing about first times is true, as opportunities to blind test from rules or get first impression are limited. But there are a LOT of board gamers out there. And many of them are willing to test games if you present it in the right light. I'm going to probably write a whole separate thing on recruiting testers, but I test with strangers or with designers who aren't close friends A LOT. I'd say about 10-15 people who have never played a specific game of mine end up testing for me per month (more if I hit a convention). It helps immensely to have multiple games going in parallel, so if someone wants to test for you again you can have them test your other game on a separate occasion. I think it's less limited than it might initially seem.

I can usually make it to an in person testing 1-2 times a week, then I run remote testing groups through Facebook that give me another 1-2 tests a week. That's 12 session/month right there + some months I can attend all day board game events or conventions with playtesting zones. 0 of those tests are me with my friends.

Not sure where I'd put the "friend as developer" in the above. I'd put development as different than playtesting, but using developers as playtesters is probably better than using just your board game group or similar because at least the nature of your relationship is still one where they are critiquing. One for me to think about.

I've become friends/colleagues with many of the game designers who I share tests with (more on this in a future post), but this is a fundamental different social relationship, so the dynamic has always been based from the beginning on critiquing each others work.

Glass shoe games
Glass shoe games's picture
Offline
Joined: 02/28/2017
Well when I playtest with my

Well when I playtest with my fiancé she is the most critical. She will lay into me on everything.
I also found people will play your prototype more when it looks good. Digital design is easy for me but it does take some time.

radioactivemouse
radioactivemouse's picture
Offline
Joined: 07/08/2013
The list

This list that you've put up doesn't sound like "myths" as much as it sounds like, "do nots".

A myth (using the first example) is more a statement like "Friends and family are the best playtesters". No, of course they are not and you've explained it quite well :)

I will have to disagree with #2. Yes, you need to put enough information so that the player will understand the components in the game, but don't go all out on the first prototype. Placeholder art is ok and putting in finished work as you continue to test is fine too, but looks should be always secondary to gameplay in prototype mode.

Myth #4 is somewhat of an elusive fox. There are times where you absolutely need to balance exactly, but you are also right in that there's a perception of balance that needs to be achieved. You shouldn't throw away exact balance if it can be done. It's like luck vs. skill...you kinda want both and it's up to the designer how much of each they want in their game.

All in all it's a great explanation that's come from experience. Kudos.

Zeto
Offline
Joined: 03/06/2017
Agreed on most parts

Thanks for the read, really interesting!

I agree with you on Myth #1. Tho how to find those strangers to test with is usually what people have problems with. Friends and families are more reachable so people prefer to have them playtest instead. Do you have any tips on finding those testers?

I agree with you on Myth #2 too. People's expectations are way higher these days when everyone has access to the whole world. How it looks can really change the opinion of people.

Myth #4 is the one I absolutely agree with you. Games do not need to be balanced. A good game will balance itself with the meta, because everyone is given the possibility to counter other people. I prefer to buff weak players rather than nerf unbalanced mechanisms. It is always better.

pelle
pelle's picture
Offline
Joined: 08/11/2008
#1

If I agree to playtest for someone and it turns out the game is completely rubbish and was never tested at all that would really make me hesitate to help that designer again. I expect them to playtest as much as possible with friends/family to avoid that situation. Of course for stated reasons friends/family are lousy testers so I do not expect wonders, but at least an effort to not waste stranger's time. And finding playtesters is time-consuming and most are no good. Treat them well, not by sending them raw untested, unplayable games.

evyoung
evyoung's picture
Offline
Joined: 03/11/2017
Mostly agreed

Thanks for sharing! I agree with most of your points with the exception of the first.

I don't see the first few play tests as an opportunity to get real feedback on the game so much as an opportunity to see the machine in motion and to deal with any huge oversights before taking the game in front of a more critical audience.

You need a group willing to help out when the game is at its roughest stage so that you don't put strangers through something that you haven't had a chance to see working.

Isegrim
Isegrim's picture
Offline
Joined: 03/11/2017
Artwork is fun

...

BenMora
BenMora's picture
Offline
Joined: 02/13/2012
Pelle and evyoung, well

Pelle and evyoung, well said.

I think there is a danger (I am aware I am prone to it) of constantly thinking it's not ready for fresh eyes just because every time I test it myself I make one little change and think, "no they can't see it yet cause I didn't test THIS part yet.." That could spin your wheels forever so at a point you just gotta let your baby take it's first steps.

mcobb83
Offline
Joined: 06/07/2016
I disagree almost completely with #1

I don't think your perception of #1 is accurate, at least for me.

The very first design I had for my game I tested with family (my gamer brothers and not gamer dad), and it failed miserably. From that play test and the suggestions i received, my game changed from a dismal failure to its current sought after form. I have people lining up to play test it now.

In addition to that, testing on my family (once the game design moved from the dismal failure stage) opened up new avenues for testing. My siblings invited their friends to test the game, with great success as multiple siblings have brought together a group of 4+ friends to test with. The circle has expanded from there, as more and more people have agreed to test my game, and many of them are either complete strangers, or bare acquaintances. In fact, the best ideas have come from family.

However, I will say that there is value in testing on strangers too, particularly if you are as fortunate as I to have a dedicated game design group that meets regularly for the express purpose of play testing. Having access to literally dozens of strangers who design, test, and play games like its a religion is a huge boon to anyone designing any game.

JohnBrieger
JohnBrieger's picture
Offline
Joined: 11/04/2016
Really good points around the "early process"

@Ben, agreed. I think the points Pelle and evyoung brought up are really excellent, because they are real things that can/do happen.

I'd sum up this concern as: "I'm working out the kinks, so I don't want strangers to have a bad experience testing one of my games. If I test with friends and family first, I can work those out before I play with others."

There are some good ways to structure tests to avoid those issues:

For those early "not sure if it will break or not" tests, it's really key to frame expectations at the beginning of the test. In my job as a design researcher, we do this a lot as the start of a study session. The introduction to the session should help focus their feedback and make them understand why participating in the study is valuable. I take a similar approach in board game testing.

These early tests, I always conduct in person instead of running remotes. I typically lead with "This is a new design that could have some issues. If we encounter some, I'll probably stop the game in the middle and we can have a discussion about what you enjoyed and especially what you didn't enjoy, as knowing what you didn't like is what helps make a better game."

I think there are many designers that are slightly too enthusiastic about their early prototypes, and if you set up the your test with "This is a great game I've been working on and I put my heart and soul into it." even strangers might hold back critique, or continue playing after they aren't having fun to humor you.

I always am trying to communicate to testers that they will and should point out flaws in the game. It's also good to set expectations with playtesters about what level the game is at so you don't get feedback on iconography when you need feedback about mechanics, etc.

The second part of this is understanding what motivates your testers. If you can sell them on the fun of "being part of the game making process", you don't need to worry as much about them leaving a broken game having had a terrible time. Instead, they can get satisfaction from the feedback they provide you. If you can be really engaged and appreciative of negative feedback, they'll get engaged with the testing process, rather than the game.

Getting buy-in isn't necessarily easy, but lots of people really enjoy being part of or feeling like they are part of a creative process (see: Kickstarter) so it's less difficult that it might seem. Communicating that value to your testers helps them engage.

I think I want to expand this (the "Only Test with nonfriends/nonfamily" process & tips) into a full separate post at some point this week.

mcobb83
Offline
Joined: 06/07/2016
I have had success with

In my play testing, whether it is with friends, family, or strangers, or with gamers or non-gamers, I try to always highlight a few aspects that I specifically want tested. For example, at my last play test I opened with something similar to this -

(a little context- the game is a co-op game with unique characters and player powers)

"This is a test. We may find something is broken. If anything feels off or broken to you as we play, bring it up so I can write it down. We can talk about it, and if it isn't too broken we can keep playing.

But specifically I would like you to pay close attention to two things: first, do you feel like your character is as effective in his arena as the others are in theirs? Or, in other words, do you feel as though you are contributing to potentially winning?

Second, does the game feel balanced you? By that I mean does it feel like it is too hard, or too easy? Would you have preferred it to be harder or easier?"

By putting specific questions to the testers, especially if they are not regular gamers or used to playing prototypes, I find I get clearer results when I ask them questions at the end of the test. If I don't give them something to watch for at the beginning, most of the time I get something to the effect of "that was pretty awesome!" and the end and very little criticism or critique.

GameKnight
Offline
Joined: 12/18/2008
There is no hard and fast

There is no hard and fast rule about whether to start with friends and family or to go to strangers for initial playtesting. Sometimes your friends are excellent playtesters, and sometimes they are just masochists willing to do you a solid.

The bottom line is, it's easier to get friends to try out the initial crappy version of the game you have designed, and be OK suffering through your creation. If you brought your unvarnished v1.0 game to strangers, and it was full of holes, you would likely not get another chance to tap them in the future.

Friends will tolerate a half-baked concept and are motivated to help you out, because you suffered through helping your friends move in the past. Or you gave them your concert tickets when something came up in your life. You help each other out because you are friends.

Strangers want to try new games and see what new ideas are out there, but they expect it to be in relatively decent shape. They want to have fun playing games, and help you fine tune the burrs. They do not want to play a broken, highly unbalanced game that has obvious flaws. They want to know you put in a lot of work to chop away the roughest pieces and make it look somewhat presentable before bringing it to them.

After all, they are choosing to playtest for you rather than playing highly rated published games that all but guarantee a good time. Give them an experience that keeps them interested in playtesting. Make sure your game is ready for strangers.

Syndicate content


forum | by Dr. Radut