Stage 5:
Testing & Refining

We learn some tricks of the trade for testing your experience, getting feedback from possible players and what to make of it.

In this stage you will:

Learn about how to best test your creation yourself.

Learn how to get the most useful feedback from users.

Iterate, iterate, and iterate some more to really refine your creation.

Overview

Welcome to Week Four, when we will finally, directly introduce our creation to the world at large! Well, at least to a few trustworthy friends, and to some members of your target audience, to start with. As the last step of our playful design process, we’ll now be testing our creation to get feedback and iterate on its basis to improve the experience. Remember, this is playtesting we are talking about, not a design review, nor bug hunting, nor quality assurance. As in stage three, our key focus is the experience of our players, and not the technical refinement of our creation. Before diving into this stage, here’s a couple of considerations:

Firstly, playful design is a thoroughly iterative process, continuously informed and amended by feedback,  and you’ve actually been working on this basis since the very first steps of ideation and empathic engagement, so this is nothing new. The only change is that we are now specifically testing not simply an idea, but a playful, interactive experience that should be able to stand up on its own.

Secondly, let me take a few lines for a little aside: you don’t need to create something perfectly refined, there is a value in creating even crappy games, as it is possibly the only way we will create something truly innovative. However, even so, we always need to be respectful of our audiences, so unless you are using play as a medium to express some very personal concern, you should try to engage with them and ensure that the experience is worthy, interesting and meaningful for them.

This said, let’s dive into the testing!

Mission 1

Of course, the very first thing you can do (and that probably you have already done informally) is playtesting your creation yourself. While building it you surely had in mind what the final experience would have been, and we did already explore that two weeks ago. However, we now need to proof our creation before letting it out “in the wild”.

At this stage, the thing you particularly want to focus on is the flow of the game, that is how strongly players can get fully involved, and how this relates to its smooth, well-designed pacing as an experience. On the most basic level, test your creation for completeness: can you play through all of it without getting stuck? Would someone who doesn’t know the rules and the scenario in advance be able to do the same?

A way to test for this is to think out loud all the actions that you go through, and either note them down or record yourself, depending on the nature of your creation. Yes, this will knock you out of flow, but will also enable you to reflect on whether that mental state can be achieved at all throughout your playful experience. Now reflect on the learning curve for your creation: is it too simplistic and easy at times, and too obscure and hard to engage with at others? Ideally, your experience should always stay within the “Flow Channel” between boredom and frustration/anxiety.

IMAGE

Now reflect on all the steps you went through on how you changed things to enable players to engage in a more smooth way. If you really took care to verbalise all the steps, you should now be able to edit them into an open-ended explanation of how anyone could engage with your playful experience. If it’s still unclear, go through the experience again, and try to verbalise any gaps. Once you are done, you should have something akin to a tutorial and rule book (here are some useful tips and discussions on how to write them), so share it on #GChangers!

Mission 2

The second step is playtesting with people you possibly know well, and that you know that will take the time to give you appropriate feedback, but that still don’t know your playful creation. Of course, this is also a great way to introduce a lot more friends to the #GChangers community if they want to try their hand themselves at creating something playful! It would be useful to invite a relatively small (4 to 6, let’s say) but quite diverse group, and, as always, it is important to keep involved someone who is a member of your target audience, especially when your creation tells a story about them.

Before actually moving to the playtest, take some time to get to know them better, as this will help you get their perspective and the context of their experience (it’s all about empathy, remember?) and especially as regarding their habits in respect of play and games: what do they play? How much do they play? What do they think is fun? What things do they hate? (Also have a look here, to get some ideas of the current discussion on gamer motivation and demographics). Once you have a good grasp of their playful habits and tastes, have them play with your creation! You don’t necessarily need to play through the full thing, 15-20 minutes should be enough to give everyone a basic idea of the experience. Ask them to think out loud while they play, as you did when playtesting yourself so that you don’t lose anything of their experience.

Now on to the hard bits, and here’s the most important recommendation, for playtesting (as for any other kind of research with people): LISTEN! Really, just this: listen, and listen carefully, and write everything down. Do not interrupt them, do not try and respond to every point they raise, do not get defensive. It’s not going to be easy: at this point of the creative process, you will have definitely developed some attachment to your creation, but learning to actually listen and never, ever dismiss feedback and criticism is possibly the most important skill a designer can have. Still, it’s good and safer to start with people that you know.

After the play session, open an informal discussion and try to touch not only on their general thoughts (first impression, pacing, best and worst moments) but on the formal elements of the experience (objectives, choices, rules, loops, interactions), its narrative (characters, development, appropriateness), usability (rules readability and clarity, interface) and any general further ideas they might have for revision. Once again: listen, and do not lead their answers, no matter how strong you feel toward their feedback. Note everything down, and once you are finished take a photo of your feedback and share it on #GChangers! 

Before moving to the next mission, you should also update the rule book/tutorial you created in Mission 1, to include any aspect of the flow of the experience that is still unclear to your audience. Once you are done, get ready to show your creation to the world!

Mission 3

I know, this might have been a bit hard. But this Mission is going to be even harder. But first, let’s take a step back: go through your notes once again, and try and write a script of how the discussion went. Think back to your first playtesting experience, and reflect on the questions that yielded the most interesting responses, and write them down. In the same way, take into account some unforeseen direction the discussion took, and try and integrate it as a question in this script: effectively, you are now trying to create a questionnaire for your playtest. Given the huge diversity if your creations and your target audiences, we can provide you with a general one and, although, of course, there are “industry standards”, once again they are mostly fixed upon digital games, and you and your testers are the best judges of the specific context and issues that need to be discussed when navigating your playful creation. Once you are done, please do share it on #GChangers!

This questionnaire will be useful because the next step will be playtesting with people you do not know, and without you in the room (using both the rule book/tutorial you wrote in Mission 1 and this questionnaire) or, as it is called in the field, Blind Playtesting. Again, try and gather a diverse group for your test, though obviously, it might be harder than the first time.

Game related internet forums (such as those you came across last week) and game fairs might prove great “hunting grounds” for prospective playtesters, possibly with a high level of game literacy and able to provide you with in-depth feedback on the mechanical aspect of your creation. Often they even provide specialised spaces for this purpose and can support you with data collection to the booth. However, since we are trying to create something that might have real-world impact, your best choice, in this case, will probably be a local association that focusses on the same issue around which you built your creation, and, as a community, we would be happy to support you in getting in touch with them and support the case for playful engagement. Let us know if you need our help via #GChangers!

…And of course, when you have your first blind playtest results, also share those via #GChangers!

So, how were the results of your testing? Was it harsh? Was it useful? You might have noticed that I did not tell you what to actually do with all the feedback you have. This because, again, playful design is inherently iterative, which means you now have more information to go back on your steps and reconsider the design process as a whole.

Keep this in mind: as noted designer Greg Costikyan wrote, “listening does not mean obeying”. Not everything your tester said should be incorporated in your design, but it’s all useful to inform your choices.

So having gone through the last four stages of the open course, you should now quickly go through them again, and have a look at all the questions that I posed you, and how would you answer differently now that you have plenty more information. Does it look like it might take a few more weeks? Yes, it might, particularly as, in the spirit of iteration, you will have to go through this again! This might start looking like a Sisyphean task, however, do not despair, and keep in mind that design is never finished, it only stops.

When does it stop? Well, for commercial projects that happens when you run out of time and/or money. But for a DIY project, you set your own criteria.

We’ll discuss those (and indeed ourselves stop after that) in the final stage: Publishing and Play!

(and yes, there’s going to be more to it than that 🙂 )

What you’ll need:

  • A playable prototype/ruleset of your game/playful experience.

  • A few people of sufficient diversity: friends, gamers, non-gamers, your target audience, anybody else.

  • Good connectivity to sift through a ton of resources, and to share your thoughts using #GChangers.

Check these resources out; they're really good!

Join the Party!

Now you’re a Game Changer why not join the #GChangers community and share your adventures in playful education.
Playing is a social activity, so let’s share!