Usability test

We did our usability test with our social group members individually.  This is because it was hard to control an entire group as we had experienced in the past especially with our participatory design workshop.  By doing individual sessions we could isolate each member’s feedback from the group and make better decisions.  In group settings one member will often dominate the discussion and thus the feedback.  We wanted to avoid this scenario.

Three out of four of our participants for the test attended our previous workshop. Two of them are experienced dungeon masters and the other two are mainly players. We made appointments with each of them and had them be their normal role in the game while we filled the other roles to make for a playable session of Dark Heresy. For example, if they are the usual dungeon master, we will have the dungeon master user test, and we will be the players.

There are two main parts of our application that we wanted to test. First, it is the map generating part, where the dungeon master can create their own desired map or random a map and edit on it. The second part is to test the players and dungeon master as they use our application through the battle phase.

Usability test for dungeon masters:

Tasks:

1: Randomly generate a map
2: Edit the map to your desired specs
3: Play a battle with two players (us)

Questions:

1: What do you think about the application?
2: Did you encounter any problems using our application?
3: Do you find the map generating function useful?
4: How do you think an electronic application related to Dark Heresy could be integrated into your games?
5: What would you want to change or add that was not available?
6: Any other comments or feedback?

Usability test for players:

Tasks:
1: Play a battle with the dungeon master (us)
2: Make an icon
3: Move player icon
4: Check turns order

Questions:
1: What do you think about the application?
2: Did you encounter any problems using our application?
3: How do you think an electronic application related to Dark Heresy could be integrated into your games?
4: What would you want to change or add that was not available?
5: Any other comments or feedback?

Finding Summary:

The battling portion of the Dark Heresy game is very difficult to control because it is very complex. There is already an extensive amount of characters the game utilizes; each has their own actions they can take which the DM must keep track of.  Our application was successful due to its very simple design.  There were not a lot of buttons and not a lot of screens.  At each screen there were minimal actions the user could take which often led them to what they wanted even if they were unsure. Thus, the prototype improved the process of setting up a battle field. What was especially helpful is that this can help visualize most, if not all aspects needed to for a battlefield. One participant suggested that we show the direction of the players and monsters are facing as this is very important to battle mechanics. The Dungeon master said to improve the battle phase, the turn orders should be able to be manually changed during a battle instead of it being set. The participants liked the fact that it shows the turn orders on the application and the location of each character. With this application, they do not need to repeatedly ask the dungeon master the distance of the monsters. The dungeon master no longer needs to flip back and forth on the book to look at the movements list and the map. This will make it more convenient for everyone to play through the battle. All four users said it would be neat to be able to use this application during battle.

There were no large difficulties found with the test.  Some of the map editor tools weren’t immediately understood and we had to change the icons to better convey what was meant by the tool.  We initially had a paint brush to “paint in” landscape but this was not figured out by anyone so it had to go.  Again, the map generation was really simple and there are only a few options they can choose from.  They said it was okay that it was minimal because they don’t need incredibly detailed specs to play.

Some quotes from the test were:

DM: Finally, they can stop asking me how many units away enemies are.. you wouldn’t believe how many times I have to repeat things to them.

Players: I’m glad we can all have the same picture of what is going on, it helps us strategize together more effectively.  This app may save our lives one day!

Players: The turn order is so nice and I’m glad I don’t have to hold the map page anymore.  They always make me hold the map page..

Players: It’s really fun to huddle around the prototype or app and strategize while the DM looks on.  I can’t wait for this to be on a tablet.

DM: I liked the map generation.  I don’t really have specific ideas for what I want when I’m creating maps so a random generator is a nice easy start for me.

Players and DM in response to a post test question: We could definitely integrate this into our game play.  We aren’t old school and as you can see already use a laptop to help us take notes.  There just isn’t enough room for us to all bring one.  In that case technology could get in the way but a simple tablet that can be passed around actually brings us together more.

Reflection

The usability test had two parts based on how our application is going be be used. This helps us test out every part of the application from creating a map to using the map for a battle. We tested our application with our social group members individually, and received some insightful feedback. They all liked the designs of the interface at first glance. It is clean and simple with good affordances. They used the app as if they had seen it before.  It is easy to learn and use. If we had more time to improve the interface, we would like to test the directional display for the player and enemies. Also the movement of the character would show lines, which help players calculate the steps they have moved. Adding some animation when player/enemies attack would improve game playing. In conclusion, the application does great job help the user visualize the battle, it still can have some minor improvements.

Additionally we think we should have had a more structured usability test despite our club enjoying less structured activities.  If we had filmed some of the game play in early stages we could actually count page flips required without our app and then page flips with our app.  We know we have improved the process but by how much we cannot say.  It would be better to say we reduced 40% of pageflips rather than we simply reduced the number of flips required.  We also could have recorded “error” data from the test as well when users made mistakes but since they were just learning it it was hard for us to distinguish between what was an error and what was simply learning.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s