Date: Wed, 18 Feb 1998 12:18:07 -0600 (CST)
From: "Jeffrey M. Rye" <ryex0006@itlabs.umn.edu>
To: "David S. Cargo" <escargo@anubis.network.com>
Subject: Putting together a user test.
In-Reply-To: <Pine.GSO.3.95q.980218103212.1735A-100000@bambam.itlabs.umn.edu>
Message-Id: <Pine.SGI.3.95.980218115718.8358B-100000@sparrow.itlabs.umn.edu>
For part two of the assignment, we need to put together a test plan for
our users.
What follows is my part one. It should give us a good basis for our test
plan.
Part 1:
Background:
Briefly describe the purpose of the program (i.e. "MAC is designed to
take email archives and convert them to web pages.") If the user
has questions about why or how, give answers that do not talk about the
interface.
Tell the user that this is for an interface class. Let them know that we
are not evaluating them; rather, we are evaluating how easy to
use/natural our interface is for them. Also mention to them that it is a
big help to us if they can think out loud throughout the test, and tell
us what they are thinking as they make selections. Remind them that
these thoughts are for us to determine if the interface is working
properly.
Maybe also explain what we will be noting. This has the advantage of
making their comments more directed at our improvements. Unfortunately,
it might make their actions less natural, and make the test less
effective.
Tasks:
For sure do a simple task (i.e. take two archives a and b, where we give
them the names and locations, and convert it to HTML in directory c,
which we also give them). Depending on time, and on how well they
understand the interface so far; have the user do another conversion,
this time incorporating filters. Potentially they may need some
explanation to use the filters properly.
Our Role:
We will stand back and watch the user. We will record stuff. We will
also ask the user questions to figure out why they did the wrong, or the
right thing. We will also give them guidance when necessary, or
appropriate.
If everything works great for them, we will need to do very little
guiding; however, we will be prepared to point them in the right
direction (without leading them).
Help:
We want to give as little help as possible. We will intervene in a
couple of situations: the users asks for help, or we see the user is
lost.
If the user asks for help, we will start with general instructions (i.e.
instructing them to specify an ouput directory next). If they still need
guidance, we will tell them what tab they should be on. If they need
even more assistance, we will give it only if it will get them to another
part of the task that we want to see tested, and we feel that their
continued use of the system will benefit us.
If the user appears stuck, and doesn't know what to do, we may intervene
to get them on to the next step (so the test doesn't last too long, and
doesn't frustrate them).
Record:
We will record places where the user appears to be struggling to
determine which selection to make. We will record where the user gets
lost and doesn't know where to go next. We will record any out loud
thoughts the user has that will help us improve the interface later on.
we will also record any parts of the task where the user has no trouble
figuring out where to go/what to do.
General:
We want to remember that we are testing how easy the interface is to use.
We are NOT teaching them to use the interface. We do not want to give
them instructions like, "Follow the tabs from left to right." This kind
of instruction doesn't tell us if they naturally get it, it doesn't help
us evaluate the interface.
The user should be able to use the interface with little guidance. If
the user needs guidance, it is somewhere that we need to try to improve
the interface.
We should pay attention to the users reactions. When the click on a
button, does it do what the thought it would do? If they are surprised
that a button doesn't do something, we need to make it clear that that
is not what the button is supposed to do, or we should make it so that
the button does what the user expected. If the user is surprised that
the button does do something, we need to make it clear that the button is
going to do that something, or make the button not do that something.
Is the user able to make any undoable mistakes? Can they figure out how
to undo their mistakes? Can they figure out that they made a mistake?
____________________________________________________
Jeff
ryex0006@itlabs.umn.edu
jrye@pigchamp.caps.umn.edu