Date: Thu, 26 Feb 1998 14:24:10 -0600
From: escargo@anubis.network.com (David S. Cargo)
Message-Id: <199802262024.OAA10110@brutus.network.com>
To: calvin-ui98@dagobah.stwing.upenn.edu
Subject: Cargo's Heuristic Evaluation
Here is the content of my heuristic evaluation of our application.
David S. Cargo
CSci 5110, Winter '98
"Heuristic" evaluation
Simple and natural dialog
- Simple means no irrelevant or rarely used information.
Natural means an order that matches the task.
Speak the user's language
- Use words and concepts from the user's world. Don't
use system-specific engineering terms.
Minimize user memory load
- Don't make the user remember things from one action
to the next. Leave information on the screen until it's not needed.
Be consistent
- Users should be able to learn an action sequence in
one part of the system and apply it again to get
similar results in other places.
Provide feedback
- Let users know what effect their actions have on the system.
Provide clearly marked exits
- If users get into part of the system that doesn't
interest them, they should always be able to get out
quickly without damaging anything.
Provide shortcuts
- Shortcuts can help experienced users avoid lengthy
dialogs and informational messages that they don't need.
Good error messages
- Good error messages let the user know what the problem
is and how to correct it.
Prevent errors
- Whenever you write an error message you should also
ask, can this error be avoided?
------------------------------------------------------------
Group 4's application is what we have called the Mail Archive
Converter. It is an application that takes UNIX mail-format mailboxes
and converts them to a set of HTML pages with links between the pages
and with four indexes that allow viewing the pages organized by thread,
by subject, by date, and by author.
The application is currently organized into a tabbed display with six
tabs. We attempted to make the tabs form a natural order that matches
the task. If we can determine that functions on a particular tab are
not necessary, the tabs are grayed out. (They are not inactive, because
we wanted to allow exploration.)
The first tab is the "Project" tab. In earlier definitions of this
application, this meant more than it does now. Since "project" doesn't
seem related to mail or HTML, it probably no longer speaks the user's
language.
Also on the first tab, we use the word "filters." Novice users might
have no idea of e-mail filters, although there are filters for more
sophisticated mail agents; in this case, the word "filters" might not be
communicating in the user's language.
The first question also uses the word "discarded." It is not clear from
here what is really meant, which is that a message that is discarded
simply doesn't have an HTML page generated for it; it's not the case
that it might be removed from the original mailbox. The question could
be worded more clearly.
The second question meantions modifying the subject of a message. It
might not be clear why anybody would want to do that; again it's not
clear that this means changing things in the generated HTML page and not
the mailbox.
The third and four questions ask the user if there will be one or more
than one input mailbox. This pair of questions was to prevent errors,
since knowing this allows the application to infer something about the
user's intentions, and to do some consistency checking.
Selecting the answers to these questions causes tabs to become grayed
out or black if the functions on the tabs become enabled. A more direct
form of feedback is possible.
The tabs provide what we thought were clearly marked exits, but it would
be possible to add "Next>" buttons on the pages to give a firmer notion
that steps could be done in sequence.
It's also not clear when certain steps can be skipped if default values
are acceptable; this is a feedback problem to be resolved.
No shortcuts as such are provided. This is related to the problem above.
The second tab is the "Mailboxes" tab. There aren't many controls on
this tab.
The entry field does not accept return as an action equivalent to the
add button. (I just hadn't added the approriate bind command yet.)
This is not consistent with other entry fields; more work on them is
known to be needed.
If the user does not see the "Add" button, it may not be clear what the
user should do.
The users should know what mailboxes are, and labels of the buttons
should be clear.
Feedback for the buttons is immediate, since there will either be an
item added to the listbox, or an error message.
One error message problem that appears here is that when the question on
the project tab indicates that only one mailbox is being used, a second
mailbox can't be added to the list. The size of the listbox doesn't
seem to reflect that limitation, and the error message does not direct
the user back to the project tab to change the answer to the question.
Again, relying on the order of the tabs and their grayed out state does
not provide the most clearly marked exits.
The third tab is the "Archive" tab.
Like the word "project," the word "archive" may not mean the right thing
to the intended users. This tab really controls functions for the
generated HTML pages, which is what we call an archive. It's unclear to
me what term in the user's language would describe what we are creating.
Another problem is that some of the text refers to the "test archive"
and the "real archive" and those may not be clearly distinguished
enough.
After answering questions and filling out or not filling out some of the
fields, there is no obvious feedback that what was done was right or
wrong. There is also no way for the users to find out. Feeback clearly
needs to be improved here.
In cases where the user says that there is one input archive, we provide
defaults for all of the fields. That could extended to every case.
The fourth tab is the "Filters" tab.
Again, beginning users might not know what filters are, nor what they
can do, nor why they would use them. Perhaps each page should have a
little "about ..." button that pops up an explanation for the feature.
There is a bug in the error checking for the "before" date; even if a
date is filled in, it says the field is empty. That is a problem
probably easy to correct.
Another bug seems to be that the "Current filter" entry field accepts
data; it is supposed to be diabled so that it only displays data.
The filters tab is an advanced function that beginners won't need, so
some complexity here seems unavoidable.
The fifth tab is the "Processing" tab.
Even if users need to do something here, nothing forces them to come
here. In that sense the natural order is not being enforced by
anything, nor is guidance being provided except by the fact that the tab
is not grayed out.
Once again, the concept of filters may be outside the beginning user's
language. One of the interactions between the filters tab and the
processing tab is that the user can delete filters that are used by
mailboxes on the processing tab. This needs to be prevented.
Some of the feedback that could be provided that isn't is that users
answered a question on the project that that said that they needed
filters, but if they haven't created any or applied them to any
mailboxes, then something important has been omitted.
The last tab is the "Generate" tab.
Since we are calling the application the Mail Archive Converter, this
represents an inconsistency; it should probably be the "Convert" tab.
The fields on this tab are intended to provide feedback to the user; in
the sense that they don't see them until they get here, it is not
immediate feedback. Additionally, if there is a problem, the users are
not directed to the tab where they can fix the problem. (Nor is there a
shortcut button they can push that will take them there directly.)
If the text fields are considered as error messages, they could be less
terse and more clear.
There is an inconsistency between the "From mailbox" and "To archive"
text fields; one shows a full path, and the other shows only a directory
name; these should be made consistent.