Methods Used
- User interviews
- Personas
- Scenarios
- Comparative evaluation
- Heuristic evaluation
- Surveys
- Usability testing
Technologies Used
- Morae
Home > Portfolio > Evaluation of Zotero
Usability and Heuristic Evaluation of Zotero
A semester-long project to use various methodologies to evaluate a software program with a client.
Date: January - April 2009
Teammates: Ke Sun, Kathy Totz, Weihua Wang
Course: SI 622: Evaluation of Systems and Services
Client: Zotero
My Role: Most work was done collaboratively with the group. I did lead us in the heuristic evaluation phase, and wrote and edited much of our survey report.
The Challenge
Work with a remote client to determine their needs in evaluating their software product. Conduct the evaluation throughout the semester using several methodologies, and create reports for the client that communicate the results clearly and effectively.
Process
As part of our SI 622 course, we were instructed to use several predetermined methodologies to evaluate our assigned product. Our first assignment was to conduct in-person interviews with five current Zotero users about their use of Zotero and attitudes toward it. We used our findings from this phase to inform our personas & scenarios, which we created next. The next phase after that was to conduct a comparative evaluation of competing products and their features. This was easy to do for Zotero since there are so many other well-know citation managers, such as EndNote and RefWorks.
In the next phase, we conducted a heuristic evaluation of Zotero, employing Jakob Nielsen's 10 heuristics as our guide for finding potential issues with the interface. Next, we conducted a survey of current Zotero users. The Zotero team helped us to publicize the online survey, and we collected over 1,000 responses to our questions. This large sample size allowed us to do statistical analysis of the results and find significant differences among user groups in their use of Zotero. Finally, we conducted usability testing of Zotero with five users using Morae to record and analyze the sessions. We focused on recruiting and testing new users who had never used Zotero before, with the goal of determining how easy Zotero is to use for first-time users.
Results
The Zotero team has been great to work with, and has been very excited about the results from our evaluations. We have worked with them to come up with ways to fix the various issues we have found; hopefully these changes can soon be implemented.
Examples of our work
Read our survey results report Read our usability testing results report