Allison Bloodworth, ETS—Learning Systems
Ian Crew, IST—Data Services
|
In the previous articles in this series (User-centered design for IT services: the low-hanging fruit, User-centered design for IT services: The usability toolbox, part 1, and User-centered design for IT services: The usability toolbox, part 2), we covered some simple techniques you can use to improve the design of any IT product or service including websites, web applications, and desktop programs. In this article, we will give a brief summary of how we have applied these techniques to three recent projects here at UC Berkeley.
In spring 2007, Information Services and Technology (IST) and the Office of the CIO (OCIO) were nearing the end of their reorganization. It was decided that a new, permanent IST/OCIO staff website should replace the "transition" website used to inform staff of new developments, processes, and procedures during the transition to the new IST/OCIO organization. The new IST/OCIO staff website would include information from the "transition" website, any pre-reorganization websites with information which was not outdated, as well as any new information that was needed.
When all the necessary information had been collected from the various existing websites and domain experts, it was catalogued and listed on index cards in preparation for a card sorting exercise (see User- centered design for IT services: The usability toolbox, part 2). A card sorting exercise helps designers understand how users would group information in a particular domain. Each piece of information to be organized is put on an index card and participants are asked to sort the cards into groups that make sense to them. Determining the proper granularity for the large amount of information collected was difficult, and the evaluators ended up creating more than 100 cards for the card sort participants to sort.
It was decided to do "group" card sorts both to get as much feedback as possible, as well as to allow the facilitators of the card sort the chance to observe the discussion and debate within the groups. There were five card sort groups of three people each, which included participants from many different levels and every unit in the IST/OCIO organization. As this was an "open" card sort (meaning the categories were not predefined), participants were given blank index cards on which to write the categories into which they would group the cards. The groups had an hour to an hour-and-a-half to complete the card sorts. Some groups finished the sorting relatively quickly, and some groups took longer because they had more debate about the categories. A key learning from this process was that having too many cards could be frustrating for participants, and that it may have been helpful for the evaluators to create fewer cards that were a little less granular so the groups could finish the exercise more quickly.
Watching five different groups allowed the evaluators to see common patterns in the way the participants viewed the information, and several of the groups came up with very innovative ways to intuitively organize the information. In the end, the navigation on the IST/OCIO Staff website was very much influenced by the fantastic ideas participants generated, and the patterns seen among groups participating in the card sorting exercise.
Another technique used during the creation of the IST/OCIO Staff website was the Card-Based Classification Evaluation. This is a twist on a "traditional" usability test (see User- centered design for IT services: The usability toolbox, part 2), and, like a card sort, is done with index cards, and is used to ensure the information on a website is organized in the most intuitive way possible.
In this method, the full "site map" of the website is actually placed on index cards. For instance, level one of the IST Staff website would include:
1. Getting started at IST
2. My desk
3. IST buildings & equipment
etc.
Inside "My desk", would be
2.1 Computer
2.2 Telephone
2.3 Network connection
etc.
After all the pages on the site were mapped out on index cards, the evaluators selected 10 important scenarios that the site's target users (which can be represented by personas; see User-centered design for IT services: The usability toolbox, part 1) would perform.
Card sort participants from IST/OCIO who represented target user groups were then asked to "act out" a scenario, which involved finding something on the IST/OCIO Staff website, using the index cards. The evaluator laid out cards for the first level of the site (e.g., Getting started at IST, My desk, IST buildings & equipment, etc.), and as the participants "drilled down" to deeper levels of the site to complete their task, the evaluator laid out cards for subsequent levels of the site (e.g., Computer, Telephone, Network connection, etc.).
Since this was such a quick test to perform, it was possible to simply walk around the office and find people who had five to ten minutes to participate instead of setting up meetings with participants in advance. This helped tremendously with the overhead normally required to recruit and schedule usability testing participants. Additionally, if a participant ran out of time, the evaluator simply started the test with the next participant at the next question in the list in order to ensure each scenario had a roughly equal number of participants. That way, participants could spend as little or as much time with the evaluator as they chose.
This exercise was extremely helpful in determining which sections of the site were not organized in an intuitive way and allowed the IST/OCIO Staff website team to pinpoint and address these issues before the process of creating the site had even begun.
For more details about this project, contact Allison Bloodworth, or Kalle Nemvalts,
The Fluid project is a worldwide collaborative project to help improve the usability and accessibility of community open-source projects (including uPortal, Moodle, Kuali Student Services System, and Sakai, which is the basis for UC Berkeley's bSpace learning management system), with a focus on academic software for universities. To achieve these goals, the Fluid project is creating a collection of sharable user-interface components (created with HTML, CSS, and JavaScript), which will be presented along with a collection of design patterns. These design patterns will help designers and developers determine how best to use Fluid's components in the particular context in which they are working in their application. In addition, Fluid is creating the User Experience Toolkit which will include advice, guidelines, and templates for use with several different user-centered design methods. Information on personas, contextual inquiry, user testing, design patterns, and more can also be found in the User Experience section of the Fluid wiki.
Fluid has developed a new process called User Experience (UX) Walkthroughs, which combines techniques used in cognitive walkthroughs and heuristic evaluations (see User-centered design for IT services: The usability toolbox, part 2). Cognitive walkthroughs and heuristic evaluations are usability inspection methods designed to assess the usability of a user interface. In a cognitive walkthrough, usability issues are identified by "walking through" the typical tasks users would perform in a website or application. On the other hand, designers or developers performing a heuristic evaluation find usability problems by rating a user interface on how well it complies with a recognized set of usability principles or guidelines such as Nielsen's Ten Usability Heuristics. Additionally, Fluid UX walkthroughs include a modified cognitive walkthrough which is used to assess accessibility of the user interface, along with a software code review which can assist the reviewers in determining whether the application follows code-level guidelines for usability and accessibility.
The Fluid community used the UX walkthroughs to understand the usability issues or "pain points" in the community open-source projects it is working to improve. The goal was to have this work inform the development of Fluid components that would help alleviate these pain points. Moodle, uPortal, and Sakai were all evaluated using these methods by a large and diverse group of evaluators. Kuali Student is still in the planning stages and does not yet exist as an application, so it was not included in this process. Using the protocols, checklists, and templates for the UX walkthroughs published on the Fluid wiki, a geographically dispersed group with different levels of understanding of the process, from Fluid designers to volunteers all over the world, was able to work together on the walkthroughs. The UX walkthrough report template includes space to record usability and accessibility positives, usability and accessibility issues found, as well as the steps used in the cognitive walkthrough and the issues found at each step. Often when a walkthrough was completed, evaluators would include screenshots to further illustrate usability and accessibility problems they encountered. All the UX walkthrough results were recorded on the Fluid wiki to facilitate the sharing of this information in the distributed Fluid community. UX walkthrough working groups also met via teleconference on a weekly basis to discuss and refine this new process, as well as to talk about initial results.
After a certain number of UX walkthroughs were completed, most of the evaluators met in person to inventory, group, and analyze the pain points identified. Each evaluator came to the meeting with each pain point they had found written on its own Post-it note. The evaluators presented all their pain points to the group, and everyone helped put them in high-level categories, called problem spaces, which emerged during the grouping process. Problem spaces covered diverse categories, from feedback to file management, from terminology to workflow. Then smaller groups for each problem space worked to group together pain points which were essentially the same or highly similar. Finally, the small groups identified how many of the Fluid applications were affected by each of these pain points, and also gave each pain point a severity rating. After this in-person work was completed, small working groups discussed (via teleconference) pain points and potential solutions and collected them in the Component Ideas & Definitions wiki page. This information has been used to inform the Fluid Release Plan, and will continue to help the team select future components to design and develop.
For more details about this project or to get involved, see the Fluid website, the Fluid wiki, or contact Allison Bloodworth,
The Campus Collaborative Tools Strategy Development project is working to develop a strategic roadmap that will help collaborative tools providers in IST and across campus to guide investments to enhance collaborative work in all aspects of campus activities through the use of collaborative technologies.
Though the product being produced by this project is a report, user-centered design techniques still contribute valuable information. One of the key goals of this work is to make sure that the strategy reflects the breadth of activities (research, teaching, administrative, public engagement), populations (faculty, researchers, students, staff), and needs across campus. In order to accomplish this, the team is using a number of different methods to gather input from real users of these tools.
The team first broke down activities on campus into a number of different contexts of collaborative activity, including:
Next, the team thought about how to best gather the data from each context, given the characteristics of the participants and the availability of resources to complete the data gathering. The final data gathering methods and associated target user research participants included:
With each group of participants, the team starts by investigating the goals, tasks, practices, conventions, and communities involved in the collaborative work. The participants are asked to think about these aspects of their work independent of the tools or methods they use. By first asking users to focus on their goals rather than the specific tools they use to accomplish those tasks, it is easier to see areas where goals are similar across different contexts, or could be better supported through newer tools or methods. For example, a principal investigator's real goal might be to let their funding agency know the status of their grant. This fact may get lost, however, if the conversation is started about using BAIRS to look at account balances, importing those numbers into Excel, etc.
After this part of the discussion is complete, the team asks more about the tools the participants use and what they're considering using. This is to establish both what currently works particularly well, as well as areas where there may be room for enhancements.
After the user research is complete in mid-March, the team will assemble this user data along with data from other sources and look for commonalities across contexts, in order to suggest common themes for ways that the campus IT providers could help enhance these important collaborative activities.
For more details about this project, see the iNews article Campus Collaborative Tools Strategy Development project, or contact Ian Crew,
[ iNews | Search | IST | UC Berkeley Computing | UC Berkeley ]
iNews: UC Berkeley information technology news channels
Copyright 2008, The Regents of the University of California