Evaluation of SI Academics website

Aim

To evaluate the functionality and usability of the academic section of School of Information’s website using a variety of methods.

Summary

The SI622 course is a key component of SI’s HCI curriculum and we evaluated the School of Information’s website (academic section) using various methods as part of the course:

Interaction Map
Interviews (w Personas & Scenarios)
Comparative Analysis
Heuristic Evaluation
Survey
Usability Testing

As part of the course, professional reports were generated for each individual method. Many of our recommendations and suggestions were implemented in the new redesign of the SI website.

My Role (What did I do, learn)

It was a group project with 4 members. I supported my team throughout the course performing different roles based on different evaluation methods. In addition to learning how to use the methods, I also learnt which methods give what kind of answers based on the kind of effort.

Design and Methodology

Interaction Maps
Interaction Maps helped us figure out the interaction patterns of the SI website. It also allowed us to identify inconsistencies and dead ends in the design. Moreover, it also helped us to determine which sections to evaluate during the course of the study. Based on the interaction map, scope of the SI website and interviews with stakeholders, we limited our study to the academic section of the SI website.

Interviews (with Personas and Scenarios)
Current and prospective students were interviewed in the process of a systematic evaluation of academic section of SI website. During the interviews, the users were asked about their experiences with SI website, what they thought it did well, and what aspects of the website could benefit from improvement. From the information collected from the users about their experience with the system, analytical tools called “personas” and “scenarios” were constructed to build a better model of the website’s user population. The key findings from the interviews are mentioned below:

Finding 01: Information overload is a major challenge faced by new users
Finding 02: There is a need of easy access to ‘Prospective Students’ and ‘Quick Links’ sections
Finding 03: Users find it difficult to form mental models of the website
Finding 04: Students expect the SI website to represent what it teaches

Comparative Analysis
Comparative evaluation of SI website was conducted against its competitors and systems with similar models. The important advantages or flaws in the designs were noted. This resulted in six key findings:
Finding 01: Information and images could be laid out more effectively on the SI website.
Finding 02: Course Registration process on the SI website may be more unified and less time consuming.
Finding 03: Navigation on the SI website is not user centric, inconsistent and could be improved.
Finding 04: The Search and Help of the SI website needs to be more user friendly
Finding 05: Showcasing the achievements and events at SI upfront could gain help it gain an extra edge over competing schools
Finding 06: Integrated collaboration tools are missing from most academic websites including SI

Heuristic Evaluation
Our team conducted the heuristic evaluation of UM SI website academics section based on Neilsen’s theory and design principles. The findings and recommendations gathered based on the evaluation are mentioned below:
We identified the most critical findings and came up with recommendations:

Finding 01: Persisting navigation issues in the whole website
Recommendation 01: Several pages on the SI website needs redesign to maintain consistency and simple navigation

Finding 02: Inconsistent layout results in more cognitive load for the user
Recommendation 02: A standard layout for pages on the same level should be used consistently throughout the site

Finding 03: Issues with labels and styles negatively affect the learn-ability of the user
Recommendation 03: Appropriate labels should be used across all the pages

Finding 04: Search functionality
Recommendation 04: Several different mechanisms for addressing the problem of helping users recover from incorrect searches

Survey

Our team conducted a survey in Qualtrics to learn more about how useful the SI academics website is for what current students are trying to accomplish, how well the website supports the success of their efforts, and how satisfied the students are with SI academics site. To create these research questions, we considered the most actionable items, the website goals to attract potential students and give the most success to the current students, and the findings from our earlier usability studies. The SI academics website administrator invited SI students to participate in this survey by sending an e-mail the school-wide listserver. Of the approximately 570 students on this listserv, 48 completed the survey. The team examined the distribution of the answers the various questions and looked into possible correlations between questions. From this, we generated the following findings and recommendations.

Finding 01: The course section is very important and used a lot, but many respondents find it difficult to use.
Recommendations 01: Gather the course information into one area, provide a flexible search function for courses, and integrate the calendar and table views to be part of the course list function.

Finding 02: Students are having problems finding information.
Recommendations 02: Consider using task based organization and progressive disclosure to make information easier to locate.

Finding 03: Mixed content for prospective and current students increases the cognitive load.
Recommendations 03: Create separate sections for prospective and current students to present each with the information that is important to them.

Finding 04: Navigational difficulties causes dissatisfaction with the SI website
Recommendations 04: Rename or reorganize the academics section into several different channels.

 

 

 

 

 

 

 

 

Usability Testing

To recruit participants for the usability test, team members recruited users from their social networks who had never used the SI website prior to participating in the usability test. Our team developed a list of tasks which we believed were representative of tasks that a typical student user might perform on the site. In our usability test, we asked participants to carry out these tasks. To gain more information about each user, we developed pre- and post-test questionnaires to gather background information on participants and to better understand their overall experience and satisfaction with performing the tasks on the site.

As a result of our study, we identified five issues and offer recommendations which should help to improve the website. They are listed here, in order of importance, starting with the most important:

Finding 01: Users found the search function unpredictable.
Finding 02: In the case of the search bar in the course catalog, users ignored written instructions which told them what to enter.
Finding 03: Users struggled with finding TAPS sheets.
Finding 04: Users found it inconvenient to remember information from one page to another.
Finding 05: The multiple forms of navigation provided by the site served the users well but minor improvements to the style could be useful.

To address these problems, we recommended the following:

Recommendations 01: Enable users to use the search bar in the course catalog to search for course number rather than by course name or course description.
Recommendations 02: Provide pre-written text in the course catalog search bars to instruct the user what to enter in.
Recommendations 03: Help users better understand the search results or the reason behind the lack of search results.
Recommendations 04: Put a readily apparent link to the TAPS sheet inside of the main body of each specialization.
Recommendations 05: Alternatively, keep the link on the right pane and float the link down as the user is scrolling down the specialization page.
Recommendations 06: Enable users to spend less clicks navigating between degree requirements and the course catalog by linking lists of courses which fulfill degree requirements to pages which display course details.
Recommendations 07: Use bullet points, bold text, italics, graphics or different font styles or colors to highlight important details in text-heavy pages.
Recommendations 08: Apply a single color to signify hyperlinked text.

Overview

Collaborators
Mukul Bisht, Bethany Huseman, David Ross, Shuo Yang

Methods Used
User Research

Technology Used
XHTML, CSS, PHP, MySQL, VISIO, Photoshop, Illustrator

Client

UMICH

Services

4 months (Jan 2012 - Apr 2012)

Skills

User Research
Web Design