My First Usability Study

Authors, Jenny Vogel, Undergraduate Perspectives No Comments

Jenny Vogel | November 9, 2012

Being an information technology student at RETRO has its pros and cons. I feel like I can put my technology background to use while solving problems in the lab, however, I am often left out of fully understanding projects that are heavy in specialized psychology terminology. That’s why I was excited to enroll in my Human Computer Interaction class this semester! Although it is just a basic usability class for I.T. majors, it has given me a good insight into what is really going on in our studies, and I got the opportunity to have hands-on experience in developing my own usability test.  For our class project my group chose to analyze the user interface of Google Hangouts and develop a usability test for it. If you have never seen Google Hangouts, it looks like this:

We were only able to run about 11 participants for our study because of time constraints, but the most important part was getting a feel for the process of designing the study and what to test for. We started out by writing a paper which was largely an opinion piece, substantiated by literature, on how we felt about the design of the interface. Then, as a group, we formulated some tasks that we felt would cover most of the basic functionality that an average user might come into contact with while using Google Hangouts. Here is the task flow that we came up with:


















Then, by combining information from a demographics survey and a post-task opinion poll survey, we developed some conclusions on how well the interface matched up to Nielson’s ten usability heuristics. They are:

1.) Visibility of system status
2.) Match between system and the real world
3.) User control and freedom
4.) Consistency and standards
5.) Error prevention
6.) Recognition rather than recall
7.) Flexibility and efficiency of use
8.) Aesthetic and minimalist design
9.) Help users recognize, diagnose, and recover from errors.
10.) Help and documentation

Since we were attempting to evaluate the entire interface with no specific goals, we tailored all of our post-task questions to directly test these guidelines. For example, for #3, we asked participants to rate on a Likert scale how much they agreed with the statements “I felt like I had freedom to do what I wanted within the application” and “I felt in control of my experience while using the application.”

Analysis of the data in the surveys showed that all participants, regardless of age, experience with technology, or other background factors, found the interface of Google Hangouts overall easy to use, clean, appealing, and indicated that they would use the application again in the future. One important finding from the study was that it is very easy to learn how to use the application. Even though none of the participants had ever used Google Hangouts before, they completed most of the tasks error free and with almost no hesitation. One of the participants commented “Google products are so easy to learn how to use because they are so consistent across all of their services!” Another extremely important trend that the data suggests is that the users’ opinion of the application was overwhelmingly positive. The graph below shows the responses to a multiple-answer multiple choice question that we asked at the end of our survey. The instructions were “Please choose which of the following words you feel describe the interface. (Choose any that apply.)”

Comparing two different demographics, you can see from the two graphs below that participants with a higher self efficacy with computers found the application less challenging to use and more helpful than users who rated themselves with low self efficacy. The graphs show the percentages of words chosen by participants to describe the application.

The graph on the left shows only the users with low self efficacy and the graph on the right shows users with high self efficacy. None of the users with high self efficacy described Google Hangouts as challenging whereas none of the users with low self efficacy described Hangouts as supporting creativity or cognitively stimulating. This seems to suggest that users who are more familiar with computers, the internet, and Google services may have been more able to enjoy the creative and stimulating aspects of Google Hangouts more because they did not feel like the interface was challenging to use and they were not struggling to understand how the interface worked.

None of the participants chose any of the negative words, demonstrating that not even one person had a negative opinion of the interface! As a measure of error recovery factors and effort required to complete a set of tasks, the following data were obtained:

The responses gathered by our study seem to indicate that a majority of the participants felt that very little effort was required to complete the tasks and that it was easy to recover from errors, if errors were encountered. A study by Watson, Besmer, and Lipford confirmed our results [1], stating that users thought that Hangouts was “easy to use due to the interface design and responsiveness.” We also found that Google Hangouts survived the test of Nielson’s Usability Heuristics pretty well, especially in terms of flexibility, aesthetic, consistency, and match between the system and the real world. Our results indicated that more than anything, users found the software to be fun to use, which I’m sure would be good news to Google!


1. Watson, J., Besmer, A., & Lipford, H.R. (2012). +Your circles: sharing behavior on Google+. Proceedings of the Eighth Symposium on Usable Privacy and Security (no. 12). New York: ACM.

No Responses to “My First Usability Study”

Leave a Reply

− five = 1