What is Usability Testing? Interesting Facts and 10 principles to know

What is usability testing? For many who are reading this, these word rings in their ears every day. Many people asked me this question before. And many people still asks it everyday.

In the same way, many believe that Usability is the same as User Experience and use both terms interchangeably (anecdotally, many years ago, when I started in this area, I did the same thing, only that confusing Usability testing with User Experience Design).

What is usability?

Well, actually, Usability is one of the many disciplines or areas that forms what we know as User Experience.

To understand it a little better, let’s look at the definition of usability that Wikipedia gives us:

Usability is the ease of use and learnability of a human-made object such as a tool or device.

In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

and a little below:

Usability includes methods of measuring usability, such as needs analysis and the study of the principles behind an object’s perceived efficiency or elegance.

In human-computer interaction and computer science, usability studies the elegance and clarity with which the interaction with a computer program or a web site (web usability) is designed.

Usability considers user satisfaction and utility as quality components, and aims to improve user experience through iterative design.

In summary, and willing to avoid long definitions, we can say that:

Usability is the measure of effectiveness of using a certain tool, real or virtual object

What about usability testing?

Usability testing is the main part of usability. Without objective user testing, everything becomes subjective.

And subjectivity is what a UX designer should avoid at all costs.

When you design an experience, you start with certain logical steps related to a UX design process.

There are different methodologies, but in general, we know that we will probably find the following:

• brainstorming or an ideation process
• wireframes
• maybe some interactive mockup
• a prototype

And although it is possible (and advisable) to do tests along all the mentioned points, when we reach the prototype state, we must test.

And these tests are performed in different ways, but there is a type of test that cannot be ignored:

Usability testing with real users.

For sure, we can do automated testing, and we’ll get great insights from that.

But nothing replaces user testing in UXD (User Experience Design).

Types of user testing

There are different types of usability testing. We can define them in 3 groups, which could be mixed between them:

• Moderated / Unmoderated
• Remote / Local
• Explorative / Comparative
  

Moderated Testing

This is a kind of testing that is moderated by a UX research person. It could be remote (for example, a phone interview) or local (in a lab, or in user’s environment)

Unmoderated Testing

As you probably imagine, this is the opposite of moderated user testing. Here, we use automated tools to study user behavior and get insights for usability and other UX issues.

A great example of a tool for this purpose is Hotjar .

You can see an example of usability testing using heatmaps we did for one of our clients. Here, we wanted to test what users were more interested in after the redesign we did for them.

Remote Testing

This is a type of testing methodology where the researcher and the user are not in the same location.

It could be automated and unmoderated (as in the example above), or moderated (for example, a phone review, interviews using Skype, etc.)

Local Testing

Contrary to the previous methodology, local testing happens when both user and research person are in the same place. It could be a lab, user’s home, a place with people (very common in guerrilla testing), etc.

Explorative Testing

Explorative user testing methodology is not very common in usability tests, since the method is open ended. However, there are some specific scenarios where you can use this type of methodology. The most common is card sorting, a pillar method in Information Architecture.

Comparative Testing

Contrary to explorative testing, comparative testing doesn’t rely on open questions. Instead, it presents the user 2 or more options to compare and choose from.

Attention! While this might sound as A/B testing, is not the same thing!

Usability Testing Methods

The user testing types mentioned above may include different methods, some of them already mentioned in previous paragraphs.

However, those types may apply to different UX research methodologies, not necessarily usability.

When looking for insights on usability, we may use some of the following methods, tools and techniques:

• System Usability Scale (SUS)
• Lab Testing
• Guerrilla Testing
• Hallmark Testing
• Heatmap Analysis
• Eye Tracking Analysis
• User Observation (remote or local)
• Interviews (remote or local)
• Software Usage Analysis

Interesting facts about usability

For those who want to know how important usability is in our lives, ask Apple. They changed the way we see the world through the usability of their products.

And they did it by following german designer Dieter Rams‘  essential 10 principles for good design:

1. Good Design is Innovative

The possibilities for innovation are not, by any means, exhausted.
Technical development is always offering new opportunities for innovative design. But innovative design always develops in tandem with innovative technology, and can never be an end in itself.     

2. Good Design makes a product useful    

A product is bought to be used. It has to satisfy certain criteria, not only functional, but psychological and aesthetic. Thus, good design emphasizes the usefulness of a product while disregarding anything that could possibly detract from it.     

3. Good Design is aesthetic    

The aesthetic quality of a product is integral to its usefulness because products we use everyday affect our person and our well-being. Because of this, only well executed products can be beautiful.

4. Good Design makes a product understandable    

It clarifies the product’s structure, better still, it can make the product      talk. At best, it is self-explanatory.     

5. Good Design is honest

It does not make a product more innovative, powerful, or valuable than it really is. It is not an attempt to manipulate the consumer with promises that cannot be kept.     

6. Good Design is unobtrusive    

Products fulfilling a purpose are like tools. They are neither decorative objects nor works of art. Their design should therefore be both neutral and restrained in order to leave room for the user’s self expression.    

7. Good Design is long lasting    

It avoids being fashionable and therefore never appears antiquated. Unlike fashionable design, it lasts many years – even in today’s throw-away society.     

8. Good Design is thorough down to the last detail    

Nothing must be arbitrary or left to chance. Care and accuracy in the design process show respect toward the consumer.       

9. Good Design is environmentally friendly   

Design makes an important contribution to the preservation of the environment. It conserves resources and minimizes physical and visual pollution throughout the life cycle of the product.     

10. Good Design is as little design as possible    

Less but better – because it concentrates on the essential aspects and the products are not burdened with inessentials. Back to purity, back to simplicity!     

And even more data!

Other data extracted from the Wikipedia page cited above:

1. Usability shows reductions in the product development cycle of 33-50% (Bosert 1991).
2. 63% of all software development projects exceed their budget, being the four most important causes related to usability. (Lederer and Prassad 1992).
3. The percentage of code that is dedicated to the development of UI has been increasing over the years. Nowadays, it reaches an average 47-60% of the whole application. (MacIntyre et al. 1990).
4. The Ricoh company discovered that 95% of the users surveyed never used the three key features designed to make the product more attractive. Either because they were unaware of its existence, not knowing how to use them or not understanding them. (Nussbaum and Neff 1991).
5. 80% of maintenance tasks are due to user requirements not foreseen. The rest remaining due to failures and errors. (Martin and McClure 1993; Pressman 1992)

In conclusion: get to work and spend more time studying this fascinating discipline. Who knows? Maybe the next Dieter Rams or Steve Jobs is reading these lines!

More Usability Testing Resources and Bibliography

A practical guide to usability testing – J. Dumas, J. Redish (PDF)

Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing – T. Kallio, A. Kaikkonen (PDF)

Usability testing of an academic library web site: a case study – B. Battleson, A. Booth, J. Weintrop (PDF)

Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications – D. Zhang, B. Adipat

Fabio Devin

Fabio is Bachelor in Marketing, Master in Human Computer Interaction and a UX Evangelist.

He worked for some of the biggest companies in the world. As such, he has developed entire systems and businesses for clients of all sizes.