YouX subscription
Navigation Test

YouX Academy

Methodologies for Measuring Usability

Picture of Luís Filipe

Luís Filipe

Managing Partner YouX

Picture of Miguel Félix

Miguel Félix

Managing Partner iBloom

Measure Usability

In the previous article we talked about User-Centered Design and how knowledge of the user and the context in which they interact with a system is essential to developing successful interfaces. Let's now talk about a set of methodologies that allow us to optimize the responses we give to users' needs in certain contexts of use. As mentioned by (Rubin, J., Chisnell, D., 2008) “ UCD [User Centered Design, or User-Centered Design] comprises a variety of techniques, methods, and practices, each applied at different points in the product development lifecycle. Reviewing the major methods will help to provide some context for usability testing, which itself is one of these techniques. Please note that the order in which the techniques are described is more or less the order in which they would be employed during a product's development lifecycle .”

These techniques are:

  • Ethnographic research: observation in the place where the product will be used.
  • Participatory Design: involvement of users and stakeholders in product design.
  • Focus group design: can be used in the preliminary stages of the project to evaluate preliminary concepts with potential users.
  • Surveys: allow you to gauge the preferences of a broad base of potential users.
  • Demonstration (exploratory): once we have an idea of ​​who our target is and their objectives and tasks, demonstrations can be used to explore how the user relates to and uses the product through a first concept.
  • Card Sorting (open and closed): is very useful for designing the “findability” of content or functionality.
  • Paper Prototyping: users are shown some aspect of the product on paper, then asking questions about it or asking about other ways.
  • Heuristic or expert evaluations: Expert evaluations involve an evaluation of the product or system, typically by a  usability specialist or human factors specialist with little involvement in the project.
  • Tthis usability: (the focus of this book) the application of techniques to collect data while observing potential users using the product while performing tasks in a realistic way. This type of testing can be divided into two main approaches:
    • the first involves formal tests conducted as true experiments with the aim of confirming or disconfirming certain hypotheses;
    • the second, is less formal but still rigorous, employs an iterative cycle of tests that aim to expose usability deficiencies and gradually shape the product under study.
  • Follow-up studies: occur after the formal presentation of the product. The objective is to capture data for subsequent versions of the product.

For a team that has implemented User-Centered Design, measuring usability is important from the outset so that the entire multi-disciplinary team involved in the project understands the objectives and goals to be achieved. The ISO/IEC 9126-6 standard (now replaced by ISO/IEC 25010:2011 ) defines the effectiveness, efficiency and satisfaction requirements that an interface must have so that the user can achieve their objectives.

Measure Effectiveness

Objective: evaluate whether the user was able to successfully complete a task.

  • Error-free task completion: number of users who can complete the task without making any errors (the golden path , the flow or interaction that we would like the user to make).
  • Task completion with non-critical error: percentage of users who complete the task with errors that have no impact on the success of the task but represent lower efficiency.
    (the user asks for help to proceed).
  • Critical errors: number of users who are unable to complete a task.
    You can also consider cases in which the user can only perform the task with help.
  • Number of wrong clicks or wrong touches: considers the totality of the wrong interaction that the user makes in an attempt to carry out the task. The heatmap is a good way to capture users' cognitive effort.
    Suggested MAZE software.
  • Frequency of help request: when the user requests help from the moderator to correct errors or help them complete the task.

We could thus measure effectiveness using the following general formula:

Measure usability

Which we could multiply into success rate without errors, success rate with errors and success rate without critical errors.

Example:
Total tasks successfully completed = 7 Total number of tasks = 10
Effectiveness = 7/10 x 100% = 70%

Measure Efficiency

Objective: evaluate how much time the user needs to complete the task.

Task execution time: amount of time the user needs to complete a task. In operational terms, the values ​​of the various users are added together and an average is calculated. Example:

Task execution time 13”+10”+7”+10” = 40 seconds

Average = 10 sec.

Task completion time compared to that of a heavy user : this is a lesser-known metric that aims to see how intuitive the flow is for people with less technological proficiency.

Satisfaction When Performing a Task

After completing a task, even if it was not completed successfully, users must complete a questionnaire that measures the degree of difficulty of the task. The most popular:

• ASQ: After Scenario Questionnaire.

• NASA-TLX: The NASA Task Load Index, which is a measure of mental effort.

• SMEQ: subjective mental effort questionnaire.

• UME: usability magnitude estimate.

• SEQ: single ease question : this is a good alternative because it is simple. Something like, “Overall, how difficult or easy was this task to accomplish?

“Very difficult – 1 to 7 – Very easy.”

Overall Test Satisfaction

It serves to understand users' perception of what was tested. At the end of the test, a questionnaire is administered to determine the user's level of satisfaction with the test as a whole. Most used questionnaires:

  • SUS: System usability scale. It is one of the most used and provides very accurate results. From a 10-question questionnaire, the user has 5 answer options (from totally disagree – from 1 to 5 – totally agree).
  • SUPR-Q: standardized user experience percentage rating questionnaire.
  • CSUQ: computer system usability questionnaire.
  • QUIS: questionnaire for user interaction satisfaction.
  • SUMI: software usability measurement inventory.

References:

 

Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: how to plan, design, and conduct effective tests (2nd ed.). Indianapolis: Wiley.

You cannot copy content of this page

Skip to content