Navigation Test

YouX Academy

Methodologies for Measuring Usability

Luís Filipe

Luís Filipe

Managing Partner YouX

Miguel Félix

Miguel Félix

iBloom Managing Partner

Measure Usability

In the previous article we talked about User-Centered Design and how the knowledge of the user and the context in which he interacts with a system is essential to develop successful interfaces. We are now going to talk about a set of methodologies that allow us to optimize the answers we give to users' needs in certain contexts of use. As mentioned (Rubin, J., Chisnell, D., 2008) “ UCD [User Centered Design] comprises a variety of techniques, methods, and practices, each applied at different points in the product development lifecycle. Reviewing the major methods will help to provide some context for usability testing, which itself is one of these techniques. Please note that the order in which the techniques are described is more or less the order in which they would be during a product's development lifecycle .”

These techniques are:

  • Ethnographic research: observation in the place where the product will be used.
  • Participatory Design: involvement of users and stakeholders in product design.
  • Focus group design: can be used in the preliminary phases of the project to evaluate preliminary concepts with potential users.
  • Surveys: allow you to gauge the preferences of a broad base of potential users.
  • Demonstration (exploratory): once we have an idea of ​​who our target is and their goals and tasks, demonstrations can be used to explore how the user relates to and uses the product through a first concept.
  • Card Sorting (open and closed): is very useful for designing the “findability” of content or functionality.
  • Paper Prototyping: Users are shown some aspect of the product on paper, then asked questions about it or asking about other avenues.
  • Heuristic or expert assessments: Expert assessments involve an evaluation of the product or system, typically by a  usability expert or human factors expert with little involvement with the project.
  • Tthis usability: (the focus of this book) the application of techniques to collect data while observing potential users using the product while performing tasks in a realistic way. This type of testing can be divided into two main approaches:
    • the first involves formal tests conducted as actual experiments with the aim of confirming or disproving certain hypotheses;
    • the second, less formal but still rigorous, employs an iterative cycle of tests aimed at exposing usability deficiencies and gradually shaping the product under study.
  • Follow-up studies: take place after the formal presentation of the product. The objective is to capture data for the next versions of the product.

For a team that has implemented User-Centered Design, measuring usability is important from the outset so that the entire multi-disciplinary team involved in the project understands the objectives and goals to be achieved. The ISO/IEC 9126-6 standard (however superseded by ISO/IEC 25010:2011 ) defines the requirements of effectiveness, efficiency and satisfaction that an interface must have in order for the user to achieve its objectives.

Measure Effectiveness

Objective: Assess whether the user was able to successfully complete a task.

  • Error-free task completion: amount of users who can complete the task without making any mistakes (the golden path , the flow or interaction we would like the user to do).
  • Task completion with non-critical error: percentage of users who complete the task with errors that have no impact on task success but represent less efficiency.
    (the user asks for help to proceed).
  • Critical errors: number of users who are unable to complete a task.
    Cases in which the user can only perform the task with help can also be considered.
  • Number of wrong clicks or wrong taps: considers the totality of the wrong interaction that the user makes in an attempt to perform the task. The heatmap is a good way to capture users' cognitive effort.
    MAZE suggested software.
  • Frequency of request for help: when the user asks the moderator for help to correct errors or help him to complete the task.

We could thus measure effectiveness through the following general formula:

measure usability

That we could scale it into error-free success rate, error-free success rate, and critical-error-free success rate.

Example:
Total successfully completed tasks = 7 Total number of tasks = 10
Effectiveness = 7/10 x 100% = 70%

Measure Efficiency

Purpose: to assess how much time the user needs to complete the task.

Task execution time: amount of time the user needs to complete a task. In operational terms, the values ​​of the various users are added up and an average is calculated. Example:

Task execution time 13"+10"+7"+10" = 40 sec

Average = 10 sec.

Task completion time compared to a heavy user : this is a lesser-known metric that aims at how intuitive the flow is for people with less technological knowledge.

Satisfaction when Performing a Task

After completing a task, even if it was not completed successfully, users must complete a questionnaire that measures the degree of difficulty of the task. The most popular:

• ASQ: After Scenario Questionnaire.

• NASA-TLX: NASA's Task Load Index, which is a measure of mental effort.

• SMEQ: Subjective Mental Effort Questionnaire.

• UME: usability magnitude estimate.

• SEQ: single ease question : this is a good alternative because it is simple. Something like “Overall, how difficult or easy was this task to perform?

“Very difficult – 1 to 7 – Very easy.”

Overall Test Satisfaction

It serves to understand the users' perception of what was tested. At the end of the test, a questionnaire is advanced to find out what is the degree of user satisfaction with the test as a whole. Most used questionnaires:

  • SUS: System usability scale. is one of the most used and provides very accurate results. From a 10-question questionnaire, the user has 5 response options (from totally disagree – from 1 to 5 – totally agree).
  • SUPR-Q: standardized user experience percentage rating questionnaire.
  • CSUQ: computer system usability questionnaire.
  • QUIS: questionnaire for user interaction satisfaction.
  • SUMI: software usability measurement inventory.

References:

 

Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: how to plan, design, and conduct effective tests (2nd ed.). Indianapolis: Wiley.

You cannot copy content from this page