User testing is a qualitative research method used for gathering critically valuable feedback, enabling product teams to roll out enhancements that improve user experience and optimise the success of a digital product.
Introduction
Typically user testing is conducted directly with a small focus group, representative of the product’s target audience, to identify issues or friction points relating to:
- Usability
- Functionality
- Design
- Content and messaging
- User experience
- Feedback about a product and/or proposition
It’s common to ask participants to complete tasks, such as navigating to specific pages, often following a scripted journey or scenario and observing/recording their interactions with the website. Sessions can be conducted in a variety of ways. For instance; in-person or remote (ie. via a third party testing platform, I favour using Userlytics or UserTesting); moderated or unmoderated. Users are asked to verbalise their thoughts and experiences, and rate the difficulty of specific tasks.
Insights gathered through user testing can be interpreted alongside wider user research and UX principles to produce a strategic roadmap of website improvements, that are substantiated by data as opposed to being theoretical. User testing can also complimented by quantitative data, such as;
- GA4 path explorations
- Bounce rates and exits
- Dwell time
- Analysis of heatmaps, scroll/click data and session recordings from tools such as Microsoft Clarity
Collating this quantitative and qualitative data enables digital teams to optimise and cater a product precisely to the needs and behaviours of it’s target audience, often with the wider commercial goal of increasing conversions.
“The benefits of user testing are plentiful, from uncovering customers’ needs and challenges to designing better solutions, making faster decisions, and encouraging a customer-first approach, to name a few.”
Recruiting participants for user testing
Third party user testing service platforms enable teams to select participants by specified characteristics, whether the product be B2C or B2B, in order to ensure that they align closely with the intended target audience or buyer/user personas.
These characteristics may include, but are not limited to;
- Age
- Race/ethnicity
- Spoken language
- Gender
- Marital status
- Parental status
- Employment status
- Job role, seniority and industry
- Income
- Skills
- Technology (browser vendor and version, or operating system)
- Device
- Location
- + other bespoke screening questions
There’s a common misconception that user testing is costly and/or time consuming. In reality, the Nielson Norman Group framework for optimal qualitative results from usability studies advises that testing need not be conducted with more than 5 users per device type.
Caption: 5 users, the optimal sample size for qualitative usability studies.
“If you think good design is expensive, you should look at the cost of bad design.”
Rating the severity of issues
Nielson Norman Group also provides a guide for rating the severity of issues raised by user testing, based on the following 3 factors;
- The frequency with which the problem occurs: Is it common or rare?
- The impact of the problem if it occurs: Will it be easy or difficult for users to overcome?
- The persistence of the problem: Is it a one-time problem that users can overcome once they know about it, or will users repeatedly be bothered by the same problem?
| Severity | Issue |
|---|---|
| Low [!] | Fixing this issue should be given low priority |
| Medium [!!] | Important to address, so should be given high priority |
| High [!!!] | Imperative to address urgently |
Analysing feedback
Conventionally, user comments are analysed and recorded in a detailed matrix (often a spreadsheet) in order to identify similarities and common experiences. Feedback can then be consolidated and presented as a roadmap of recommended improvements.
Caption: An example of a user testing matrix analysis.
“People ignore design that ignores people.”
The business case
Coming soon.
https://twitter.com/ormanclark/status/1673682682243317762?s=20
“Every dollar invested in UX brings, on average, 100 in return. This amounts to an ROI of over 9,900%.”
User-centred design: putting your users first
What is user-centred design and why is it important to build human focussed empathy driven solutions into digital products?