What gets measured, gets managed. - Peter Drucker
Unfortunately, this wisdom is known by various product startups & companies but seldom it gets applied into reality with clear vision. If vision is set without clear results and figures you will find it difficult to convince stakeholders about the ROI & inline future UX activities (especially budget allocation).
Customers & stakeholders always keep an eye on the performance & user adoption of the app whether it's B2B or B2C category. More often than not, UXers share this information in excel sheets or in basic graphical representation, but this experience can be elevated further by gaining some basic understanding about the metrics and statistics. There is a misconception that the learning curve to understand these metrics is steep, in-fact it's other way round. This article will help you to break this misconception.
Data is a KING, but not all the time...
In ancient times having power meant having access to data. Today having power means knowing what to ignore, writes Yuval Noah Harari in his book ‘Homo Deus’.
Many product companies now are able to gather large amount of data and with sophistication in analytics tool, interpreting these data through specialist has become much more easier. However, having huge volume of data doesn't mean it will yield relevant output.
More visitors → means success → is a wrong assumption.
But with these data are you really making effective usage of this data and make any real benefits? Without clear indicators & matrices it's really going to be tough decision.
UX aficionados distinguish between behavioural and attitudinal UX indicators:
Behavioural
Task success rate
Time spent on a task
Navigation vs Search
User error rate
Attitudinal
System usability score (SUS)
Net promoter score (NPS)
Customer satisfaction (CS)
1. Task Success Rate (TSR)
Task success rate is one of the most widely used and easily understood UX metrics. It shows the percentage of participants that successfully complete a task.
This helps designers to identify user experience issues. However, you need to be clear about what tasks & goals you consider a success in a particular case before you start collecting data. TSR doesn't communicate why users failed to complete a task.
Task Success Rate (TSR) = Number of tasks completed successfully
Total number of attempts This category is most applicable to areas of your product that are very task-centred, such as search or an upload flow etc.
Example:
On a retail eCommerce site, 10 users are given tasks to purchase 5 different retail products through credit card payment. Out of 10, 8 manage to complete the task successfully whereas 2 fail to complete due to some reason. In this case, the task success rate is calculated as follows: 8/10 = 0.8 x 100 = 80%.
Naturally, difficulty levels to complete a task will vary based on type of user. New user, will take slightly longer time compared to regular user of the product.
2. Time Spent on a Task (TST)
TST (in minutes and in seconds), is the average time a user needs to complete a task successfully.
It’s a useful usability metric as it tells you how certain actions affect efficiency and user productivity, which usually plays a big role in how satisfied customers are.
Example:
On a retail eCommerce site, 5 users are given as task to find a specific product of a brand.
User 1 : 45 secs
User 2 : 25 secs
User 3 : 30 secs
User 4 : 50 secs
User 5 : 15 secs
The TST is calculated as (45+25+30+50+15) = 153 seconds
5Needless to say, the shorter the time, the better the user experience.
3. Navigation vs Search
Navigation bar is considered to be most important element to orchestrate the overall website or app. Inefficient navigation bar will push users towards using Search fields.
In many cases, its advisable to user navigation bars instead of search fields as it delivers better user experience. This can vary on case-to-case to basis though.
N vs S (%) = No. of completed tasks via Navigation or Search
Total no. of completed tasksExample:
On a retail eCommerce site, 9 users are given as task to order a specific brand of T-shirt. Analyse how these test users go about performing this task by checking how many used the navigation and how many approached through search field.
User 1 : Search field approach
User 2 : Navigation approach
User 3 : Navigation approach
User 4 : Search field approach
User 5 : Navigation approach
User 6 : Search field approach
User 7 : Navigation approach
User 8 : Navigation approach
User 9 : Navigation approach
So the calculation of the Navigation vs Search Ratio:
Search 3/9 = 0.33 x 100 = 33%
Navigation 6/9 = 0.66 x 100 = 66%4. User Error Rate (UER)
The user error rate (UER) is the number of times a user makes a wrong entry. Let’s take the example of the usually unsuccessful attempt to enter the user’s date of birth in the address field.
UER data clearly communicates how usable is your website or the app is. The higher the UER score, the higher the number of usability problems which clearly indicates website needs to improve its usability.
Before you perform this activity, you need to be very clear as to what actions are perceived as errors.
Two most popular ways to calculate UER are:
Error Occurrence Rate (EOR):
If you want to track only one potential error (out of many other errors) than you use this matrix.
EOR = Total no. of errors occurred for all the users
Total no. of error opportunities for all the usersExample:
If 5 out of 100 users enter incorrect "Re-enter Password" field than the error occurrence rate is calculated like this: 5/100 = 0.05 x 100 = 5%
Error Rate (ER):
If multiple errors are possible per task (or you want to measure multiple errors), you can do this using the error rate.
ER = No. of errors
Total no. of task attempts Summary
UX Matrices presented in this article are the powerful tools which can used smartly to present your analytical data to stakeholders and superiors. Keep reading PART 2 section of this article where we unfold matrices relevant to Attitudinal approach.
Do you need any help product design or building product strategy?
We have helped numerous budding startups & product companies from ground-up to successfully launch their product through better UI/UX strategy. Feel free to drop us a line.