Usability Testing Tips: How to Retain a User in 15 Seconds?

Yep, that is how much time 55% of users spend on a web page. This means that in these 15 seconds, you have to interest them so that they stay on your website. This is a quick summary of why usability testing is needed.


As for the formal definition, usability testing involves evaluating the UI for intuitiveness and simplicity for the end user (usually – the beginner). It takes place before the product is completed, therefore, as a rule, needs some kind of digital prototype of the interface. Below we will tell you more about usability testing and share our practical experience.

Usability Testing is Not Just "Playing Dumb": 5 Types of Usability Tests

Usability testing is not about "clicking on the buttons on a page and typing a long string with Unicode characters into the TextBox elements". This is a much more complex and comprehensive procedure. So, let’s define five main types of usability tests.


A/B testing ideas
A/B testing ideas. Designed by Avantika

#1 – Problem discovery tests

The essence of such type of usability testing lies in finding problems with the UI use. As a rule, several QA specialists (further – samurai) run the same test scenarios (test cases) at once in order to get objective conclusions, where the majority of users stumble. This procedure is performed in iterations throughout the product development cycle.

#2 – Tests for benchmark study

As versions are updated, the UI is often updated as well. In this regard, samurai conduct tests for benchmark studies, the purpose of which is to maintain the former level of usability or even improve it. If problems are found, samurai return the tested prototype for revision.


A/B Testing in Email Marketing
A/B Testing in Email Marketing. Designed by Avantika


#3 – Tests for competing products analysis

This type of usability testing draws parallels between the user experience of the tested solution and the competitive analogs. In particular, QA samurai synchronously execute a number of identical test scenarios to the customer's product and to its counterparts from competitors. The results obtained are compared with each other to determine the gaps in the customer's product.

#4 – Tests for eye-tracking study

This is a very interesting testing technique that uses heatmaps, common to most users. Thus, samurai reproduce the behavior of real users of a particular software solution and identify the highest priority flaws.

#5 – Tests for learnability study

The main part of the UX is formed based on the assessments of beginners - users who are not familiar with a specific interface and work with it for the first time. This means that test scripts are created and executed taking into account these nuances. As a rule, with this type of testing, the test tasks are performed several times so that the samurai can understand how quickly the stage of mastering the product begins.

Secret Usability Testing Guide From 80 Lvl QA Samurai

Now, it’s time to find out what are the main stages of usability testing.


Designed by Avantika


1. Define the goal

So what is your goal? Understand how user-friendly the UI developed by a super-creative designer is? Or find out if your product can be used by people with disabilities? Perhaps your goal is to increase conversions and eliminate problems that were keeping visitors from reaching the final stage of the sales funnel? In general, the more questions you ask about your expectations about the target audience’s behavior, the more clearly you will be able to formulate the expected testing results.


In addition, everyone knows that it is impossible to fix all bugs (i.e., non-compliance with product specifications). This means that you need to understand which test scenarios you have to execute first of all in the time period allotted to you. 

2. Pick the version of the product to test

Since usability testing begins at the stages when the product is not yet ready, it is important for you to have some kind of prototype where all critical UI elements will be displayed - for example, a website with limited functionality or a demo version of the software. If some features are not yet implemented by the developers, you should come back to testing them later.

3. Find the users from TA

You are unlikely to have problems testing corporate products - to study the UI, it is enough to invite several employees (with different user roles, if necessary). But with general-purpose products the situation is more complicated because you don’t know who exactly will use your product after release. For this reason, it is not worth hiring those who have been developing the product under test and are already aware of some of its features.


But what about the size of the team? Jakob Nielsen and Tom Landauer calculated the average number of usability bugs found in the series with n users:


N * (1- (1- L) ^ n)


For example, there’s a curve for the average L value = 31%:



where N is the total number of usability bugs and L is the number of usability bugs  checked by one user (in percent). Furthermore, they pointed out that groups of more than five people give almost the same results as groups of five people. So, hiring more than five samurai is pointless.

4. Conduct tests

And now the fun begins. Of course, you can start testing everything at random, manually, but you will definitely miss something. Therefore, you need test scripts. To compose them and not forget anything, we recommend using customer journey maps like СJM by FlowMapp – that will complete your path of QA samurai and visualize the customer's experience. Here you can create several customer portraits to use ready-made templates for their typical behavior. To view heatmaps you can use Crazyegg tool.


If you are testing on live users, try to abstract from their personal opinion (for example, about design aesthetics) in favor of assessing the usability and clarity of the UI.

5. Document the test cases

Now you have to document all the test results obtained. Usually, standard bug tracking systems. As for the best ones, they are:



They will help developers follow your test scripts to understand where they went wrong.

6. Analyze, analyze, analyze

As practice shows, usability testing does not end after the release of the product. After all, it can also happen that website visitors do not reach the checkout procedure precisely because of UI flaws. And this is just one of the possible problems. To understand how smooth the UX is on an already working solution, use the following software:



These tools will help you analyze existing gaps to optimize usability over time. Lest you neglect this step, recall that in the first year of Amazon's launch, Jeff Bezos allocated a hundred times more budget for usability testing than for marketing.

Mistakes That no QA Samurai Can Make in Usability Testing

Now let's talk about mistakes that no QA samurai can allow.


Designed by Kenzi Quigg


"The samurai has no goal, only a path"

As we said above, just clicking on buttons is not about usability testing. Based on the specifications and portrait of the user, you have to determine his or her real needs and, according to them, build hypotheses regarding his or her behavior. Each test scenario is a solution to one user problem (no more!).

"Do something!"

If testing is carried out with a group of users with a basic level of gadget proficiency, you may confuse them with your vague problem statement and not get any useful results. Therefore – always be precise, for example: "register on the site", "add a product to the cart", "remove an item from the cart", etc.

"It doesn't matter who, it matters how"

No, it does! Choosing the right users for testing is extremely important! That is why, in the absence of living people, it is sometimes better to use digital tools to simulate their behavior than to test it yourself or entrust this task to non-target users (for example, to developers).

"Don't worry, I'll tell you everything"

You should understand that in real conditions of using the product, the user will not have a helpful mentor behind his or her back. This means that any hints about what to do are superfluous here.

"Silence in the room!"

Do not forget to ask the users to comment on their actions and feelings while completing the assigned tasks. Otherwise, you probably won't understand exactly where they stumble.


Designed by Ivan Mesaroš


"I am smart and you are dumb"

Forget terminology. The group of participants selected for testing is unlikely to be familiar with all design terms, so it is better to speak in a language they understand.

"Why do we need repetitions? After all, everything is fine!"

It is a mistake to think that one stage of usability testing will be enough for you. As a rule, objective results are observed after at least the second stage (it all depends on the complexity of the UI).

"I thought so"

Often the testing procedure is identified with the confirmation of the team lead's hypotheses, and this is a gross mistake. To understand in which direction to go, you must put yourself in the shoes of the average user of your product.

Pinnacle of QA Samurai Skill: Best Ever Usability Test Session Example

Here's a great example of how to conduct test sessions.


  1. The head of testing explains to the participants their tasks.
  2. The participants indicate information about themselves and answer questions that are important for determining their user portraits.
  3. The head of testing asks about possible additional questions and tells how to start performing the tasks.
  4. The participants begin to execute test scenarios, commenting out loud their manipulations (observers record all comments).
  5. The head of testing records the problems the participants face and the successful completion of their assigned tasks.
  6. Tasks are executed until the session time expires.
  7. At the end of the session, the participants fill out questionnaires about their personal experiences and leave the venue.


To document all the steps made by users, you can use this tool in Notion. This will help your samurai team to fix and store detected problems in the same place.


As you can see, usability testing is an integral part of product quality assurance. It usually takes 1-3 months, depending on the complexity of the interface. As for the team, it doesn't make sense to hire more than five people. And of course, make sure that your samurai follow scenarios specific to your product’s target audience so that the usability testing process is as effective as possible.

People also Like