VWO Logo
Like this post?
Read our in-depth guide to Conversion Rate Optimization
Share this post
8 Min Read

How to Conduct Usability Testing to Get More Accurate and Actionable Insights

“No matter how much you try, you’re not your user. Every test or interview, I’m amazed at the way they use it, hack it, or create their own workaround. You can’t imagine these interactions with your product until you’re actually sitting there watching them” ~ Jenn Downs, Design Researcher, MailChimp.

Above is the statement from a recent interview where Jenn talks about her experience with user testing and how it plays a pivotal role in design strategies at MailChimp.

Since you’re still reading this article, I think it’s fair to assume that like Jenn, you are one of those smart online professionals who do understand the importance of usability testing.

If a user is having a problem, it's our problem - Steve Jobs

If you’re contemplating to conduct a usability test soon, listed below are a few tips that will help you proceed with the testing process to get the most accurate results and the actionable data you want to frame hypotheses for your AB tests.

1. Choose the Right Test Participants

If your product targets tech-savvy Internet marketers, testing your site with ladies in their mid-twenties who know nothing about search engine optimization will not get you the right insights. This is the reason why your test participants must match your user personas.

People who know you personally are also most likely to skew test results as usability test participants. They would know that you have put in a lot of effort into this website. So, even if you ask them to critique it, they will probably say nice things about it. You want your users to complain when they feel irritated or frustrated on your website. That will serve the purpose of user testing for you.

2. Test for Different Personas

You do realize that your target market doesn’t comprise of only one perfect user persona. Your personas can be differentiated based on their income group, age, gender, profession, their level of education, marital status, or any other criteria.

For example, you may notice that an older user might be struggling with drop-down menus on your site because of their poor control over fine-tuned mouse movements. But anyone else might be able to get through with it in a breeze. This can be an important concern if your website is an online drug store and majority of your target consumers belong to the older generation.

Create relevant personas for your target market and conduct different usability tests for each of these personas. This way you will understand the specific needs for each segment. Once you have inputs from every persona, you can use them to make a site that caters to the needs of your entire target market.

Although you might not be able to cater to every small need, it’s okay. Don’t beat yourself up for it. Use your discretion to identify what points you need to prioritize, and what are the suggestions you should skip.

3. Use Your Lead Language Cautiously

When you explain your test participants about the task(s) they have to complete, it’s easy to influence their answers unintentionally. Your lead language can sway them easily if you’re not careful about using the right words.

For example,

“I’ve put in a lot of effort on this website, can you please tell me what do you think are its strength and weaknesses?”Wrong – This means, I’m looking for praise here, you give me any serious suggestions and my face will drop.

“If you get stuck anywhere, it’s absolutely okay. Just speak out loud and do whatever feels natural.”Right – This means, don’t worry about me feeling bad, just express what you’re thinking without any restraints.

Another scenario can be how you frame your task questions:

Let’s say you want to see if people will be able to find your live chat option before your customer service number on the homepage. Here are the two ways you can lead your test participants:

“Contact the live chat agent”Wrong – This is fine when you want to see if people will be able to find your new live chat icon on the homepage.

“If you want to contact us for any query, what would you do?”Right – This is better if you want to track preferred user flow on the site if they want to contact you.

When writing your test tasks, make sure you put down the exact words in which you want to communicate them to your test participants.

Pay attention to the possible ways in which your words can be interpreted. And know exactly what you are testing – positioning of an icon, use of a page element, or anything else. Then frame your questions accordingly. Avoid leading wherever it is not required.

4. Plan Your Test Structure

Let me ask you something – Do you like monkeys?

Okay next question, give me the name of one popular fruit?

Did you just think “Banana”?

I know this is what majority of you thought. Now imagine if I had been conducting a survey to find the “Most popular fruit.” My result has been tampered.

Similarly, when you ask specific questions, you call attention to that element. This can indirectly influence the answers of test participants for the questions that follow.

To avoid this, it is best to start with general questions and then get to the specifics. So instead of asking general questions like, “What do you think about the navigation?”, maybe you should ask, “What are the three things you liked the best about the website?”

Apart from this, it is best to follow the logical test structure. It makes no sense when you ask someone to make an account, complete the purchase, and then find another product and comment on the navigation. Plan your test structure in a way that it follows the natural pattern of tasks as a real time user is expected to follow them.

Additional tip – If you have any complicated task for which the probability of failure seems quite high, try keeping it for the last. You wouldn’t want to leave the impression on your test participant that the website is too complicated or problematic.

5. Quantify Your Results

When you are asking opinion questions that users may answer with a “Yes” or “No,” you can ask them to answer them on a scale of 1-5. For example, instead of asking, “Was the content clear?” you can ask, “What do you think about the content?” It was:

  • Clear and easy-to-understand
  • Quite clear, I couldn’t understand 1-2 sentences though
  • Didn’t seem interesting, so I didn’t read much
  • I couldn’t understand much of what was written
  • I just didn’t feel like reading it

Quantifying your results can give you more actionable insights for your test hypotheses. You may even record the time taken to complete each task and number of errors made by the participants for every task. Of course, the lesser the numbers for these metrics, the better it is.

These numbers can especially come handy if you plan to conduct another user test. It would be best to conduct your second test on a new set of participants after making the improvements from the feedback you received in your first usability test.

User testing process

6. Ask Them About Homepage and Navigation

People often base their opinion of the entire website based on the homepage. You can start your testing with the popular 5-second method. Yes, show them the homepage for just 5 seconds and ask them if they can:

The designer is an idiot, not the user.

  • Tell what the website is about?
  • Which is the first thing they noticed/would want to click on the homepage?
  • What are the three things they remember on your homepage?

If they were not able to answer any of these questions, you sure have a lot to do to improve your homepage. Questions about navigation can also give you a lot of test ideas to work upon. Like, you can ask them if they:

  • Understood the words/terms used for all the categories/products in the product filter/drop-down menu
  • Were able to find the product they were looking for?
  • Were able to interact with the drop-down/product filters easily?
  • Have any suggestion to include any other option in the product filter/navigation?

7. Ask Them About Checkout and Forms

These parts of the website are usually closely related to your conversion goals. It’s important for you to know if there are any friction points here. You can ask your participants if:

  • They think the form/checkout process was confusing at any point?
  • Anything seemed annoying to them during the process?
  • They got stuck at any step/form field?
  • They would like to shop from your website again?

For checkout, you may even ask them questions related to upsell opportunities, or if they felt trapped with no navigation, and other such questions.

Of course, a lot of these questions can be skipped depending on if you are conducting remote usability testing or not. Don’t forget to add your own specific questions though. If you’re interested in remote usability testing, you can sign up for our free 30-day trial and use our usability testing feature in the app.

Once you have your user test results, you should have a lot many actionable data for your test hypothesis. Remember your test participants are only a very small size of your target market. So these results are never statistically significant. Before you make any permanent changes on your website, be sure to A/B test them for your real customers.

Image credits:

  • Fabio Murru via Dribbble
  • Conversion Rate Experts
  • Wireframes & Usability


More from VWO

10 Quick Copy Tweaks for High-Converting Websites

Copy tweaks are the easiest of the lot. There’s usually no hassle of running to…

Read More
Smriti Chawla

Smriti Chawla

10 Min Read

How To Build A Culture Of Experimentation

It’s one thing to run an A/B test correctly and get a meaningful uplift. It’s…

Read More
Alex Birkett

Alex Birkett

12 Min Read

How To Conduct User Research To Supercharge Your Optimization Efforts

What Is User Research? Usability.gov defines user research as “understanding user behaviour, needs, and motivations…

Read More
Emma Travis

Emma Travis

9 Min Read
Shanaz Khan from VWO

Hi, I am Shanaz from the VWO Research Desk.

Join our community of 10,000+ Marketing, Product & UX Folks today & never miss the latest from the world of experience optimization.

A value for this field is required.

Thank you!

Check your inbox for the confirmation mail