You just spent $5,000 (or more) on a designer doing an overhaul of your website. And you also got a fancy enterprise web analytics package for more than $10,000 per month. Plus you have an army of web analysts in-house who churn out reports night and day to take your online business to the next level. But you know at the back of your mind that there is still something missing that numbers or expert opinion cannot capture.
What is that nagging thought? Yes, that’s right: it is the need for feedback from *real* people. “Usability testing”, as the industry likes to call it. It is a process wherein you observe people browsing your website and get feedback from them on how was their experience. Unlike numbers, real people have feelings. They get frustrated when they can’t find the Buy button. They get annoyed when the contact us link is hidden. They get confused when you present them with plethora of options. The real value of usability testing is unlocked when you start observing patterns in the responses. For example, if 90% of the users found it difficult to find out if your software is free to download then you should better fix that issue. Such insights are not rare when you do usability testing and are potentially worth thousands of dollars.
But usability testing is expensive. Proper usability testing has been limited to big companies who spend hundreds or thousands of dollars on it. And they do this activity because it is worth it. However for a small website owner, spending such a large amount of money isn’t possible.
Enter Feedback Army. Powered by Mechanical Turk, it lets you get feedback from 10 real people for just $10. What you simply do is ask people to visit your website and provide answer to questions related to their visit. I wanted to give it a shot for Visual Website Optimizer, and here is what I asked them to answer (based on their visit):
- What does this site do?
- What aspect of the site confused you?
- What would improve this site?
- Would you signup on the site for using the tool? Why (or why not)?
The responses were very interesting. For example, I always knew back of my mind the text-heaviness of the homepage but the answers from real users confirmed it. Take a look on what they had to say on the homepage:
It’s a bit confusing that in all areas the fonts are around the same size, not emphasizing on a certain more important zone. Simply put, you don’t know where to begin reading it.
When I read the text on the website, it was not really clear to me, still a little abstract. I would like to use more concrete terms. Less text on the page, there is a lot of information on it.
The technical terms are very confusing.
I would suggest reorganizing the information available on the landing page. With all that information at once it’s difficult to glean a simple understanding.
This was obviously a big insight. I now know the homepage needs a redesign to make it less confusing and less technical. (But designing homepage is always a challenge – however the issue here is to identify opportunities for improvement.)
On the flip side, what works on the homepage was also evident in the responses:
I watched the youtube video and that made it more clear.
The demo video in the features page is working well.
Not just only feedback, few responses actually gave direct suggestions for improvement. Take a look here:
I would like to use more concrete terms. Like one slogan: Alter your website and see the impact.
Even after reading the FAQs it is confusing to understand certain terms. Probably explaining them with the help of an example (something related to daily life or for a layman) would certainly improve the message of the website.
You have an excellent retention scheme a + for a marketing tool site. Add the business customers that want to share their success stories on the site, add information on the training aspect, (all BA’s are good in one thing only add a second tool and when it goes wrong it is handy to fall back on). Add some links to some business only social networking sites to improve the target audience and add something related to recent exhibitions attended or planning to attend to increase the customer base.
All in all, here are three million dollar insights (which in hindsight may look obvious)
- Homepage needs a revamp. Simpler design. Less technical terms. (Designer, anyone?)
- Video clearly works, so needs to be emphasized.
- Lay people still don’t really understand A/B testing that well. Need to really drive that point in most simplified terms. (Anyone with ideas?)
Though it was a great experience doing usability testing, there were minor hiccups along the way. If you use a service like Feedback Army, my suggestions would be to:
- Make the questions as specific as possible. (People may interpret generic questions in widely different ways)
- Ask people to give comprehensive feedback. (In absence of an explicit instruction, people may think you are okay with one line feedback.)
- Avoid responses from countries where Mechanical Turk is seen as an income source. (While I won’t name any countries here, but you can do a quick google search to know places where Mechanical turk has become a business. You will find responses from those areas to be least useful.)
I hope I have convinced you to do at least one usability test. It doesn’t cost much and is definitely worth the money. So, why don’t you try doing a usability test today?
(By the way, while writing this article, it also struck me that such usability testing is an excellent precursor to A/B testing for providing testing ideas. So, integrating Mechanical Turk within Visual Website Optimizer (an A/B split testing tool) to get feedback ahead of a split test can be a great idea. What do you think?)