{"id":32473,"date":"2015-09-10T12:22:35","date_gmt":"2015-09-10T12:22:35","guid":{"rendered":"https:\/\/vwo.com\/blog\/?p=32473"},"modified":"2025-04-28T14:47:02","modified_gmt":"2025-04-28T09:17:02","slug":"user-feedback-fuels-ab-testing","status":"publish","type":"post","link":"https:\/\/vwo.com\/blog\/user-feedback-fuels-ab-testing\/","title":{"rendered":"How to Convert User Feedback Into A Goldmine for A\/B Testing"},"content":{"rendered":"\n

We have finally learned to <\/span>mock HiPPOs<\/span>, as they <\/span>no longer can serve as a reliable basis for making decisions<\/span>. Gut feelings don\u2019t always work and businesses cannot afford to act upon them.<\/span><\/p>\n\n\n\n

That is why we A\/B test to verify<\/a> our hypotheses. It is a clear solution to the problem of being unsure whether your ideas will actually work. Sometimes ideas come from inspiration and sometimes they come from research. If these two fail, panic ensues; unnecessarily, as you\u2019ll see below.<\/span><\/p>\n\n\n\n

There is a low hanging fruit, one that is a goldmine for A\/B tests: feedback. Feedback coming from your users and customers pinpoints all the issues they are facing and allows you to instantly identify possible hypotheses for your future A\/B tests. <\/span><\/p>\n\n\n\n

Download Free: A\/B Testing Guide<\/span><\/em><\/a><\/h2>\n\n\n

Users Know Your Product Better: Listen To Them<\/h2>\n\n\n

A\/B tests are great for optimizing content. All decisions come from evaluating the performance of tested elements. A\/B tests aim to achieve an expected result and start by establishing a hypothesis. Design, or changes in design, revolves around user behavior and designers\u2019 aim is to bring the product towards the people.<\/span><\/p>\n\n\n\n

\n

Yet, sometimes it is users who bring your product closer to you.<\/p>\n<\/blockquote>\n\n\n\n

See, you probably know well that people are not that easily programmed. We can\u2019t just go to Google, enter \u201cHow to make people do X on a website\u201d and browse through results that tell us \u201cRed CTA button<\/a> will increase your conversions tenfold\u201d. Because then you will be making the carnal mistake of thinking it\u2019s design elements that bring out the desired changes in people; While actually, it’s user behavior that steers design changes.<\/span><\/p>\n\n\n

\n
\"A\/B
Image source: Instapage<\/a><\/figcaption><\/figure>\n<\/div>\n\n\n

We need to constantly ask ourselves \u201cwho is using our product?\u201d and then design according to what we learn about the users.<\/span><\/p>\n\n\n\n

A\/B test hypotheses gathered from users can indirectly solve one of the issues often covered when discussing A\/B tests. Kohavi et al. point out limitation of A\/B testing<\/a>, where it’s mentioned that they provide quantitative metrics without explanations. Yes, usually you are able to verify only which variant is more successful, but you do not understand why. Knowing why can be the basis for future improvements in order to ensure coherence of your designs. Hypotheses gained through users are already able to tell you why people want a change.<\/span><\/p>\n\n\n\n

\n

A\/B tests help find which changes convert best, but they don’t reveal why.<\/p>\n<\/blockquote>\n\n\n\n

Guys from Dropbox created a simple solution for people<\/span> who wanted to share their files effortlessly. However, their road to success<\/a> was not as effortless \u2013 they needed to iterate their product and marketing based on feedback they received from their users.<\/span><\/p>\n\n\n

\n
\"Feedback
Image source: Slideshare<\/a><\/figcaption><\/figure>\n<\/div>\n\n\n

HackerNews gave efficient feedback for Dropbox creators, and they used it in order to improve their landing page and value proposition.<\/span><\/p>\n\n\n

\n
\"HackerNews<\/figure>\n<\/div>\n\n\n

Of course, the actual process that Dropbox followed must have been more developed and extensive, and so should be yours. The feedback you receive serves as a great basis for any further developments. If your users are pinpointing an issue, you need to use it as an opportunity for improvement. However, every conscious designer realizes that his or her ‘ideas’ are not always the best solution to a problem.<\/span><\/p>\n\n\n\n

That is why we A\/B test to verify the hypotheses based on the feedback.<\/span><\/p>\n\n\n\n

Since we learned that feedback is of paramount importance, it is best to focus now on the methods that can be used to gather it.<\/span><\/p>\n\n\n

How To Gather User Feedback and Fuel Your Hypotheses<\/h2>\n\n\n

Being a thorough listener and gatherer of customer feedback can be an arduous task. However, many methods make that process fundamentally easier. <\/span><\/p>\n\n\n

#1 Live Chats<\/strong><\/h3>\n\n\n

are a great way of cumulating info about smaller and ongoing issues, which can be resolved on hand. They do not necessarily need to be A\/B tested, but sometimes users and customers may point out issues that are seemingly small, yet can result in rather large changes on the website \u2013 these do require A\/Bs.<\/span><\/p>\n\n\n

\n
\"How<\/figure>\n<\/div>\n\n\n

<\/span><\/p>\n\n\n\n

With live chats, you can also immediately ask important questions that would enhance your process of introducing improvements. Simply asking \u201cwhy\u201d and learning the answer will inspire you in various aspects.<\/span><\/p>\n\n\n\n

Related Post<\/span><\/strong>: How Badly Does Your Online Store Need Live Chat Support<\/a><\/p>\n\n\n

#2 Exit Intent Popups<\/strong><\/h3>\n\n\n

Exit-intent popups<\/a> are another option. If you set up a script that shows a popup when a user attempts an exit, you can align it with the user\u2019s action and see <\/span>when<\/span><\/i> they decide to leave. Pairing it with a response coming from them can paint a bigger picture showing the reasons behind their actions, in this case – their exit.<\/span><\/p>\n\n\n\n

Or you can ask them to subscribe.<\/span><\/i><\/p>\n\n\n

\n
\"Subscription<\/figure>\n<\/div>\n\n\n

Sample situation<\/em>: \u201cI\u2019m leaving because the page is unreadable!\u201d says the customer. The data from other tools (for example heatmaps<\/a>) show that the user scrolled only halfway through the page. It indicates that these areas need grave improvement (and so does the content below, as it doesn\u2019t attract enough attention). Comparing this to other data coming from other visitors may narrow the problematic areas even further. A great basis for formulating a hypothesis of an A\/B test (e.g. The article headlines are uninteresting).<\/span><\/p>\n\n\n

#3 Feedback Widget<\/strong><\/h3>\n\n\n

Feedback widgets present on the website are a third form of gathering feedback. They are usually attached as clickable boxes to the website, at either side of the screen. <\/span><\/p>\n\n\n

\n
\"Review<\/figure>\n<\/div>\n\n\n

Users can click them and provide answers to questions or general open descriptions on issues about which they want to talk. <\/span><\/p>\n\n\n\n

If you are suspecting that there is a problem with a given element on a page, you can inquire about it and verify whether you should pursue that topic any further. <\/span>Here’s a great source of possible questions you can include<\/span> in your widget, which you can use as a basis for gathering your A\/B test hypotheses.<\/span><\/p>\n\n\n

#4 User\u00a0Forum<\/strong><\/h3>\n\n\n

The last form of gathering feedback has been already aforementioned using the Dropbox example. You can set up a contact forum (or use an already existing one! Go to reddit.com and find a specific subreddit, for example), where your users (or future visitors) are able to create discussions and talk about your website. Problems might be unusually isolated, and other users can verify that for you, so you can verify what the priority is.<\/span><\/p>\n\n\n\n

Download Free: A\/B Testing Guide<\/span><\/em><\/a><\/h2>\n\n\n

Common Mistakes in A\/B Testing<\/strong><\/h2>\n\n\n

Remember, there are certain mistakes that you should avoid in order to make your A\/B study viable. <\/span><\/p>\n\n\n\n

First, be sure that your results reach a statistical significance. Basing yourself on a small portion of the results will not verify your project efficiently. Second, be sure that you have enough traffic, as studies dragged out in time might even make some changes obsolete – time, technology, and people progress unbelievably fast.<\/span><\/p>\n\n\n\n

Related Post<\/strong><\/span>: 5 Reasons Why Your A\/B Tests Fail<\/a> (and how not to)<\/p>\n\n\n\n

Moreover, be sure not to go crazy with the variations \u2013 too many of them and you will spend too much time on the study, thus invalidating the study. Also, prioritize ideas that are quick to implement \u2013 more often than not a small change can yield very positive and fast results. <\/span><\/p>\n\n\n\n

Here’s an engaging discussion from the VWO Podcast with CRO expert Stuart Scott about running better A\/B tests and prioritizing test ideas.<\/p>\n\n\n\n

\n
<\/div>