Guest Introduction
Niti Sharma: Hey everyone, my name is Niti. I’m the editor in marketing at VWO and I welcome you back to another episode of the VWO Podcast. I am very excited for this episode because our guest today is someone from VWO, someone who has been an integral part of the VWO growth journey. So without further ado, let me introduce Kandeel Chauhan, Director of Product and Engineering at VWO. He has been leading the transformation of VWO Insights, which is one of our flagship products. He’s passionate about building products that not only help businesses better understand user behavior, but also create a real impact on growth and revenue. Over the years, he has worked across various areas of product management from shaping product roadmaps and studying user needs to making data-driven decisions that guide innovation. Kandeel has also for a time period, led quality assurance at VWO, where he built strong processes, mentored the engineers, and helped shape how this function worked across teams. In today’s episode, we will of course, talk about Kandeel’s own journey, but we’ll also ask him what goes into building a product that customers absolutely love, and how VWO Insights has evolved under his leadership. Kandeel, welcome to the VWO Podcast. How are you doing?
Kandeel Chauhan: Thank you so much. I’m very excited to be here and thank you so much for that lovely introduction. I think I’ll have to steal that for my LinkedIn bio as well.
Niti Sharma: Oh yeah, that’s a good idea. Absolutely.
Conversation
Personal Habits and Current Work
Niti Sharma: So can you tell me some quirky habit about yourself? Like is there something that you do before a big day, before a big presentation, or do you have some ritual or any quirky habit of yours?
Kandeel Chauhan: Before presentations, I will typically take a few deep breaths and that’s about it. If you’re prepared well in advance for any presentations, it’s just about controlling how you breathe. So I’m big on exercising. And so that is why I know the value of breathing fine. And yes, so that is my quirky habit that I always do before a new presentation. Even in town halls and camps, when we have to present, you can always find me doing my deep breathing before we begin.
Niti Sharma: Awesome. That’s fantastic. And is there something very exciting that you are working on currently that you can share with us, either on the personal level or professional?
Kandeel Chauhan: Yeah, so on the personal side, I recently did the Oxfam competition, quite popular worldwide. So I managed a decent time of one hour, 31 minutes. So I’m planning to do that again next year. Hopefully I’ll breach the one hour 15 minutes, so by that time I’ll be 40. So maybe I can do well in my age category at least.
On the professional front, I think the way that VWO is shaping up, that is really exciting. Every day there are new challenges and new frontiers to explore, especially with the AI coming in. And the way that we are now starting to enhance our quantitative analysis bit as well. So I can’t disclose much at this point of time, but that will become a very strong tool of VWO as a platform entirely. So, very excited to be working upon that.
Journey as a Product Leader
Niti Sharma: Awesome. Awesome. And that gets me to ask you that you’ve been part of VWO’s journey obviously for years now, and you’ve gone from, you know, while we started as an A/B testing platform, we are now an enterprise grade product. Right. And so when you look back at this journey, do you think there were specific moments or specific decisions that really defined your growth as a product leader?
Kandeel Chauhan: As you rightly pointed out earlier, so when I was hired at VWO, I was hired to lead the quality assurance team in the engineering team. So that really gave me a great platform because the role of a QA analyst is to be the advocate for its users. You know, find the friction points, find bugs that will prevent the users from getting to value.
So that acted as a great platform when I moved into a product role. Again here, I would really like to compliment the open culture that we have at VWO. So when this opportunity really arose, I just put my hand up and volunteered that I wanted to try this out, and I was given that opportunity and the relevant guidance as well, which gave me the platform, or a launchpad rather.
So going forward, when I just took up this role, we launched a massive research drive or campaign. So I still remember, we thought that we should be big on testing our own medicine. So we started looking at a lot of session recordings, doing quantitative data analysis through funnels and metric reports moving forward into heat maps, launching a lot of surveys.
So my first instinct was to talk to a lot of people. So we did dozens of interviews with not just users of VWO but we also interviewed other agencies. Given that VWO has a strong network, we were able to get those done through our sales teams, et cetera. And that gave us a real insight, right? So that showed me what is the motivations of our users? What does success look like for them? Right? And what are the frustrations? What are the pain points of using tools like ours? So, as I said, these were users, not just of VWO, they were users of our competitors as well.
Niti Sharma: Right.
Kandeel Chauhan: So that really gave me a great starting point. And to give you an example, something which seems very simple now, but when working from the engineering point of view, when we work on, let’s say a session recording tool, it is obviously a very powerful tool to have while doing behavioral analysis. But you think in terms of features, right? That, okay. My session recording supports rage clicks. It gives you an event stream. It can play for single page applications. That is how you think of it. But when you start discussing that with customers, you get to know what a pain session recordings really are because you cannot look at thousands of session recordings, or even if you find bits and fragments of some interesting insight, you don’t know if this is the only visitor who’s faced this issue, or is this something that others have also faced and worth tackling.
So when you come to the product point of view, then you look at these pain points, and then you look at your product. Then you think of things like, can my customers directly shortlist the session recordings that they have to see to only the people who dropped off at a funnel step, or if they’ve discovered that there’s somebody who is doing dead clicks on a certain banner on their hero section, mistaking it for a clickable CTA, are there others? So then you can quickly come back and segment the session recordings or filter them based on people who dead clicked on a certain element on a certain page within a certain date. So that really narrows it down. So that is how the perspective really changed.
So I think my growth as a product manager or leader is basically I could—there will be several things, right? That I could say retention, looking at data and looking at competition industry landscape. But the one thing that typically stands out is to be able to look through the prism of empathy. And interestingly, that is the product that we are building, right? So VWO Insights itself, it also allows our customers to look at their customers by stepping into their shoes. So that was a great match, and I think coming from the QA background and then evolving into this, it was a great opportunity for me. Very challenging, but rewarding nonetheless.
Building VWO Insights
Niti Sharma: Absolutely. Absolutely. And what you mentioned, I think, has been such an amazing breakthrough. I feel so that’s absolutely fantastic. You’ve spearheaded the growth of VWO Insights. Just for our listeners, VWO Insights is the behavioral analytics module within the VWO platform. So were there some defining turning points? I know you mentioned session recordings is one example that you’ve already given us, but any other turning points, whether technical or strategic, that taught you—you felt like, you know, this really taught me what it takes to build a product that customers trust to use. You know, they trust this product, they love this product.
Kandeel Chauhan: So I’ll leave the technical part out. What comes to mind is a strategic aspect. So I think from a strategic shift, I would say going for—for a product manager, it is very tempting to go after blockbuster features, right? And miss the bigger picture while doing so. So if I were to give an example, so in Heatmaps, which is one of the tools that we work on in Insights, in the past year or so, year and a half, we’ve built multiple wow features, right? So we’ve got friction maps, we’ve got zone analysis. We have side by side comparisons. We’ve built a host of them. But what we realize is that none of them standalone can create those aha moments for customers. So what the customers really want is not the blockbuster features, but a platform or a system that enables their workflows to flow seamlessly through. Right. So that they can get the value.
So that was a big strategic shift. Although I think this got a little too abstract, so I’ll just try to zoom out a little bit. And this is where I typically start all my discussions and I think I will bring this up again and again. So I will try to paint the picture of an overly simplistic lifecycle of a CRO lifecycle, right?
So if we could imagine it in three nodes. So the first node is monitor. So when any team starts their CRO lifecycle, they start by monitoring. By monitoring, they’ll monitor what is the trend of their revenue? Is it going up, is there a certain dip? Then if they want to enhance it, they go deeper into funnels. So they monitor their funnels. So maybe they have a checkout funnel which goes landing page, product page, add to cart, checkout. Then they say, where are the dips or the maximum drop offs? Keep monitoring this. So this is the monitor part. This is the first node.
Then comes the analyze part. So analysis comes to where, let’s say there is a 30% drop off at a certain step in the funnel. What is the segment that is dropping off the most? Why are they dropping off? So then you jump into behavioral analysis. You look at heat maps, you look at session recordings, try to identify friction points. Then you create a hypothesis. So the hypothesis could be as simple as that, Hey, these guys are struggling to find the CTA that they should find, and let’s just change the color to red. So again, taking an overly simplistic example. So based on this, then you come to the next node, which is optimization. So now you run experiments. If you find something interesting, you deploy it. If you think that only one segment is working for one segment, this variation is working perfectly, you launch a personalization. But this is not the end.
So this lifecycle keeps on going. So even when you do this, then you again monitor the test participants. Then you again analyze, then you again optimize and you keep going through it. That is how you compound.
So what we found out, and coming back to my fragmented feature set problem that I highlighted. So what happens is in the industry, the tools are absolutely fragmented. A customer would be using an analytics tool on one platform, a session recording tool on another, their experimentation is somewhere else. Personalization is another one. There is some marketing automation happening downstream, so this leads to a lot of friction. A) They have to train their teams on multiple platforms. B) They’re always dependent on development teams because there are API glues that are needed. Data inconsistency is a big problem.
So the strategic shift for us was to—can we as VWO evolve into a platform that we’ve evolved into over the past few years where we can provide this end to end CRO lifecycle on one platform itself. And that is what reflects in all our strategies now. So if you look at VWO from Monitor, you can monitor your funnel dropoffs. Immediately from the dropoffs, you can see segment filtered session recordings of people who dropped off. You can compare dropoff versus conversions in heat maps, all seamlessly connected. If you see even within Insights, if you see a cluster of clicks on a specific area in a heat map, you can quickly create an area over it and see the session recordings for people who clicked there. Right? And then of course, getting into tests and being able to see session recordings for people who became a part of the test, who was served a particular variation. Same with personalization.
So that entire—it’s a closed loop system that we try to create everywhere. So that was a big strategic shift, and that has really given us a lot of results in terms of our customers being happy, being able to get a lot of ROI and of course great for our numbers as well.
Using Behavioral Analysis During and After Tests
Niti Sharma: Wow. That makes a lot of sense. Thank you for explaining. And you touched upon this slightly, but I want to get into a little bit of detail. Conversion research—there’s, I don’t think there’s any debate around that. I think everybody agrees that user behavioral analysis is an absolute must. But we get this question a lot that we understand that it needs to be done. If we specifically take just A/B testing, people understand it needs to be done before the test, right? So that it feeds into your hypothesis, but what is the use of this process and of a module like VWO Insights after the test? How can it be used after?
Kandeel Chauhan: Yeah, I think you make a right point, but I will take it a step further and say that, not just before and after, but it gives you a lot of value even during your test.
Niti Sharma: Hmm.
Kandeel Chauhan: While your test is running, you can still leverage session recordings to get early feedback. So I’ll just try to elaborate a little bit on what happens while the test is running. So let’s say that I’ve just launched my test and visitors have started to be served with certain variations. So immediately you can start looking at session recordings of people and see if they’re interacting with your webpage the way that you had hypothesized. So maybe you thought that after making such and such change, the flow of the user will change, but is it really happening? So you get that early feedback. Right.
Even we’ve seen our users, our customers identify conversion blocking issues on those session recordings. As soon as—like a CTA is not working. So their test would fail for the wrong reason. If it weren’t for session recordings and them finding out at an early stage, they would lose time and they would lose conversions. As a result, they would lose on revenue as well. So it’ll be a very expensive test. Right. Even other scenarios such as your guardrail is being triggered, your test running for a significantly long time, and getting very low convergence. All of these can be investigated during while the test is running.
Of course, this is no substitute for a statistically significant test result. You’ll always have to wait for that. This is something that can complement it. It’ll never, it is never meant to be as a replacement for it. Right?
And once your test ends, we always have this—we always talk about this, sometimes a failed test can be of more value than a small win because once you dive into knowing why, there is a treasure trove of insights to be found. So just like immediately after ending a test or even during it, you can compare the control and the variation or multiple variations side by side view. You can see synchronize, scroll them down and see how attention has shifted. Right. Even in session recordings you would do that.
So in VWO specifically, we’ve built session recordings in a way that you can see all the sessions of a user after the point he became a test participant, you can see how many sessions he took to convert the metric or did not. You can compare them. There’s AI co-pilot there to assist you with summarizing it in context of the test. So that gives you a lot of value.
So an analogy that comes to mind there is that when you’re a CRO professional, running a test, you’re like a coach of a team, right? And if you’re only looking at the result, it’s like looking at the scorecard at the end of the game and saying that, oh, my team lost, but you don’t know why they lost. So in the next game, you don’t know what to tell them or how to make them improve. So whether you win or you lose, knowing what has gone wrong or what could be done better, how could you win bigger? All of those definitely require behavioral analysis.
Customer Understanding of Behavioral Analysis
Niti Sharma: Wow. Awesome. That was a big light bulb for me, this analogy with the coach. Thank you. And have you ever tried to understand what customers actually think of behavioral analysis? Because, you know, we’re part of the industry and we read about it. We deal with it day in and day out. But people who use products like VWO, our platform—how do customers define it? Like, you know, what do they associate it with and do you believe that today in the industry, people have a certain mature understanding of this?
Kandeel Chauhan: Actually interesting that you ask this because we’ve actually researched this quite a bit in the past. So what we realized is that the understanding or adoption of behavioral analysis as a whole, it is not a point. It is an entire spectrum. So there are people at the top of the spectrum. There are people at the bottom of it as well.
So to put it into a framework, there is an awareness spectrum or awareness levels. I keep getting this wrong. So it is by Eugene Schwartz. So it breaks down your customer base or your prospective customer base into bands of awareness. So there is the unaware state, there is the problem-aware state, and then there is the solution-aware followed by the most aware or all aware state. Right?
So I’ll try to answer this question through this framework. So there are some—the awareness of behavioral analysis is directly correlated to the CRO maturity of that organization. Right? So if you are just starting out. I am an absolute beginner, new digital business. I have just brought together a website and put it live, and now I want to start improving it. I am currently unaware that I need anything such as behavioral analysis or session recording. Let’s take an example because at this point of time my website is in such a poor state that even a normal UX audit will give me a lot of results. There are still a lot of low hanging fruits to be plugged.
So these teams will typically hire a UX agency and they will tell them that, oh, okay, your product page structure is not right. You move your reviews up, there should be a sticky add to cart button, things like those. And they keep doing it and they see the numbers growing. So they are currently unaware. They think that their intuition is working perfectly fine. And things are all great. But the trap is just very close and they slip into the problem-aware state now.
So from the unaware phase of the spectrum, they’ve now slipped into the problem-aware. How this typically happens is they start to get counterintuitive results from their optimizations or experiments. So for instance, let’s say that they thought that, oh, to increase the checkouts, let’s introduce a discount coupon field. So they introduced a discount coupon field, and they discover that the checkouts actually went down. So now they don’t know what to do. Right? And now they are problem aware. Now the only way to get out of this is to become solution aware.
So if I was solution aware, I would know that, okay, this is a problem. There is a solution in the market, which is session recordings. I should go and get that. And they typically, once they are all aware or product aware as well—so how they will typically approach this is that they will run an experiment with the new coupon code. When they look at session recordings, they will find an insight. Maybe they find that people see the coupon code, they don’t have a relevant coupon code. They’ll see a lot of tab outs happening. People are leaving the page, going to other coupon sites, looking for coupons that they could use. That is causing the friction. And then they just frizzle out. Right. So this is an insight that was waiting for them to happen.
So again, coming down to the—in the long run, the CRO success is significantly higher for people who are on the bottom end of the spectrum, who are all aware or solution aware at least. But for the unaware also because they’re just starting out, it might not impact them for a while, but they will also get to it.
And at VWO also, we typically have this as a core mission, you could say, that we want to help people transition from those states, right? Because we do through content, we do some education. We’ve got in-app educations for our customers. And then of course our plans are tiered in such a way that the unaware and problem aware can also get certain benefits. Like our AI co-pilots can act as a UX auditor, give you certain low hanging fruits. Then you graduate to the next level on the spectrum. You get session recordings attached into your test and personalization reports.
So there is something for everyone and it is a tool that basically scales with you. So as and when we promote this, that is a win-win for everyone, right? Our customers get a bump up in their revenues or leads. Their visitors, they get a seamless experience, which they enjoy. We are able to expand the market for the industry as well. So yeah, so that is why we had researched it in depth and this is what we found. And we’ve seen customers really evolve gradually, right? Even smaller companies now have very mature CRO lifecycles, which is very interesting to see.
Empathy and the “Why” Mindset
Niti Sharma: Yeah. Wow. And I can imagine how much pride you must be feeling for the work that is done because we are not just offering solutions, which, you know, some people may or may not use, but we are actually helping them reach there and move forward in the spectrum, like you said, and we are actively participating so that they succeed. That’s just so great.
Kandeel Chauhan: I think that comes from the DNA of the company as well. I think that was the vision set by the leadership and the team on a whole. I’ve been really blessed to be working with a great engineering team, great CSMs, great sales team, so that makes my job very easy.
Niti Sharma: Hmm. Yeah. Yeah. And again, just coming back to a very basic question that’s also I think asked many times. So when we talk of user behavioral analysis, we usually compartmentalize it into the what and the why. So what is happening on the website or on the app and why it is happening. And how do you think businesses can connect with this “why” mindset and how important it is so that it’s just not seen simply as a process or a feature, but like a mindset that, you know, drives better understanding of the user and ultimately leads to better experiments?
Kandeel Chauhan: I think the mindset is the key word in that question, and I think I’ll try to tackle this by specifying what kind of mindset has been adopted by teams who do really well at behavioral analysis. Right. So the first thing is that you have to detach from looking at behavioral analysis tools as just another report. You can call it an empathy engine, right? So this is where you start to draw empathy with your visitor base or your user base, right?
So rather than just putting a number to it and asking a question that, Hey, why is my bounce rate so high? You must make that cultural shift and start asking Why is my user so frustrated? So it is a simple switch of mindset and that is a change that is needed when you’re looking at behavioral analysis.
So this is from the few of the interviews I’ve—I’ll just quote a few customers. So there are certain processes that they have designed. One of them, which comes to mind, is they used to—they made this a team activity. So every week, every two weeks, they would order pizza and they would sit and watch session recordings after a release. So the entire team, the developers, the UX guys, all of them would be there. And what they pointed out that it was great in terms of breaking personal biases. So something like a design can be very subjective. So I am the designer. I hold a very strong bias towards it that this is the best. But when you see users struggling with that design, there is no need of convincing anyone. They have seen how users are using it and they can see the problems. So that was one thing. So again, empathizing with the user, not just looking at the numbers. Right.
Another one that came to mind was that in the email reports, they would not just say, oh, okay, 35% drop off in the funnel, they would also attach a session recording where a person is rage clicking on the checkout button. So then this becomes an undeniable and very tangible factor, right? This is no longer hypothesis in the air, right? The only thing that you now need to see is to quantify it, that how many other users have also faced this? But that is the mental shift that is required, or a mindset change that is required to be great at behavioral analysis.
Starting with Behavioral Insights
Niti Sharma: Right. A hundred percent. Like it makes a lot of sense, especially how you call it an empathy engine. I think that’s so cool. Do you have any advice for product or marketing teams who now want to start integrating behavioral insights into their CRO process, but perhaps don’t know where to start?
Kandeel Chauhan: Yes, yes. So my advice to anybody who’s starting out with this is to not—my advice would be to go top down rather than bottom up. I’ll just explain this. What we’ve observed is new teams who are just starting out. What they do is they get overwhelmed by the breadth of data that they have, right? So they need to remember that they don’t have to analyze the entire website. They have to analyze a problem. Right. So they need to start with the goal.
So what is your most expensive goal at the moment? That is what you have to think. So my top line is struggling. The biggest drop off happens at my cart page, before my checkout page. That is the problem that I need to solve. I don’t need to jump into heat maps and look at every page. I don’t need to jump into session recordings and see thousands of session recordings. I start with the funnel. I have a goal in mind, then I go deeper.
I look at heat maps specifically for that page. I ask this question to myself: What are my users seeing? That is answered by the heat maps. What are users converting versus users who are not converting seeing different? So you compare those two and you’ll immediately see what is the difference in engagement.
The same question: What are they experiencing? Now you ask that and the answer is in the session recordings. Now you look at 10 session recordings of people who’ve just dropped off. You look at 10 who’ve not dropped off. You do a comparison. Even after this, even if after this you do not have any clue or you are still on the fence, then you would simply go and ask your visitors. You run a survey, right? Hear it from the horse’s mouth.
So you have to just find your first why. So that is what you’re after when you’re just starting out. You don’t need to analyze the entire website. You just identify a problem. That is what I mean by top down, you have a problem. Then you go down. So if you take a bottom up approach, which is you’re looking at session recordings, you’re looking at heat maps of various pages, you’ll get fragmented insights here and there, but you won’t be able to tie them up into a bigger picture, quantify them or justify your hypothesis into why you want to solve them.
So my only advice to people who are starting out is that go top down, start with the goal, and then dive deeper.
AI and the Future of Behavioral Insights
Niti Sharma: That’s great advice. And there’s so much talk about AI also now, right? Seeing how AI is transforming experimentation and analytics, both. Do you predict any specific breakthroughs or, you know, something that’s likely to change in the way teams will use AI for behavioral insights or even for conversion optimization in the next couple of years?
Kandeel Chauhan: Yes, actually, I am very bullish personally. And I think I resonate with what we think as an organization as well on how AI is going to shape this industry. I think it is going to revolutionize it entirely. What we’ve already launched is just scratching the surface.
So right now where we are at is—so let me just paint a picture of what it could be in the future and then I’ll try to trace back my steps. So I think AI will help us finally reach a state where there is real one-to-one real time personalization happening. So what I mean by that is that, let’s say that I have a visitor, right? On my e-commerce website, I sell clothes, right? So the AI is constantly monitoring, right? It has got access to behavior. It has got access to analytics, it has got access to all the tools that are currently on the system.
So it observes that there is a visitor who comes in and the visitor looks at red dresses, checks out three red dresses, right? So now my AI realizes that this is not somebody who’s just casually browsing. She’s probably someone who’s shopping for a specific event. It’s given by the choice of the browsing. So it’ll personalize an experience for them. When they land on the homepage, they will see a banner, which is specifically for them, a one-to-one realtime personalization, saying that, check out our party dresses or something, right? And maybe a discount coupon with that. So that nudges them forward. So that can be the power of real time personalization. So that is the direction in which we’ll be going eventually. Right.
But we are still just scratching the surface and we have somewhat of a phased plan to get there eventually. So currently where we stand, at the moment, we’ve got co-pilots, which are more like reactive assistants, right? You call upon them, you tell them that, Hey, summarize this heat map. Summarize this set of recordings. These are my survey answers. Give me the key actionables from it, right? But they are going to transition gradually into proactive creators. They will have access to personalization campaigns running. So that is the direction in which we will be going. Right?
So the reason why we started off on this AI journey was—it comes from two basic missions that we had. The first one was to decrease the time to value of a customer, right? The time and effort that goes into behavioral analysis is massive. So you’ll have to look at a lot of recordings. You need a separate team maybe. But AI, our AI co-pilots drastically bring that time down. So the data currently shows that our customers have—only if you only talk about session recording co-pilot, they have already saved thousands of hours. Right? So that is a big impact.
So that is the current capability. It is a reactive assistant, which only has expertise of a certain area. So there is an assistant for session recordings. The next phase that we go will be agents. So these agents will be experts at specific tasks. So there will be a data analyst agent, which is an expert at monitoring trends, identifying anomalies. There will be another which is an expert UX agent, which can look at heat map and tell you UX inputs. And then there will be workflow builders, which people will be able to generate themselves. They will be able to attach these two together. That, okay, whatever findings you have, Mr. Analyst, you pass that on to the heat map. So now the heat map knows what is the problem area with context to the data, where are the drop-offs? And the heat map agent will only focus on those pages.
So a string of micro agents together will form a super agent, which will be able to generate this. The next enhancement in this would be where it has got access to all the capabilities of creation in the platform. While we still have in early stages a one click experimentation setup, all this is going to really become super intelligent in the future until we come to that one-to-one real time personalization. So that is the direction in which we are moving currently.
Human Role in AI-Driven Analysis
Niti Sharma: Sounds really awesome. Where, you know, how we are going ahead. And also how fast we are going ahead. I think sometimes the speed is also like, it just gets too much to take.
Kandeel Chauhan: Yes, yes. Very exciting times, I must say.
Niti Sharma: Yeah. Yeah. Super exciting. But I think I also wanted to talk about the opposite of this exactly. You know, like how—you touched upon empathy a couple of times in our talk today. And some of our previous guests on the podcast have also spoken about empathy, about intuition. And how as CRO practitioners, it’s important for us, of course, to upskill, to use AI, to do a lot of research and to stay on top of market trends, to come up with processes and tools and all of this. But it’s also important for us to hone our intuition more and more.
So my question is like, do you agree with this? And if you do, then do you think that there are some parts of user behavioral analysis that AI cannot take over. And, you know, AI just cannot touch. And hence these cannot be automated, where judgment, empathy, human interference will make all the difference.
Kandeel Chauhan: Correct, correct. So I think I made it sound like the humans are going to go out of the loop in the previous answer, but we still believe that the human in the loop is very much relevant even in the future, even when it gets to that stage which I tried to paint a picture of. So human in the loop will always remain, right.
It’ll be required because—okay, I’ll try to tackle the behavioral analysis aspect first. So when we do behavioral analysis of a visitor on a page, it is like trying to study its digital body language, right? We do it through signals, right?
So again, drawing on an analogy, let’s say that a person walks into my physical store. If he is angry or in rage, it is very hard to miss for a store manager, right? He’ll know immediately that this guy is completely enraged. So how do we do that for our website visitors, right? So we do it through behavioral signals. Let’s say rage click is a simple explanation of that. There could be many page refreshes, cursor thrashes, all of those kind of signals. So CRO experts will typically learn to read those signals as digital body language and form their hypothesis accordingly.
So even AI now, as it can interpret language also, so it can be trained to now be an expert on reading the digital body language as well. Right? So we are already about to release a feature where your survey open text fields responses can be segregated on themes and the sentiment as well, right? Similarly, this can be done, but the human will always be needed to approve it. At least in the beginning, right?
So the AI will never have a few things, right? It does not have the context of the goal on its own. Right, so the human will be needed—what is my brand awareness or its business goals or maybe his specific vertical. What is it looking at? Right? So those things from the individual has to come. Then there will be guardrails to set. So you don’t want the AI to just to increase the number of conversions, give a 99% discount to nudge. So the guardrails are also there. And then approvals. So at least in the short term, there will be a lot of hypothesis that the AI will generate and the systems will enable the user to manually verify if the user is doing that right or not. Right.
Of course, even when it comes to empathy, AI can actually be a force multiplier there as well. So one thing comes to mind is that we as humans, again, we are prone to personal biases. Right. So we’ve had customers who tell us that when they did a co-pilot analysis of the heat map, there were some very obvious insights to be seen, but they just couldn’t see it because they had some bias or they were looking at it from a certain perspective. But the AI doesn’t have any bias. It is totally objective. So it helped them.
So the human is not going out of the loop. It is going to be a force multiplier. So if a CRO agency today is running with 10 people, maybe their productivity can go up 50 times or a hundred times even because their manual work is gone. Right? Although their manual work is gone, the value of each person working now—every human in the loop, the value is improved because they are now strategic visionaries rather than doing manual work. So that is how I see the evolution of humans in the loop. But AI is definitely going to revolutionize this. It is going to be really powerful and I think we at VWO are also—a lot of research, a lot of POCs are currently ongoing and very, very excited for the future.
Niti Sharma: Great. That’s great. And thank you for explaining so practically how this is to go out and how the mindset should really be that AI can be seen as an ally and a force multiplier, like you said, right? Instead of a threat. And it’s an opportunity for humans to take a more strategic role. I just love the way how you—
Kandeel Chauhan: That’s right. So us versus them I think is the wrong way of looking at it. It is a tool that will definitely help us improve productivity, design better experiences for our visitors.
Evolution of Experimentation Culture
Niti Sharma: Okay. And then Kandeel, like if you see how teams use VWO today. Now this is a very VWO specific question that I’m bringing here, but like how teams use VWO today versus how they used to use 5, 6, 7 years ago. Is there one most rewarding change that you have observed based on, you know, how people think about experimentation and learning today?
Kandeel Chauhan: I think more than VWO, it’s overall in a larger picture as well. So I would say the previous five years or 10 years maybe even. So in that period, there is a huge transition around the democratization of experimentation. So earlier it was a very tech focused—I mean, it happened as a siloed task happening in certain areas. A good to have rather than a must have is how I would put it right.
But gradually as the tools have improved, and I think VWO has to be given a lot of credit here. VWO is the pioneer with the visual editor that came in, and it really took experimentation into the non-technical areas as well with marketers, product managers, all of them having the capability to run experiments without the interference or dependency on a development team. So that was one.
And the second one is even coming to smaller businesses. I think I touched upon that earlier as well, that even smaller businesses have a very mature CRO optimization process running. So even four people teams are very serious on how they look at it. They’re using the entire lifecycle. They’re monitoring it, they’ve set alerts, they’re looking at behavioral. They’re continuously churning, right? They’re failing. But they again, learn and that is how compounding works, right? And you can clearly see what competitive advantage this culture of experimentation is bringing to them.
And I think as Visual Editor did this for testing or experimentation, I think AI is going to do that for behavioral analysis as well. Because the biggest constraint was of time. And smaller teams can also have these co-pilots point them in the right direction. Have the human focus only in areas which do have—you don’t need to find the needle in the haystack anymore, right? So that is going to democratize this further.
And gradually, I think more and more digital businesses will gravitate towards this entire process. And more and more they will continue to ask for a unified platform where you’ve got everything in one place. So I think that is something that across the industry, there is a trend. You can see all legacy players trying to build out other capabilities now. And it’s going to be a very interesting time ahead. I think the industry is going to grow. The people are going to get better experiences, the end users and our customers are definitely going to get better business.
Final Thoughts
Niti Sharma: Fantastic. Any final thoughts or message that you’d like to share with our audience before we conclude?
Kandeel Chauhan: Okay, this really went by fast. I just looked up at the time. Now, I thoroughly enjoyed sharing my thoughts. A lot of times, this gives—maybe we’ll get a lot of comments. We’ll get to engage with a few of the people. It’s great to put our thoughts out there for the world to see and discuss. So yes, if there are anything that anybody, any of our current customers, anybody else who faces these kind of problems, want a solution, have a great idea. We are all ears. Even if you’re not a subscriber of VWO, we’d still like to get a lot of inputs and let’s see how we can build this together.
Niti Sharma: Fantastic. Before I let you go, I would like to play a quick rapid fire round with you just about nine, 10 questions, and you need to try and answer them as rapidly and with as much fire as you can.
Kandeel Chauhan: Sure, sure, sure. Putting me on the spot, but okay, that this might not be my strongest forte.
Rapid Fire Round
Niti Sharma: Okay, Kandeel. So if you were starting a career in CRO today, what is the one thing that you would do differently?
Kandeel Chauhan: I would rely on a tool that gives me the scope to scale, and I would definitely leverage as much AI as I could.
Niti Sharma: Awesome. Is there one thing that your non-industry friends still don’t understand about the work you do?
Kandeel Chauhan: I think the ones in digital business, they understand. The ones not in digital businesses, they’ve got no clue. I’ve given up explaining what I do to them.
Niti Sharma: Right. One person that you feel every CRO professional or every product leader must absolutely follow on social media.
Kandeel Chauhan: There are many, actually there are a few of my friends also that I follow. So I will give a shout out to one of them. Hanan was—he used to work previously, but he’s now started his own consulting, I think. So you must follow that. We can leave his handle in the comments.
Niti Sharma: That’s Hanan Goel, right?
Kandeel Chauhan: Yes, yes, yes.
Niti Sharma: Fantastic. He used to work as a data scientist with us, and he’s doing amazing work. Awesome. Three books that you would recommend to our listeners and even if unrelated to CRO is fine.
Kandeel Chauhan: Yeah. So I had a very love hate relationship with books, but it was in VWO itself that I started reading together. So I think Atomic Habits, one of the books, the first books that I probably read and that really helped me out. And This is Marketing by Seth is what I’m reading right now. So I think that is giving me a good insight. So these are the two books.
Niti Sharma: Awesome. Yeah. Awesome. For our listeners, just a little tidbit. We have a very awesome book club here at VWO and we read fantastic books together.
Kandeel Chauhan: All the books that I read have been through the book club.
Niti Sharma: Yeah. Yeah. What a great habit to be formed.
Kandeel Chauhan: Yes.
Niti Sharma: A go to travel destination, Kandeel?
Kandeel Chauhan: My go-to travel destination is actually my native hometown or village. So that is in—so I go there multiple times a year. But apart from that, I love the beach. I mean, we as a family love the beach. So each year we definitely go once to any beach destination. Right now, Thailand is our favorite destination.
Niti Sharma: Awesome. Great. If you had not been into CRO, what other profession do you think you would’ve chosen?
Kandeel Chauhan: I think I would’ve grown apples. Yeah.
Niti Sharma: Wow.
Kandeel Chauhan: That’s the retirement plan for me. So yes, that is what I would be doing.
Niti Sharma: That’s great. Thank you for being honest. And one CRO metric that you wish people should now stop obsessing over.
Kandeel Chauhan: I think bounce rates. So I think there are better things to monitor than simply looking at bounce rates in isolation. So you need to see the entire journeys and only then see whether those bounce rates are actually impacting your business or not.
Niti Sharma: Right. Right. And last question, is there a dream or a goal at the personal level or on the professional front that you want to achieve in the next three to five years?
Kandeel Chauhan: Ah, so actually I’m not looking at a lot of things in the long term. I’m just trying to take it day by day, living in the present. So I’ve got a 6-year-old boy who keeps me honest, no time to think about anything else. And of course, from the professional front also, in the present point there is so much challenging work to do. It is an exciting time to be working in this industry, in tech. So I would like to just continue adding value to our customers and keep on learning and breaking boundaries. Yeah, that’s it.
Closing
Niti Sharma: Super. Thank you. That brings us to the end of today’s conversation.
Kandeel Chauhan: Rapid fire.
Niti Sharma: Oh, but you were, I think you did really well.
Kandeel Chauhan: Yeah. Yeah. I just answered what first came to mind.
Niti Sharma: Yeah. No. Thank you so much for joining us today, Kandeel and sharing your journey, sharing your thoughts. It’s just been so fantastic hearing how VWO Insights was shaped and you know, how we are going today and what are the plans for the future. It’s just so enlightening. I work at VWO, but you know, sometimes you still kind of—you don’t get time to sit and chat like this and really understand what’s going on. So personally also, I think this has been very enriching for me, and I hope our listeners got value in the conversation today. If you like this episode, please don’t forget to follow or subscribe to the VWO podcast. Thank you so much for tuning in. We have lots of interesting conversations lined up. So until next time, keep experimenting, keep learning, and we’ll see you in the next episode. Thank you. Have a great day.
