Experiments, Events, and More: Fireside Chat With Mark Kilens

[NOTE: This transcript is pending review and, hence, may contain errors. The final version of the transcript will be available soon.]

[00:00:00] Vipul Bansal: Welcome to ConvEx 2022. The annual conference on experimentation by VWO, a full funnel experimentation platform. 

[00:00:15] Today we have with us Mark Kilens, who is the CMO at Airmeet also very much evident from the just freshly installed new lights in the background because Mark’s going to announce some really cool things tomorrow that is on 12th of October.

[00:00:32] So Mark’s here and of course, many of you might already know Mark. He, of course with his team, was the person behind HubSpot Academy and Drift Insider. 

[00:00:43] Isn’t that true, Mark? 

[00:00:46] Mark Kilens: That is true. Yeah. Thank you for having me. Good to see you all. I want to make this a very interactive session too. So please, please get your questions ready.

[00:00:55] Hard questions are what I look for. So the harder the question, the better. 

[00:01:01] Vipul Bansal: Sure, sure enough and we have Jan th us, who is the Director for Europe and Latin America at VWO. So he will be moderating this fireside chat. So with that, I’ll just jump off the stage, guys, over to you, Jan. 

[00:01:14] Jan Marks: Thank you very much.

[00:01:16] This is going to be a very interesting session, Mark. Because you know what happened? I was on the way to the airport and then suddenly the train stopped because of some technical problems. 

[00:01:23] So I’m literally sitting here on the floor of a train station next to the airport of Barcelona. So don’t be surprised if you hear these announcements.

[00:01:33] This is due to that. This is real life. So that shows this is not recorded, anything like that. But thank God I’ve got a good iPhone connection here and if I disappear, then the stage is all yours and you know what has happened and somebody cut it off. Good to have you here, Mark. How are you? 

[00:01:49] Mark Kilens: I am well. That sounds like quite the adventure.

[00:01:52] So yeah, who knows? Maybe I’ll take this in a completely different direction. If you get cut off, we don’t know any audience. We don’t know what’s going to happen. 

[00:01:58] Jan Marks: I’m not concerned at all about this. I’m sure that then you go ahead and you go with the flow. I would say. Mark, I have one goal for today.

[00:02:10] I’m trying to get a sneak preview information about what’s gonna happen tomorrow. I’m not sure if that’s gonna happen. 

[00:02:17] I’m a little bit concerned that I will have to wait to tomorrow, but everybody is really excited. Do you think we have a chance? Is there anything that we could say, tease you into giving us a sneak preview of what’s going to happen tomorrow? 

[00:02:32] Mark Kilens: Maybe, maybe throughout the conversation. Maybe throughout. 

[00:02:35] Jan Marks: Okay, alright. 

[00:02:38] Mark Kilens: Yeah, seven new things tomorrow. So, I hope you folks can tune in. If you do anything with live webinars, events like this, I definitely don’t think you should miss it. 

[00:02:48] Jan Marks: Yeah, I think so.

[00:02:51] Mark, we don’t know each other too long but we are kind of already best friends. Just for the reason that, you’re also very deep into experimentation and you are the living example how important it is and how experimentation can drive successful businesses. 

[00:03:07] 17 years that I’ve been part of your students when you were driving HubSpot Academy, so I know a little bit from first hand experience of what we’re doing but 17 years that have been exciting and the headline has always been hyper growth in a certain way. 

[00:03:27] So what is your takeaway? If you look back, what have been the most exciting moments and what has driven your career so far? 

[00:03:35] Mark Kilens: For me, it kind of comes down to your personal values or principles. So one of mine is, two of mine would be always be learning and never settle. 

[00:03:46] So when you think about the theme of today, you know, VWO, I’m always experimenting in my life. Not just at work, but in almost every aspect, if you will, right? 

[00:03:56] Because that’s how you can really accelerate your learning, if you will, and create these feedback loops.

[00:04:02] So, some of the proudest moments are the actual experiments that we did early days at HubSpot, 2010-11 to prove out that HubSpot Academy was something that was needed, something that the customers really found valuable and loved and was something that could really help change the trajectory of HubSpot’s growth and sustainable growth 

[00:04:26] So we did talk about experimentation everything from your traditional stuff that you know, Sarah was just talking about to experimenting with events like this we would run actually a lot of just live training, a live workshops, live events to see what messaging would land, what resonates better to this new framework work, right?

[00:04:45] Some of it was more scientific, some of it was less scientific, right? So there’s that piece, but I think the general thesis, the general mindset I have, is this belief of never settling, always be learning. 

[00:04:59] Everything could always be better and the way you do that is through experimentation. 

[00:05:06] Jan Marks: Right. I assume that in the course of the years with the different career station you had, you’ve met different companies with a different context and different organizations.

[00:05:20] Were all of those equally prepared to go, let’s say, to go with you this path of never settling in and embrace experimentation? Or did you also sometimes have to fight for it? 

[00:05:36] Mark Kilens: Luckily for me, you those 2 companies, Hut and Drift and now Airmeet, they all have this very heavy belief around experimentation. 

[00:05:46] For me, it came from just my natural upbringing, my parents. I’ve always been this kind of entrepreneur. I’ve always tried to try things, fun stories. 

[00:05:55] I built a home snow-making machine. Anyone make snow at home? I don’t know. Let me know if you want to know the answer how to make snow at home and the questions. 

[00:06:04] By the way, please make this engaging. I think I can see the Q and A.

[00:06:08] Yeah, I experimented with how to make snow at home for three, four years when I was nine years old, 10 years old and then I was able to figure it out and that led me down this whole other kind of small business path. 

[00:06:19] So for me, it’s always been part of me and I think for growth today and especially in this economy, when efficiency is so important but also not sacrificing customer engagement and customer experience.

[00:06:32] You have to be running a lot of experiments. I was chatting with you, Jan, like your website, no matter how big or small it is, you should always be testing something. 

[00:06:41] Your website should be a living and breathing thing and really your life is one just big experiment. No one really knows what the hell is going on, right?

[00:06:48] Like, we’re just living life and you’re just trying things. Look at your today, right? At the end of the day, you could say you were experimenting with travel. So yeah. 

[00:06:56] Jan Marks: I think it’s one exception, I think it’s one exception. How often have you been married, Mark, if I may ask, is that the only thing that you can’t experiment with? 

[00:07:10] So that is where you have to make sure that and then there it is. 

[00:07:14] Mark Kilens: Oh, no, you definitely experienced marriage. I didn’t know. No, I’ve been married since 2015. Yeah. Two kids. I just think it’s a really important thing to focus on and practice. 

[00:07:24] Jan Marks: Yeah. Yeah, right. 

[00:07:28] Now, these were two past very exciting stations, HubSpot, one of my favorite companies, I must say.

[00:07:35] I would say I am now suffering under the competitor. I won’t drop a name here, but I’d love to go back to HubSpot. 

[00:07:46] Now you’re at Airmeet, you’re the CMO at Airmeet and I can’t imagine there is any other company that is so in with a product there that is so needed and so necessary and so, I don’t know, dynamically evolving, which is remote meetings, remote connections, team building, education and everything remotely.

[00:08:11] I mean, we’ve all been thrown into off remote connections and Zoom, Google Meet and so on. What is the particular spicy thing that Airmeet makes it different? I know it a little bit but if you wrap this up. 

[00:08:31] So what is Airmeet’s USP against these very known other competitors or competing platforms for remote connections?

[00:08:41] Mark Kilens: I think it’s a few things, but at the end of the day, it’s about connection, right? At the end of the day, my last 15 years has been very focused on creating content. 

[00:08:52] Content’s great. You can’t deny how important content is and how people are saying, media is important. I think they’re kind of interchangeable to some degree, right? 

[00:09:01] But for me, content is like a one way type of thing. You’re not being able to experience it together, most likely. But when you do something like this, we’re all experiencing this right now live together and that’s why I think live is more important than ever before in B2B.

[00:09:20] It’s always been important, I think, in human life but like when you share a live experience with one another and you can actually engage with one another, right? 

[00:09:29] Like I have a gripe right now with GoToWebinar, just because, ironically, I was using GoToWebinar back when I was 2010 at HubSpot and that was kind of best in class at the time and I was trying to engage with the audience, interact with them, even see them and it’s hard, right?

[00:09:43] So with what we’re doing at Airmeet and you think about the experimentation we do within our products and on our websites and whatnot, it’s all about this premise of joining together. 

[00:09:51] How do we get people to join together? Either online, either in person, new technology that you can use in person to get people to join together or to the extreme, maybe trying to get the in person and online audience to join together.

[00:10:07] I think that’s much harder these days and it doesn’t really work as well. That’s like the classic. Maybe people think of hybrid, but in my estimation, it’s all about this engagement. 

[00:10:16] But really, what is engagement is about connecting with one another. So that’s what I’m asking you folks questions.

[00:10:21] What questions you have about experimentation? What questions you have about anything really? Because that’s how I learn, right? I’ve always said that people who learn together grow together and one of the best ways to learn together is to do it synchronously live versus recorded or anything like that.

[00:10:37] So that’s kind of our general thesis and our value. I don’t want to talk about Airmeet today. This is more about how I can hopefully teach you folks listening, things you can try right now to experiment. 

[00:10:51] One thing that we did with Airmeet recently, Jan, is we did this huge experiment on our homepage. I think a homepage is a place you’ve got to be running at least one or two tests every quarter. 

[00:11:03] Again, it depends on how much traffic you get, depends on what you’re doing, but at a minimum your homepage gets the most amount of traffic, number one. 

[00:11:09] Number two, you’re most likely going to be shipping new things, trying things out on a pretty regular basis, no matter what stage your company’s at.

[00:11:19] So, the fact is your homepage is such a valuable digital place for you to engage your audience. That I’m shocked that more companies don’t focus on the homepage as a testing ground. 

[00:11:35] So anything from the hero image that we did at Airmeet to decide what is the best call to action. Basically, what’s the best offer for people visiting the homepage to actually trying to start conversations with people on your homepage through chatbots and live chat.

[00:11:48] There’s just so much you can do to try to captivate and engage that audience on your homepage. I think it’s a very underutilized thing. Sarah was talking about some of this in detail in her last talk, but homepage is everything in my opinion. 

[00:12:01] Jan Marks: Yeah. I agree with you.

[00:12:05] I’m also surprised how often we need the same whole page again and again and again after weeks, it’s still the same. 

[00:12:13] What’s the reason that you see? Is that, let’s say a lack of knowledge about the simplicity or let’s say of the feasibility. Do people think that this is just taking too much effort or what do you think is the reason for that? 

[00:12:32] Mark Kilens: As many reasons really dependent, right? One is a skill set thing. One is a focus thing. One is a cross team collaboration thing. 

[00:12:40] Meaning, if you want to test new messaging on the homepage, that’s probably going to come from product marketing, but more than likely either the demand gen team, the creative team owns the homepage.

[00:12:51] Like who owns the homepage at the end of the day? I would argue like the CMO owns the homepage. So she or he should be the one trying to coordinate the effort of what to test, what not to test, but at the day it’s a focus thing.

[00:13:01] I think the hardest thing in marketing these days is focus. I guess you could do so much at any time. You know with sales, it’s kind of like, no, I need to create pipeline, be in control of my book of business and I need to close this business, this potential business into revenue. 

[00:13:19] Yes, there’s nuance there, of course, with focus but with marketing it’s like, I can do this job. I can do that. I could create this software. I could do that. I could do this. I could change this position. I could do all these things.

[00:13:32] Jan Marks: Yeah. One thing that you decided to not too long ago was the redesign of the website which is always, I’m not talking about, let’s say, some minor things. 

[00:13:44] It was not a full redesign, I think but a substantial redesign that you said. So how did you approach that? Did you run plenty of experiments way before that and tried out different concepts and then you delivered it like a package. How was your approach to redesign? 

[00:14:03] Mark Kilens: Good question. I think when you’re looking to do any type of like major website redesign, it’s one of the best times to do testing, to do experimentation. 

[00:14:12] Because you will hopefully reduce the cycles of having to either make the same mistake or redesign, re engineer a page, a flow whatever down the road, right?

[00:14:24] So you’re proving out your hypotheses during the time in which you are redesigned. So for us, to your point, one place where we did a lot of testing is actually on landing pages that are getting visits. 

[00:14:39] We’re driving traffic from these landing pages from paid search and social because there’s something to be said about the page they land on and how you can control the audience. 

[00:14:49] So that’s one way I think about experimentation. It’s not just like what you’re testing and why you’re testing it but who are you testing it with? 

[00:14:58] Because at the end of the day, like Airbnb sells to B2B marketers, event leaders and HR leaders. Those are three different personas, three different buying personas. 

[00:15:06] So if we’re doing a test for one of them, it doesn’t mean that test is going to work for the other ones, especially when you think about messaging or offers or stuff like that. 

[00:15:14] So I’m sure others have shared today or you have great resources and VWO on how to structure your experiments but I think one of the biggest things we focus on, that I focus on the past is always like, who are you trying to test the hypothesis against? 

[00:15:30] And make sure you really understand that and then which experiment is best to test that it gets right. So for us, it’s a lot of message testing.

[00:15:39] It’s both scientific in terms of, what is getting more opens clicks. I think email is actually a great channel to do experimentation as well. I’m sure you folks have talked about that. 

[00:15:47] Email is also a great channel to get a direct click. So you can control again, the audience a bit, email to click to landing page or whatever page you’re sending to the website.

[00:15:56] So yeah, it’s more about the holistic experience and then, I think a lot of experimentation what I’ve seen also tends to sometimes fall off at marketing. 

[00:16:05] The best experimentation is when marketing is also letting sales know. It could be one sales person or it could be a larger sales team.

[00:16:12] Hey, we’re running this test. It’s more middle of the funnel. This is what we’re testing. You’re going to get some leads or some really high intent hand raising leads from this as a heads up. 

[00:16:22] Here’s how you should maybe be thinking about treating them differently because of this experiment. So it’s like that collaboration with sales is also super important and I also just don’t see a lot of times that happens because again, a lot of people think experimentation is just like inside this demand gen function inside just maybe someone doing CRO when it’s much bigger than that. 

[00:16:39] Jan Marks: Yeah. Yeah. Well, Sarah just said it. It is definitely a team sport and you get the most effective conversion rate optimization program by playing in a team. 

[00:16:51] I am always surprised when we come to a situation when we talk to people who are considering to launch CRO and they are about to redesign. 

[00:17:00] I often hear, yeah, it’s a good idea. We would love to do this but we first have to redesign our page and I’m freaking out that I am really heading into redesign project and then afterwards they wanna start polishing and correcting and so on.

[00:17:15] But anyway, you have always had experimentation of like a pillar in your in your roles that you experimented and never took it as is but continuously evolved. And what is it doing? 

[00:17:32] We notice sometimes the difference between experimenting on the front end, let’s say on the public area of a website and then behind log in, so to say, which often doesn’t get so much experimental attention because it’s of course a smaller part but is this still to come?

[00:18:01] Is this something that’s going to happen in the future that we see more private areas and account areas, for instance, in your case, the learning experience. Is experimentation part of your product development as well? 

[00:18:14] Mark Kilens: Oh yeah, I think that’s the answer. That’s pretty obvious. Yes, I definitely am not a product expert by any means for marketing.

[00:18:24] Yes, but product management, no chance. To me, it’s like the balance of scientific, like very quantifiable metrics, basic experimentation versus the more qualitative experimentation that you can also do.

[00:18:37] Yeah, I think most people listening is probably focused on more of the quantitative and I love the quantitative stuff as well.

[00:18:43] But to me, it’s all about what are you trying to change? Are you trying to change a leading indicator? Are you trying to change a lagging indicator or both? I don’t know, how often does that come up in VWO? Because to me those are very different things.

[00:19:03] Like with HubSpot Academy, yeah, we were trying to change some like leading indicators around like testing of what content or topics resonate the most of the audience, what format is going to be the best to train people, engage them with content etc, etc. 

[00:19:18] Again, a little bit less quantitative, more qualitative to some degree but really the lagging indicator that we were trying to understand and influence and measure is retention and product adoption and run tests and experiments to say, 

[00:19:31] Hey, if we gave someone a three week experience of how to use this product better, will this actually affect the adoption of said product with all else being equal, right? 

[00:19:44] So I think you have to really ask yourself again, like what metric or metrics are you trying to influence? Because it’s the classic thing, every action has a reaction, right? So what’s the downstream effect? 

[00:20:01] Jan Marks: Yeah. 

[00:20:03] You mentioned initially a couple of minutes ago the interactivity in our audience and so on. I’m a little bit disappointed. I don’t see a single question here.

[00:20:10] So I would like to encourage everybody here. 

[00:20:13] Mark Kilens: Yeah.

[00:20:13] Jan Marks: There’s not a single question here. So here’s what I’m going to suggest here. The first interesting question, right? The first interesting question, I promise I’m going to invite you for a pizza in your home office, right? 

[00:20:25] So you come up with an interesting question. You’re going to leave your details afterwards to us and we’re going to ship a really rich, tasty pizza there and I mean it. 

[00:20:34] We did this before and let’s see if hungry people are in the audience who might feel in the mood of coming up but it needs to be an interesting question.

[00:20:45] You and I have to decide if it’s worth a pizza. Okay, so we’re gonna do that. 

[00:20:51] Mark Kilens: It could be like how to make a snowmaking machine at home. I’ll give you the answer to that. 

[00:20:56] Jan Marks: Okay. Well, that’s probably even more interesting than the pizza could be. Mark, you are now at a company that is dealing with, compared to any other product, a very new product, you could say, meeting remotely.

[00:21:13] It all exists in Skype, taught us the first steps and so on but now as a mass, mass product, it is just happening now, which means that there’s probably experimentation beyond the digital channel happening, there is experimentation with completely new product features and so on and so forth.

[00:21:33] So, how does that work at Airmeet? So, is there, let’s say, a regular idea generation mechanism and then you throw it at some part of your audience and try it out and use a group. 

[00:21:47] So how does experimentation on the product development side actually look like? 

[00:21:55] Mark Kilens: It’s a great question.

[00:21:56] It’s a question that I really can’t answer because I don’t really do too much like traditional product development. 

[00:22:01] What I will say is I’m very much involved in understanding the market, right? As a CMO, like you really need to be the person to help guide the market. Latina CMO of six cents really coined the term of the chief market officer. 

[00:22:15] Not the chief marketing officer. It’s the chief market officer and I really love that and from my point of view, it’s like, I’m trying to unite all of the other executives around like a go to market, right? 

[00:22:29] That’s going to successfully make the business revenue and profit in the long term, right? I can’t just generate revenue. It’s got to create economic endurance and viability for the business. 

[00:22:42] So it starts really with working with the product leader of a cost and saying, Hey, we have this hypothesis around something our customers needs. 

[00:22:50] How quickly can we validate that with customer proof? And that’s why I like when you think about digital experiments, not traditional like digital type experiment on your home page or somewhere else. 

[00:23:00] The reason those are great is because you’re like directly going to the customer so fast, right? So when you think about product development, I always say and I learned this from my time at HubSpot.

[00:23:09] I learned this from David Cancel, who is the chief product officer of HubSpot. He was the CEO of Drift for a while, co founder of Drift. 

[00:23:18] The closer you are to your customers, the better it’s always going to be. So even before you maybe even launch an experiment, you might want to just quickly ask three, four or five people, a customer, a prospect, whoever it might be, again, goes back to the, who you’re trying to run this experiment for to validate that?

[00:23:35] Is this even a good experiment, right? Like you might have in your mind, some of them are obvious, right? Some experience are obvious but some, in your mind that might not be that obvious, right? 

[00:23:43] And when it comes to building a product or new messaging, pricing and packaging, great thing to experiment. We just did this and content. 

[00:23:52] Maybe that’s some of the stuff that’s coming out tomorrow. Very new pricing and packaging might be the best in the industry. Don’t want to spoil too much but the point is you have to think about again, who would I test this with? 

[00:24:08] When you go back to pricing and packaging, great thing to experiment with. How do you contain that? I think that’s where again, a relationship to Sarah’s point on the marketing team being involved. 

[00:24:17] It’s not just the marketing team, the entire company from CS to sales to product, we were all in it together. 

[00:24:25] When we were testing this new pricing and packaging out, was it like a true scientific experiment? 

[00:24:31] Not quite. It was more qualitative, I will say that but nonetheless, I still think it’s an experiment. But then like when the new page launches tomorrow. Will we run experiments on that? Heck yeah. 

[00:24:44] We’re gonna run experiments on that. There’s gonna be a ton of experiments on that new page, another great page to test things out on.

[00:24:50] So yeah, I think it’s just a constant evolution. 

[00:24:54] Jan Marks: Yeah. Yeah. I can see that. I can see that. 

[00:25:00] Vipul Bansal: Seems like a pipizzaer machine offer really worked out. We’ve gotten in several interesting questions from people. Okay. So let me just. Nura, who has sent her question.

[00:25:13] Nura, you’re unmuted. You can now ask your question over the audio directly to Mark.

[00:25:22] Nura: Hi, Mark. Can you hear me? 

[00:25:25] Mark Kilens: Yes. Hello. 

[00:25:26] Nura: Hello. So my question was concerning content. You mentioned that content and marketing are really important for experimentation and I work within a bigger agency that does SEO and marketing and advertising for digital online customers and I always struggle with explaining that content experimentation does not equal SEO optimizations.

[00:25:56] So I was wondering, what your take on that is? 

[00:26:02] Mark Kilens: Yeah, your take is definitely accurate. I don’t think content is not just SEO driven. Content goes back to the who. 

[00:26:16] So content is all about creating some type of reaction feeling within the said consumer of the content. Right? So it’s like, how am I using these assets? 

[00:26:29] And content is such a broad term, just like community. So you can be like, you’d be saying, is this an ad? Is this an ebook? Is this a video? Is this a class? People argue maybe this is even content. 

[00:26:40] But the question really is, first defining like what you’re trying to better understand. So one thing that I definitely learned from my past companies is like the problem statement definition is so so important getting extremely clear and crisp in defining the problem hypothesis. 

[00:27:01] And agreeing to that, right? So if you’re at an agency, I think the most amount of time is the pre production work that you can spend with your clients and this goes for non agencies too, just people at Airmeet, for example. 

[00:27:11] We just went through this. It’s spending more time on the pre production before the creation, before the event, before the big launch and nailing that because it will make what happens when the things do start to get created or happen live so much easier. 

[00:27:30] But for some reason, I find for my past and I feel like I’m getting better at this, though, if you don’t spend enough time really pre experiment, pre creating that piece of content, getting really good at defining the brief, defining what success theoretically should look like or could look like.

[00:27:48] Then like you’re asking yourself, why are we doing this, right? So that might be part of the answer there. It’s like you got to just get a little bit tighter on who are you’re trying to experiment with this piece of content? Why you’re trying to do it? What are you trying to experiment and isolating it down, right? 

[00:28:06] So in my past, we would isolate it down based off of like, would a live type of experience be better or would more people consume pre recorded, right?

[00:28:15] Because at the end of the day, we didn’t know, it was so early on. So we did both. Like, all right, we’ll have a live three part session on this and we’ll have pre recorded content as well.

[00:28:26] So that’s like a leading indicator to a metric. So the metric then was like, Okay depending on who went to the live stuff or the pre recorded and we were trying to isolate these two audiences, right? 

[00:28:36] Because if they cross pollinate, it gets tricky. We were trying to determine again, which had a bigger effect on product adoption.

[00:28:44] Some of this is, it’s never probably perfect, right? The most perfect experiment is when you can fully control and that’s almost impossible. Even today with the internet, you can’t really fully control your experiments because of third party cookies and blockers and ads and all this crap, right?

[00:28:59] So, you do your best but the point is like, we got really specific and you got really clear on what you’re trying to figure out and you write it down and you agree to it and you try not to deviate during the test. 

[00:29:12] Nura: Got it. Thank you. 

[00:29:12] Jan Marks: Yeah, I couldn’t agree more. Mark, what do you think? Is that worth a pizza? I think it’s worth a pizza. It was a good question. I really liked it. I think that is the first pizza to go. 

[00:29:23] Mark Kilens: She gets it. Oh yeah, she gets it. 

[00:29:28] Jan Marks: If there are more questions, I’m ready for more pizza, right? So please do drop me an email to Jan@vwo.com and leave your details and tell me whether it’s quattro formaggi or diavola or whatever it is your preference, preferred type of pizza.

[00:29:45] Okay, there is another question. 

[00:29:47] Vipul Bansal: Yes, I’ll come back to you Jan after I take up the next question from Whitney. Whitney Gilbert. 

[00:29:55] I’ve unmuted you so you can ask your question. 

[00:29:58] Whitney Gilbert: Yeah. Hi. All right, I’ll give you the story. So I ran a user test on some product pages and our product pages start pretty much with customer testimonials and then features and benefits.

[00:30:19] So, when I did this test, several users said that was distracting to them, that they wanted features and benefits first and then any kind of testimonials after that. 

[00:30:34] Which makes sense to me. But then to try and quantify that, I also ran an A/B test to see if by moving those components, would there be more convergence and as terms of form fills for requesting a demo. Now there wasn’t a lot of lift with the A/B test.

[00:31:02] I’m still actually running it right now but it’s been about a month and so there’s no, earth hasn’t moved in any direction. 

[00:31:16] So I’m in a quandary because I’m like, I’m doing what users said they wanted but then leaders are like, we want form fills. So, how would you proceed?

[00:31:36] Jan Marks: Can I go first? Can I go first? Please. Because I’m going to copycat from you. You said it very rightly. The interesting thing is, who are you testing for? 

[00:31:45] So because what you’re just describing that reminds me of a couple of tests, several tests we ran, which had these undecided results and then it turned out there was a completely different reaction between new first arrivers on your website and returning users on your website.

[00:32:02] So for instance, that is a very common situation that people who are the first arrivers on your website, they want to understand the product, benefits and so on and so forth. 

[00:32:12] People who come back, who weren’t quite sure but now they came back, they want reconfirmation, they want security, they want some social proof on that.

[00:32:21] So, typically, different things for different orders. You can speak to different audience. I would recommend, I don’t know if you have such a lovely testing technology like VWO but if you had, then you should try to dive into, let’s say the details of your audio and try to find out by market by first users versus returning users and you will find surprising differences.

[00:32:46] Some people might be reacting very positively and others very negatively and altogether you’re looking at nonconclusive results. This is my take on that. 

[00:32:55] Mark, what do you think?

[00:32:58] Mark Kilens: I like that. Let’s see, have you looked at that? Those two different audiences? That’s a good take. 

[00:33:03] Whitney Gilbert: No, I have not but that is a good suggestion.

[00:33:08] Mark Kilens: Yeah, and I’ll give one more quick one. That’s fantastic. My question was going to be like, what are you actually trying to affect? 

[00:33:17] So it sounds like it’s a demo request. Is that correct? 

[00:33:20] Whitney Gilbert: Yes. 

[00:33:23] Mark Kilens: And you’re trying to increase the amount of demo requests from these product pages.

[00:33:27] Do you use any reverse IP technology in your website to see like, what do you sell to like larger B2B businesses or who do you sell to? 

[00:33:36] Just a sense of that. 

[00:33:37] Whitney Gilbert: Yeah, so we’re a software company, so yes, we sell to B2B. 

[00:33:42] Mark Kilens: Okay. Do you use like reverse IP lookup to see the types of companies etc, etc like who are coming to these pages?

[00:33:50] Whitney Gilbert: No, but that’s another good suggestion. Our whole testing group, very young, so done very few experiments but I see what you’re saying to get a little deeper into knowing really who these users are is what really the gist. 

[00:34:08] Mark Kilens: Who are the users and then where are they coming from? And then one thing we did at my past company, it wasn’t a product page.

[00:34:14] It was pricing page but I would almost think about it’s not product but like with the pricing page, we told sales about any type of engagement. 

[00:34:29] Any type of pricing page visit was a marketing response that was flagged to sales. If we could identify that visitor, right? So yeah, product pages, you probably don’t want to do that because the intent is not as high.

[00:34:42] It’s probably more medium intent versus high intent and especially if it’s a first time visit, right? The intent is but what I’m getting at is, can you get a little more qualitative to see if that is even the right offer on those product pages, like for us, like actually on those product pages, a lot of the times what I found in past testing and we’re doing this testing right now on our website too, if you have this.

[00:35:06] But it’s either like more free assessment or sign up for free trial, which could then down the road lead to higher meeting rates, demo rates because of the way you nurture that, right?

[00:35:18] So I think it kind of depends on a few, it always depends on a few factors. But in this case, I think you might want to also think about testing different offers that could potentially also then lead you to more demos through that offer within itself. 

[00:35:35] Yeah, hopefully that kind of makes sense.

[00:35:39] Whitney Gilbert: Yeah. Yeah, it does. 

[00:35:44] Jan Marks: And it comes with another pizza. That’s also good. So it was worth coming on stage. Thank you for that. Thank you for that. Please leave your, it’s Jan@vwo.com, very easy to remember. 

[00:35:54] Send them over and we’ll be in touch. Okay. People, how are we going time wise?

[00:36:01] Because I think we’re trying to make up the run over that we’ve done before. 

[00:36:07] Vipul Bansal: Yes and I think because this is the live session, I would have really loved this conversation to continue. But I see that Emma is here and before we move on to the two workshops that we have scheduled for the audience, we also have the quiz number two, right?

[00:36:21] So we’ll have to unwillingly just wrap up the session here itself. 

[00:36:27] So yeah. 

[00:36:31] Jan Marks: Thank you Mark. Before leaving, it was really great seeing you. You didn’t tell too much about what’s going to happen tomorrow. So that leaves us with a lot of excitement. So we all know where we’re going to sit.

[00:36:44] We go to Airmeet.com and see what’s going to happen. Right? 

[00:36:48] Mark Kilens: Yeah, just go there in the morning, come to the event. It’s around noon Eastern. But yeah, I dropped one hint. There’s one of the seven things I mentioned, if you pick that up, folks. 

[00:36:59] So, there’s new pricing and packaging, some new plans coming out.

[00:37:02] So yeah, there’s some exciting stuff. 

[00:37:05] Jan Marks: Definitely. I will definitely sign up for the free trial, by the way, it’s a free trial for everyone? 

[00:37:10] Mark Kilens: There’s a free product which is also getting major updates tomorrow, so I give you two. I give you two. I gave you two. 

[00:37:18] Jan Marks: Okay. I’m really looking forward to it. Mark, it was great having you.

[00:37:21] It was a really fun, sorry for the bad light and the shaky monitor here in the train station of Barcelona but I’m really happy that my iPhone connection worked like a charm. 

[00:37:35] Thanks again. Thank you people for having us and over to you.