eCommerce Optimization – Secrets from the Trenches. Real-life Test Examples to Help You Set Your Winning Strategy in 2021

Today, we have with us, Craig Smith. Craig is the founder and CEO of Trinity, one of America’s leading e-commerce optimization agencies. Craig and his team work with e-commerce brands to help them achieve their potential and deliver more rewarding experiences to their customers. Trinity has been a partner with VWO since 2009 and has launched thousands of tests on the platform.

His experience in e-commerce spans a 20-year period, and he has worked with some of the largest brands across the globe in helping them achieve their e-commerce growth. So, thank you, Craig, for doing this with us today.

Craig Smith:

Thanks for having me. I appreciate the time.

Shanaz:

Yeah. So before I pass on the mic to Craig, I’d just like to tell all our attendees that we’ll be taking questions and Q&A’s at the end of the webinar. So feel free to shoot up your questions in the chat box, at any given point in the webinar, then in the end, we’ll be taking up some questions. So with that, Craig, I think the stage is all yours.

Craig:

Alright. Awesome. Awesome. I’ll share my screen now. Thanks, everyone, for joining.

Good morning, good evening, or good afternoon for wherever you’re from. Again, my name is Craig Smith, and I’m the founder of Trinity. So thanks again for joining. What today is gonna be about is, I titled it secrets from the trenches, hacks, and strategies to grow your e-commerce sales. We’re gonna be looking at different approaches for you to leverage VWO to enhance the effectiveness of your e-commerce to grow your sales.

And I’ve outlined a plethora of steps and examples for you to do so. This is gonna help you kinda prioritize which opportunities exist within your website and get an understanding of the proper methodology that you should embrace when you look to use VWO and maximize effectiveness. Real quick about me, but in the industry for a long time, used to work for eBay Enterprise and essentially started Trinity to help brands get more sophisticated in the e-commerce channel. And I’m really looking forward to sharing a lot of data and, again, secrets with you today. My goal for today is that you leave this webinar with a plan that you have exact insights into what you need to do to grow faster.

That you’re gonna be able to take this playbook that we’re gonna go through and put it into practice. And that’s a really critical thing. I mean, this information is great, but you need to take it, embrace it, and put it into practice, and hopefully, you’ll do so. What we’re gonna talk about first and I guess an overview for today is an e-commerce optimization framework. What do you need to optimize correctly?

What are the steps you take? What are the different milestones and things you should be considering? And how does VWO help you in reaching those goals? Also, we talk a lot about this first piece creating customer feedback mechanisms. Analytics and data will tell you what’s happening.

Customer insights will tell you why it’s happening. You need both. You can’t fight this fight with 1 and not the other, and we’ll go through some examples of how you can leverage that information. The second half of the webinar, and probably more interesting to the audience today, is gonna be real-life test data. We’re gonna walk through specific experiments that we’ve run and show you exactly what the impact was, what was the increase to a micro conversion or overall conversion rate?

And in the end, you know, we have a section that’s really all about, like, kind of seeing which test won because there’s a couple that were frankly very surprising. So let’s get started. Starting first, again, let’s go through the framework. When you look at e-commerce optimization, you wanna start with the right foundation. Again, research and planning, make sure you have the right data and the right customer insight to understand what’s working and what’s not working and why.

Then after you get that information in intelligence, you leverage a testing system, like VWO, to take a data-driven approach to modifying your user experience and embracing experimentation in your culture, And this is ultimately gonna deliver gains to your return ad spend metrics, your marketing, as well as your customer lifetime value, as customers have better experiences with your site and brand and app. When you look at this life cycle, a couple of key things to consider when you’re planning out your methodology. One is that your data is correct. 2, that you have a heuristic review of all the gaps that may exist in your UI and in your user experience. And 3, it’s prioritizing the experiments and the improvements you wanna make and modeling out the economic impact that’s gonna correspond.

Let’s go into these a little bit deeper. So the first thing you wanna do within any optimization effort is validate and verify that you have accurate data. I’m guessing 90% of you guys are on Google Analytics. Phenomenal tool. We love it.

Use it every single day. But there’s gonna be nuances and tracking gaps that likely exist. You can’t fix what you don’t know is the exact truth. The same way you go to a doctor’s office and every time you walk in, and you get your pulse measured, you wanna do the same thing on your website and make sure it’s absolutely rock solid, accurate. And working with brands, each and every day, we review Google Analytics installations with gaps.

Maybe the site search reporting isn’t working right. Maybe they’re double-counting checkout visits. Maybe the mobile traffic’s not firing off or the scripts are not firing off correctly. What you wanna do is make sure that either internally or externally, you monitor or, excuse me, you evaluate the validity and integrity of your analytical installation to make sure that this is the first step and that your optimization efforts are based on truth. After that takes place, we’re actually let me just stop for a second.

I wanna show this slide. This is telling you why it’s so important. What you’re looking at here is a measurement effort for what we call progression rate analysis. We’re looking at the top, a desktop percentage of add-to-cart. What is the ratio of people who view a product page and add it to the cart?

You can see 14.44 percent, and on mobile, that number is 11.58%. The reason the analytical installation is so paramount and you have to get it right is when you model out the impact saying, okay, if I take our desktop Add to cart percentage from 14.44 to 15.88, a nominal increase. What is that gonna mean to incremental revenue off the baseline in this channel? You can see this is an exercise that we do with customers and prospects alike. And it’s really correlated to understanding where is the squeaky parts of the wheel.

Where is the most meat on your bone? Is it your mobile experience, is it your desktop experience, and which section of the site does that exist in? But again, none of this type of review can be constructed without having accurate Google Analytics information. You also wanna look at the way the experience is laid out and score it across your industry. You wanna look at your brand presentation, your funnel flow, your trust and security components, and your mobile experience, and score it. Score it internally with factors that you can gather through your own research or hire a third party but it’s very important you get a heuristic review of what elements in the site are representing opportunity. Do this against the competition. Look at your competitors, and see how they’re performing in areas like your navigation, product pages, home pages, etcetera.

So you get a pulse and an understanding of what alternative tactics might be out there for you to utilize. You also wanna use scroll maps. This is gonna be a research tool to help you understand the depth of people as they look at your page, how deep do they scroll down? We use this type of tool extensively to understand where and how we order website modules that really matter for the business. If you don’t have insight into how deep people scroll down, many times, some of your core promotions and core actions are never gonna be seen by the majority of your customers.

You also wanna leverage heatmaps in this research phase where you’re gaining intelligence into your customer behavior. On all of your key templates, your homepage, your product page, your search results, your category page, your subcategory pages, you wanna look at your heat maps and identify where is the click distribution going. Where are users taking the actions that I expect and users not taking the actions I expect? You can see in this case with a customer of Trinity, that we’re seeing a lot of activity right here in the merchandising grid, including the bread crumbs and this is very insightful. Why are users clicking on these breadcrumbs to a degree that maybe we didn’t expect them to?

And you can see the majority are clicking this first product in the carousel. Great intelligence you can derive to help in constructing your strategy. You also wanna use usability testing as a way to document user behavior. In the old days, you had to create your own tests, develop a discussion guide, go out and recruit 10 to 30 panelists, and deliver it in a usability lab or within a retail store. Those days have ended.

You now can go online to a variety of third-party vendors, and provide essentially an online discussion guide that gives people tasks that you want them to go through. And then ultimately going back and watching the tasks, listening to the feedback, and seeing the mouse movements of the user, again, you’re gonna get great information to understand what you need to change within these key pages that you’ve already identified with the data are underperforming. In this case, you can see we asked the user or the test panelists to use the searcher navigation to find a square gas-burning fire table that is 36 to 42 inches in width. And when you are looking to buy this product, what type of information do you expect to find? So you’re asking to do an action and you’re asking for written feedback.

Again, a great lever to pull to get further intelligence into what you should change within your store. Now that you’ve done all of this assessment work, you have looked at the analytics. You know, it’s right. You know, the progression modeling.

You’ve done your customer research. You validated some hypotheses around certain things you think are wrong and right within the store. You now wanna score it and prioritize the impact. At Trinity, we’re looking at that. Here is a rating score that we utilize. We call it a pie rating and it helps us to decipher what areas in the store represent the greatest opportunity. You can see we’re leveraging metrics like paid traffic and revenue per session to help calculate it.

It doesn’t have to be this complex on your side, but you need to definitely prioritize your testing pipeline. Because at that point, you’re then gonna move to the designing and building and running and analyzing portion of your methodology. And this is where it’s exciting. Right? This is where you see the economic benefits.

This is where VWO can light up and do absolutely amazing things for your experience. It’s not just button color sizes and slight changes on pages. You can use VWO and completely re-engineer any page in your template as long as that script fires off using custom JS and custom CSS, etcetera. So now you’re in the testing set, the testing spot of the presentation where you’re actually running, you know, experiments. You wanna start out with a hypothesis and make sure that every experiment that you run has a hypothesis and that you’re targeting you’re outlining who your target audience is, the estimated duration, and a detailed list of what’s changing on the site. And, of course, the concepts, the visual changes that you’re gonna be making with an experiment.

From there, you wanna construct the designs you’re going to deploy. In this case, you can see, you know, We have an example of a troll versus a variation, you wanna be mapping out what you’re gonna be changing in your user experience. And then from that point, you develop and build in VWO, we use JavaScript and CSS to manipulate the structure of the appearance and the functionality without the user ever noticing, and it’s a great way to, you know, again, manipulate the page, and it’s not just slight changes. Ultimately, at this point, you also wanna go through a quality assurance process, which is really looking at multiple operating systems and browsers and device combinations. To ensure that your test is bulletproof to all your users.

We go through a rigorous checklist to make sure that people that it’s gonna fire off perfectly on every device. And after a test launch happens, you wanna make sure to look at it daily. I mean, we construct, as you can see here, a Google Data Studio report, which essentially you know, provides daily insights into how the control is going against the variation. But, you know, you wanna make sure that each day you’re looking at this and, ultimately, you reach physical validity in VWO. And at that point, when you get validity and the data is either significant for the variation or significant for the control, you wanna stop the test.

At this point, usually, we divert 100% of the traffic to the winning variation. You can host the changes in VWO, but at some point, it’s best practice to integrate the actual change into the site. So we would recommend after you get a winning variation, you integrate it into the site. One other thing is we’re talking about e-commerce optimization to consider, because it highly correlates to VWO because, you know, VWO is all about improving the experience. You wanna look at all of these touch points of automation that you can also leverage to work in unison with VWO.

You can kind of see this is like a playbook that we embrace with our customers, and you really need each and every one of these steps or this automation to fire off. And the cool thing is you can sync it with VWO. So if we know someone had a cart abandonment email that came in from, you know, an email system, and it came back to the site. We could tie it in the VWO. So we’re presenting a custom homepage that’s maybe spotlighting that product in a very creative way.

This is a type of user personalization that you wanna embrace as you look to enhance the sophistication of your store. Now let’s talk about some must-use strategies as we’re about halfway through the webinar. The first strategy that I want you to embrace from an e-commerce optimization standpoint is urgency messaging. With urgency messaging, and we do these tests all the time, it’s bringing in stock data into the product page so you can effectively give someone that little push that they need to convert. In this example that I’m showing with this apparel retailer that we did this test with using VWO, we were able to get a 12% increase in conversion rate by tying in their inventory data and injecting it into the product page when it hit a certain threshold.

So when it was below 10 in stock, we would then fire off this module with VWO, and you can see, again, a 12% increase in conversion rate, which was a very, very big win. The second must-use strategy that I would like to discuss is eliminating distractions and improving your usability. Improving usability on the mobile experience is vital. In this experiment we ran for the largest lingerie brand in South America. You can see that when we got into the data, the mobile experience really was the laggard.

That’s where they were not getting good progression. What we did was use VWO to reconstruct how the mobile add-to-cart process was firing off. And then by doing so, we got a 7% overall conversion increase by reengineering this process and making it much, much easier. The last must-do strategy that I want you to talk about real quick is increasing your conversion rate by simplifying your checkout. I would also include your cart in here as well, potentially. Too often, I see brands in both B2B and B2C that are not presenting simplicity.

What happens is they’re causing what we call analysis paralysis where people aren’t sure where to click on or they get distracted. Now, this was the existing website when they came to us, and we modified the checkout process to get rid of all the utility links, get rid of the search bar, make the phone number big and bold, make the live chat big and bold, try to make an Amazon buy box over here and experiment this with VWO, you know, by simplifying the checkout, drive a 7 figure impact to overall profit in one year. And now, those are the three things again that I think are important as far as must-use strategies, you know, reducing or, excuse me, building urgency, reducing, or making usability more prominent in your mobile experience, and then analysis paralysis, getting registrations. Now let’s take some of those insights and go through some tests and play this kind of game which test won, where we’re gonna show you some variations and and show you the data.

The first one is this category page for a retailer in the computer and technology space. You can see their existing version. Version A was this navigation with all of these radio buttons. You know, in version B using VWO, we were able to consolidate those radio buttons into a drop-down menu, which now made the products higher up above the fold and gave the user a more streamlined approach versus being overwhelmed with all of the radio buttons. You can see that the winning variation between the one, and this one was the second one.

And it was interesting. That page got a 30% increase in category-to-product progression by using that sticky header capability that’s firing off with VWO. Now this needed custom JavaScript and custom cascading stylesheets to be rendered like that, but something you can definitely do with VWO. The next one is a mobile product listing page. And the difference between version A and version B is a single card.

This is a single card versus a grid view. So the the hypothesis is it is better to show more product options versus just a single option on that primary mobile view pane. You can see that version B was a winner in this test. Again, the grid version got a 10.5% increase in overall conversion.

It essentially made it easier for users to browse and find products. So a big win in overall conversion just from that one experiment. The next one is e-commerce, but, a different type of e-commerce. This is travel e-commerce, and this is one of our customers that has been with us for many years. You can see they were redoing their comparison pages for their packages. Version A right here is this tab-based layout where you get presented your packages by all other luau and VIP badges in kind of this grid format.

The alternative version that we designed was this version. Where we’re simplifying the top structure of all versus luau and just kinda putting the VIP badges over some of them. And just, again, completely re-engineering the experience. This is what VWO enables. You can see this is not a button color change.

This is not a slight change. This is the complete gutting of the page and making it fire off in a different fashion. You can see that from the data this version outperformed the the other version by 20%, a nearly 20% increase in overall progression rate. Essentially, you know, requiring less scrolling, it definitely worked better. This next test is all about the checkout process and the checkout summary.

We ran this test to see if we could hit upon right here, some of the key questions that users have in e-commerce. So I always you know, look at this when we’re looking at checkout and funnel. Are we talking about security? Are we talking about guarantees? Are we talking about shipping expectations and timing?

You know, that’s really important. And if you don’t address those key considerations, you’re gonna miss an opportunity as it relates to how your folks will flow through your checkout process. In this case, you know, we ran here was the first one. Sorry. Here’s the first one.

And here was the second one. You can see we incorporated this order summary, a graphic above the fold that was dynamic. Based on the date we made some slight changes. You can see, not crazy changes, but just some slight changes and that was able to drive an 8% increase in the overall conversion. So essentially, what we did was alter the presentation of the shipping expectations and delivery date.

So users gain that insight into the logistical questions, reducing them to abandon the checkout from confusion to a nice winner there. The next one was in B2B commerce, but in this piece, I want to again, show aesthetics and reduce the form to see how that can be impactful. You know, there is a kind of a golden rule that the fewer fields you have in a form, the better your conversion rate is gonna be in that form, and it pretty much always holds true. That’s why when we’re working with customers, we like to look into great form analytics. So you can kinda see the waterfall of how people progress from their name to the email, to the company name, to the phone number, to whatever other data you’re requesting, and you can really see where people really fall off.

Knowing that and embracing that type of dynamic we did this experiment for a a large B2B company where we re-engineered our quick quote process consolidating the form or consolidating the fields, I should say. This was a very, very big win as we got a 28% increase in form submissions. By incorporating fewer fields, including a global information bar that, you know, better hit the value proposition. And also giving that free price analysis opportunity.

So, maybe not pure play e-commerce, but showing the impact of how form consolidation and fewer fields can make a huge impact on your conversion rate. The next one is all about website personalization. You know, we have some customers in the space where they’re looking to interact with the user to get them to put information in that dictates or creates a custom element of a product. This client is in the customized books category, as you can see. The original page was this page.

Where there was an overlay module asking for first name, middle name, last name, and gender to then progress someone to the next step. We saw significant leakage here. We re-engineered the page to render like this where we presented a more vertical modular-based widget to get all that information and use these drop-downs for some ancillary pieces, also including the keepsake gift box, etc. By consolidating into this process, we were able to get a 21% increase in overall conversion just finding that users liked the page layout. It kind of is similar to Amazon where everything’s in the right and that type of buy box feel.

So a good takeaway for you would be to look at your product pages. Do you have that type of buy box field that is replicated in Amazon and what can you change within your current UI on that page to make it more like it? This next one was very interesting. So here I wanna say version A is the one that was our variation where we did the $8 to free shipping. We did a custom module in the cart that gave visual representation of when someone would get free shipping. And we just used some custom code to kinda say, okay, they’re free, shipping’s at 50 bucks. You’re only 8 dollars away. You know, that was the variation. The control was this. Extremely surprised to know that or to see that the clear messaging with the free shipping had absolutely no impact.

This control and variation were essentially flat. So this is a lesson in e-commerce optimization that’s really saying, you never know. Even when you think a version’s gonna, of course, outperform the control. It doesn’t always do it. And because of that, you don’t wanna just implement and factor in changes that are gonna be incredibly exhausting your development resources and your time until you test it because we definitely say it’s an engineering time of the customer.

In this case, because now they didn’t, they knew they didn’t wanna fully integrate that type of change. The next one is a checkout page. And then this, I think, is a very good example of VWO’s capability to manipulate checkout effectively You can see here’s a shipping address, a billing address, and the cart contents. This was the existing page. Using VWO, we’re able to create this.

A much more streamlined page, including a nice buy box on the right-hand side, making it just the back these were no longer filled in gray. Just cleaning it up. Right? This is all about aesthetics. And just making it easier from a usability perspective.

This specific test drove a 12% increase in conversion. Redesigning it, following best practices, and making it more intuitive across all devices, was a great, great win for that brand. So some final takeaways before we open up to Q and A. The first thing is to fix your data. Make sure that you know you have data integrity within your analytical system so you can accurately measure what things are working well, and what things are not working well within your e-commerce experience.

2nd, make sure that you prioritize and do not neglect customer insights. Take surveys. Do the heat maps. Do form analytics. Video recordings.

It’s never been easier to get primary research feedback. That’s gonna tell you again the why behind the what. As you get to your tests, first look at where you can reduce analysis paralysis. Where are you overwhelming the customer, the web visitor? Make them think less, make it easier. And then lastly, It’s almost impossible to conduct an experimentation program unless you’re measuring first in a micro conversion.

When you’re running an experiment on a product page, look at that specific page. What is the percentage of people who added to the cart? Always look at the specific template metric KPI first. Don’t look at the macro conversion. You can afterward get validity.

But always look at the page level metric, the add-to-cart percentage, the people who go from cart to checkout, and the people who go to your subcategory, that is the KPI you wanna measure in an experiment. And make sure you start there, get the insight there because then that’s gonna have a compound effect on the overall conversion rate. 

So thank you so much for the time today. For a copy of this presentation, my email address is right there. It’s csmith@trinity.one Also, if you’re interested, I’d be happy to spend time with you on a phone call for a free assessment, really kinda going through your site and deciphering how we can use VWO to up your e-commerce game and what things could change, and we’d be happy to get an NDA in place.

Go into Google Analytics. Run some economic modeling in your templates to really kind of decipher that. And then lastly, if you go to our website right now at Trinity.1, in my calendar actually, there’s a banner there. You can click on it and book a time directly from my calendar link on Trinity.1. You shouldn’t have a hard time finding it.

Shanaz:

Yeah. So, we have a few questions from the attendees. The first one is from Nick. Nick is asking when you have created a new version of a page and changed lots of variables and when the new version doesn’t work, how do you figure out which variables are the key ones to change?

Craig::

That’s a great question. So let’s say we ran an experiment on a mobile product page, and it did not perform to what we expected. Frankly, without primary research, you really can’t tell exactly what it was. That’s gonna be, like, do you and we’ll have internal discussions. Like, do we wanna go the drawing board way and create an entirely new concept, or do we wanna build out additional functionality on the concept that did not beat the variation?

It’s very difficult to tell for sure. We can look at analytics and see, okay, did bounce rate get affected? Were there you know, pages per visit that was enhanced from this variation, but to get that exact functionality changed or to find out its impact on the overall test, it’s pretty tough to decipher without doing primary research.

Shanaz:

Yep. Thank you, Nick, for the question. I hope that answers your question. Next question is from Brett Croquet, I hope I’m pronouncing your name correctly. Brett is asking, how many tests do you recommend running at a time?

And how many tests in a given time period? Is there some sort of equation for how often to test based on traffic?

Craig:

Yeah. So you don’t wanna test too many aspects of your transactional funnel at the same time because you can kinda muddy the water. I like to do 2 to 3 tests is kinda like the sweet spot. Now when you’re a really large organization where you have a website with an architecture that’s extremely deep and it’s all these different sessions. That’s one thing.

But an e-commerce store where you have that category sub cap product cart checkout. You know, I would say 2, maybe 3. I like to maybe do, like, a homepage personalization treatment, like new visitors, returning visitors, customers, geo-targeting, that in parallel with maybe a category page test where you’re trying to get more people to the product page and then maybe a cart test training more people to check out. Try to separate them into guardrails. Right?

Like, don’t have them overlap. So sometimes if you’re doing a cart page and a checkout page test at the same time, the water can get a little bit, you know, muddied with understanding what’s really working best.

Shanaz:

We have, a lot of questions. I’m just going to pick, a few of them. So we have one from Sorentana the question is about one of the examples you shared. Soren asks I’m curious about what happened with RPV and AOV in the checkout tests with the free shipping bar.

Craig:

That’s a great question. I don’t have that data point right here. But I would be shocked if a test didn’t win. I mean, that’s why one would include to kind of give an example for when you expect something that does not come to fruition. But I do not have, at this point, you know, the exact AOB metrics associated with that experiment.

Shanaz:

Yeah. Soren, I think you can reach out to Craig, on his email or on LinkedIn and maybe there you can have that conversation with him, and he can help you out there. The next question is from Eugene Co. How do you utilize the funnel feature to optimize your site in VWO?

Craig:

The funnel feature in VWO. So we use VWO as primarily the delivery mechanism. A lot of our reporting that you saw in the well, you might not have seen the beginning of the presentation. It’s geared around Google Data Studio. So we’re taking information from VWO and essentially using it to connect to Google Data Studio.

That way we can, like, slice it and dice it and visualize it better. I’m not ultimately in VWO day in and day out all our engineers and my analysts are. So I can’t speak to the specifics of that functionality. But we usually use from our visualization standpoint, Google Data Studio which plays really, really nice with VWO. If you haven’t seen Google Data Studio, it’s a free product.

It just allows you to kind of, again, use multiple sources. So if I wanted to say in my VWO test, wanna look at the micro conversion rate, the macro conversion rate, the balance rate, and the pages per session. I could do all of that kinda get more of a global feel of what the impact was versus just one metric. That’s one of the reasons we use it. But, I can’t speak specifically about the final functionality, but maybe you can share.

Shanaz:

Yeah, we’re running a little short on time. So maybe I’ll get back to you, Brett, on your question on a solution.

Craig:

Many questions we can fire off, rapid-fire, see them all come in, actually.

Shanaz:

So I’ll take 3 more questions. I’ll start with, a follow-up from Brett actually. Greg is asking, can you talk to us about statistical significance and what role that should play in selecting a winner?

Craig:

Yeah. Sure. Of course. So statistical significance is when the math tells you that the variation is conclusive. You never wanna stop an experiment until you get that green light, or maybe red light.

95% is usually the percentage we look for. So when our analysts see 95% statistical significance in an experiment, that’s when we stop/pause the test. Go to the customer. Let them know. And then at that point, we usually flip the switch to 100% of VWO while we hard code it into the active. So 95% is really the number you wanna look for.

Shanaz:

Thanks, Greg. We’ll move on to the next question real quick. So Gustavo, I hope I’m pronouncing your name correctly is asking, is thanking you for the presentation, and, he says he works with Google Optimize and is currently having issues on validating results. Even with large sample sizes, how does VWO select a winner version? And what do you do when it takes too long to find a winner?

Craig:

Yeah. So we don’t use Google Optimize. We’ve obviously found VWO as a superior product you know, it’s got much more bells and whistles. It’s much more stable. You always wanna make sure before you start an experiment that you have enough traffic to test.

If you’re not getting 2000 sessions on this test, you’re probably gonna sit and take too long. It’s gonna be more frustrating. So use, like, at least 1 to 2000 sessions for a standard split test, but, something to consider is people come to me and like, oh, we wanna do a multi-variate test on this page over the page. Let’s get a lot of traffic. You know, multi-variate test with let’s say four aspects of the site with 4 variations a piece is 4 times 4 times 4 times 4.

It’s gonna be, you know, a ton of variations. You will wanna do simple split tests on light traffic pages. That’s the takeaway. An A/B test at max, an ABC test, but hopefully that answered the question about the traffic. And what was the other, Shanaz, the other element to that question?

I apologize.

Shanaz:

So Gustavo asked, what do you do when it takes too long to find a winner?

Craig:

Too long. There’s not much you can do. You know, you kinda have to wait it out. That’s why that 1 to 2000 sessions threshold is really kind of the sweet spot.

Shanaz:

Yep. I hope that answers your question, Gustavo. So moving on to the last question, and it’s actually a very interesting question. If you could run just one test to improve the cart rate, what would that be?

Craig:

Great question. If I could do one test for add-to-cart rate, I would try that inventory test threshold. Because it’s worked time and time again. I mean, literally, I had another 2 examples that my analyst gave me, but I didn’t include it because it’s the same experiment. Didn’t wanna bore you guys.

But, basically, what you’re gonna be doing is if you’re on Shopify, if you’re on BigCommerce, if you’re on Magento, there is a data piece that you can grab. It says how much inventory is in the platform for that SKU. And then use custom code that says, okay. When this drops below 10, when this drops below 7, whatever that threshold is, tell the user only 7 are in stock order now. You know, that has this work time and time again.

It’s not a very, you know, tough experiment to deploy, and that’s a real quick and easy one. There are a lot of other ones as well. I mean, making reviews higher up is critical, hitting upon shipping expectations is critical, making sure that you talk about guarantees and returns is critical. The product page does the selling. So all those pieces need to be very tastefully integrated in that page so you’re gonna speak to that user in that type of capacity.

Shanaz:

Yep. Thanks so much, Craig, for answering all these questions. Once again, I would like to apologize for the presentation glitch that happened. And I think with that last question, we can close this webinar. I’ll be emailing all the attendees, the presentation deck, and the presentation recording, in the next 24 to 48 hours. So, yeah, thank you for attending everyone, and thank you, Craig, for doing this with us.

Craig:

Thanks so much, everybody. Have a great rest of the day. Hope to connect with you. Take care. Bye bye.