Follow us and stay on top of everything CRO
Webinar

Two Frameworks You Can Use to Improve the Hypotheses of Your Tests

Duration - 45 minutes

Key Takeaways

  • Use the PXL prioritization model: This model is less subjective than the ICE model and breaks down the factors into more specific metrics. It helps to prioritize insights based on their location on the page, noticeability, data backing, and ease of implementation.
  • Prioritize insights that are above the fold and noticeable within 5 seconds: These insights are more likely to be seen by users and therefore have a higher impact.
  • Back your insights with data: The more research methods that support an insight, the more confident you can be in its validity. This can include user testing, qualitative feedback, digital analytics, heat maps, or eye tracking.
  • Consider the effort of implementation: Insights that take less time to implement should be given higher priority. The PXL model uses a scale to score the effort, with less time-consuming tasks scoring higher.
  • Use the PXL model to sort and prioritize all your insights: This will help you know what to do first in each category, minimizing arguments and getting everyone on the same page. This model can be used across any industry or vertical.

Summary of the session

The webinar, hosted by Shanaz, Marketing Manager at VWO, featured Haley Carpenter, a CX strategist at Speero by CXL. Haley, who has extensive experience in user research and running experimentation programs, shared her insights on practical frameworks to improve test hypotheses. She drew from her experiences working with companies like Vitamix, NextEra Energy, and OLaplex.

The session aimed to help attendees understand how to break down complex problems and identify significant opportunities. The webinar concluded with a Q&A session, allowing attendees to engage directly with Haley and gain further insights.

Webinar Video

Webinar Deck

Top questions asked by the audience

  • Do you use the PXL and research Excel frameworks when managing the interplay of demand generation and CRO/CX?

    - by Devin O'Grady
    Yes. You can use this across I would say almost any application or prioritization. Like I said, it is customizable. So if there's something that doesn't apply to your scenario, your business case, wha ...tever. It is definitely able to be changed, but I would still encourage you to use these frameworks. Absolutely.
  • Do you have an example of a checklist of A/B tests?

    - by Andre
    Not off the top of my head, that I can share. That would be all client information, unfortunately. But I assure you every time I have done, the research Excel framework and cranked out a research pres ...entation for a client I've ended up with so many tests every time I do it, varying from 20 tests to 30 tests to 50. I mean, it can just really go on and on. And especially if you're doing that continuous research, you'll just have a continuous list of tests that you can pull from.
  • When ranking or scoring, how do we make sure that we are not biased towards an idea that we believe will work, but don't necessarily have data to back it?

    How do we prioritize those ideas and test, sort of score them? Yeah. That's a good question. So the PXL template that everyone should be getting access to after today's webinar should be as objective ...as possible. You shouldn't have to worry about bias when you go to fill that out. Because there aren't any questions, like, how do you feel about this example, or what do you think about this test? As I went through the examples, is it above the fold? Can you notice it in 5 seconds? Are you adding or removing anything? All of those are really easy, yes, no objective questions, so you don't have to worry about the bias there. And then say you did not find it in any research, and it's just a heuristic idea that you came up with based on your expertise. It can still go in the framework. But in that confidence section where it asks If you found it in any research, it will just get zeros across the board. Right? So it will score lower because it wasn't any research, and it was perhaps just serious. But the buy shouldn't be in there. You should be set to use the PXL template. Not have to worry about it. And absolutely still put your heuristic ideas in there.
  • I would like to add politics as a column for context, but I'm not sure what score to add how to score in the column of politics when prioritizing ideas.

    - by Dragos
    You're getting into some muddy waters there. You're trying to cheat the system. I know what you're doing. I, you know, I can't lie. We do put a political score column into some of our frameworks, but ...this is opening up the door for some subjectivity and some cheating system. So I get the need for it because I'm not thrilled to admit that we have used a political score column, but sometimes there's really just no way around it. I do understand that politics can get intense and messy, and sometimes you just have to put those in but I would really advise you to keep out a column like that if you can. And I would recommend if you have to put a political call them in there. Maybe just keep it so it has a low weight. So maybe just make the score 0 or 1. Don't crank it up to a 3 and give 3 extra points for that because that's introducing more of that subjectivity. And then you're essentially saying, I can give all of my ideas way more points and jack it up to the top if I think it's a good idea because there's a political need for it. So, hopefully, that answers your question. Like I said, it gets muddy. Try not to add 1, but if you have to, give it a low weight, so I would do, like, a 1. And be very stingy with using it.
  • For working with new clients where do you start? Is there a process where you start analyzing first and second?

    - by Jake Young
    Yeah. Yeah. Excellent question. So if I get a new client for research, Honestly, we just go at it as hard as we can all at once. So they come in and we try and get as much data and set up at the outse ...t as we can. So we try and get the survey set up. We try and get a poll set up. If we're doing any kind of interviews, we try and get those set up. Keep maps from everything in that model, we try and get as much set up as possible so that we can get that data collection done because that's really what takes the longest. But like I said, that is a lot all at one time, and not everyone has resources for that. So you can absolutely attack that molecule. One piece at a time or a couple of pieces at a time. I don't really think it matters. You know, where you start, I think it would be on a case-by-case basis. What makes sense? So pick what you think makes sense to you and what aligns with your priorities, or maybe you have some pressing issues where user testing might make sense to start first. Something like that. It really can be done either way where you tack it all at once or in pieces. And then, you know, after we get the data, then we analyze it, do a readout where you could do that readout all at once or do the readout and pieces if you're attacking the research and pieces. Yeah, and then you just put that all into the PXL and run with it. Our testing program. That's the TLDR.

Transcription

Disclaimer- Please be aware that the content below is computer-generated, so kindly disregard any potential errors or shortcomings.

Shanaz from VWO: Hey, everyone. Welcome to another session of VWO webinars. I’m Shanaz, Marketing Manager at VWO, a full-funnel A/B testing, experimentation, conversion rate, and experience optimization platform. To ...