Editor’s Note: This transcript was created using AI transcription and formatting tools. While we’ve reviewed it for accuracy, some errors may remain. If anything seems unclear, do refer to the episode.
Trailer
Chadielle Fayad: For example, in instances where larger redesigns are being rolled out or business decisions are executed, experimentation is sometimes brought in too late.
I once ran a user test with the goal of understanding frustration points for users on a page. I had to devise my questions carefully to make sure I didn’t lead subjects — it’s important to let people think for themselves to the fullest extent.
I do use a prioritization framework to rank ideas, based on effort versus reward. It considers aspects such as build time, traffic size, baseline conversion, and the expected lift in conversion.
I’m sure most CRO specialists will tell you that emailing a final report is never enough. It’s important to present results in a meeting format with the key stakeholders to ensure they have a chance to really grasp the insights and ask questions — you want people to interrogate and ask questions, so you know they’re taking the results seriously.
It’s important to work with teams dedicated to retention, engagement, and acquisition for the products in your remit — just keeping them informed really makes buy-in a lot easier.
About VWO
Welcome to another episode of the CRO Wizards series by the VWO podcast. In this series, we speak to top CRO leaders in e-commerce, media, subscription, retail, banking, and other industries about CRO strategies and the positive impact they can have on your business. Before we speak to our special guest for this episode, here’s a quick summary of who we are and what we do. VWO is a leading experience optimization platform. Using our latest product, VWO Insights, you can analyze user journeys and identify conversion roadblocks on your website and mobile app.
So, without any further delay, let’s jump right into the conversation.
Introduction
Jinal Shah: Today, we are honored to have Chadielle Fayad, Experimentation Specialist at Nine. Chadielle plays a pivotal role in transforming user experiences and driving higher conversion rates, where data analysis, hypothesis development, A/B testing, user research, website optimization, UX/UI design, and content strategy come together.
Drawing from her extensive experience at Foxtel Group, Chadielle has honed her skills in executing CRO strategies and optimizing experiences across web and app.
Beyond CRO, she has a rich history in journalism and digital content production, having worked with top Australian media organizations like Pacific Magazines.
She also co-founded J&C Film, a venture providing video and photography services.
How’s it going, Chadielle? It’s very good, thank you for the warm welcome.
Jinal Shah: So what were your initial thoughts when you received the invitation to be on the podcast?
Chadielle Fayad: Definitely very flattered, and a little bit nervous at first, but generally very flattered and very excited to have a chat with you and see what we can talk about together.
Jinal Shah: If we start from your learnings — is there a recent project you’re particularly proud of?
Chadielle Fayad: I recently worked on an experiment involving several changes to the Sydney Morning Herald and The Age paywall. The paywall went from featuring one subscription type to multiple types. The CTAs were also changed so that they took users straight to the funnel instead of a landing page, and there were adjustments to copy and design to support the multi-package paywall.
Aside from running a direct-to-checkout experience, there wasn’t a great deal of iterative testing involved, which is a bit of a step away from the purest CRO approach. In addition, the direct-checkout test didn’t necessarily lead to a confident result. Regardless, the multi-package paywall test resulted in a 102% lift in conversion with a 100% winning probability. So it was very exciting, and the experience taught me to know when you need to step away from iterative testing and rely on intuition and instinct. The process is there as a bumper bar, but you can use other perspectives to justify a test — if I had just relied on the direct-checkout test for judgment, instead of factors like audience insights and competitor analysis, the business might not have seen this result.
Jinal Shah: That’s an interesting learning as well. Generally people follow one trend and think that’s how CRO should be done, but here you have a different learning, and that helps us learn and optimize further. Our philosophy is that you should continuously optimize and move forward.
Chadielle Fayad: Yeah, absolutely. A test like this is super risky — there are so many variables that if it didn’t work, you wouldn’t really be able to isolate what didn’t work. So it was very good that we had this result, but it goes to show that you can use intuition, you can consult with UX, there can be deep thought that goes into it beyond iterative testing.
Jinal Shah: Does your background in journalism impact how you conduct user research for optimization programs?
Chadielle Fayad: Yes — conducting user research requires a journalistic skill set. I once ran a user test with the goal of understanding frustration points for users on a page. I had to devise my questions carefully to make sure I didn’t lead subjects; it’s important to let people think for themselves to the fullest extent. Another challenge of user testing is getting interviewees to provide detailed and specific answers, similar to what a journalist faces. It’s all about finding the right prompts — whether it’s in an article trying to drive users to read your writing, or trying to get users to give you specific answers when completing testing.
Jinal Shah: Tell us about how you oversee the entire trajectory of optimization — research, design, running tests, reporting — and how you maintain a balance between data-driven and creative strategies.
Chadielle Fayad: Like I touched on before, the best approach to optimization is variety and a mixed approach. Iterative testing is important, but it can’t be used in isolation — insights from previous testing only go so far. I believe in “blue sky” sessions, where team members are prompted to be as creative as possible without limitation. I might also see something a competitor has done and find a way to flip it for our business. But I also believe in data-based insights and clearly defined business problems — I think they encourage creativity. Context is also very important: sale periods, supply and demand, and in media and entertainment, the news cycle and content campaigns. All these factors inform the optimization process.
Jinal Shah: So it’s like data takes precedence, but from your learnings, it has to be a mix of creativity as well.
Chadielle Fayad: Yes, yes — it allows it to be a bit more realistic.
Jinal Shah: How do you determine and prioritize what to test — do you adhere to a specific framework, or is it guided by business objectives?
Chadielle Fayad: I do use a prioritization framework to rank ideas, based on effort versus reward. It considers aspects such as build time, traffic size, baseline conversion, and expected lift in conversion. But the business definitely plays a key role as well — the prioritization framework is what I refer back to when devising the best possible test idea for a business objective at the time.
Jinal Shah: You have experience in media and entertainment services — what are certain critical KPIs that media companies prioritize?
Chadielle Fayad: Relationship building is the overall theme for media company KPIs, like most service-based industries. In the acquisition space, this looks like registrations, subscriptions, and newsletter sign-ups. Once a user has subscribed with us, our KPIs are more engagement-focused, with the goal of reducing churn.
Jinal Shah: So you do more with new users to drive growth in subscriptions, and then nurture them to engage more with the content?
Chadielle Fayad: Yes, correct. We also have upgrade experiences that we look to optimize for existing users, but generally that works well if your main focus is engagement at the time.
Jinal Shah: Are you currently using AI in your work, and how do you see AI impacting different aspects of CRO going forward?
Chadielle Fayad: I’m not using AI in my current work — it will be in the long-term plan. I think the impact is directly related to how tools incorporate AI, rather than a separate AI entity being plugged into the CRO process — so it’s dependent on experimentation technology and tools. I expect it will assist personalization experiments in particular. For example, at Nine we use Piano, and I believe the tool uses AI to provide recommendations for experimentation targeting, based on things like recency, frequency, and content consumption.
Jinal Shah: How should experiment results be communicated to ensure stakeholders are convinced of the value?
Chadielle Fayad: I’m sure most CRO specialists will tell you that emailing a final report is never enough. Ideally you’ve had stakeholders invested before the launch. It’s important to present results in a meeting format with the key stakeholders, so they have a chance to really grasp the insights and ask questions — you want people to interrogate and ask questions, so you know they’re taking the results seriously. It’s about making sure you’re getting some face-to-face time, and not just expecting them to read the emails when you share the final report.
Jinal Shah: What’s the conversation among Australian media companies regarding experimentation — do they recognize its long-term significance?
Chadielle Fayad: It’s very much present in Australian media companies and it is used, but I feel it’s not at the forefront of the process. While they do value experimentation, there’s work to be done in moving it up chronologically. For example, in instances where larger redesigns are being rolled out or business decisions are executed, experimentation is sometimes brought in too late. Using experimentation as much as possible before a hard rollout would be the best indicator that its long-term significance is being recognized.
Jinal Shah: So the long-term significance is yet to be scaled — the thought process is yet to be groomed?
Chadielle Fayad: Yes, yes — there needs to be a bit more consistency in terms of our presence earlier in decision-making.
Jinal Shah: Any three tips to scale experimentation — some significant tips our audience watching and listening can use?
Chadielle Fayad: I’d say my three tips would be prioritization, communication, and process. In terms of prioritization, have your ideas scored so that when you plan your roadmap, you can use this logic and the business calendar to make the best decision about what tests to run. In terms of communication, it’s important to work with teams dedicated to retention, engagement, and acquisition for the products in your remit — just keeping them informed makes buy-in a lot easier, and helps you get things rolled out when you succeed with a test. And then, making process work for you — whether it’s documentation, the pace of a sprint, PowerApps checklists, and so on. Experimentation teams tend to be full of creatives, so it’s easy to fall into a “cowboy culture.” Generating a process that’s authentic to your team provides organization without feeling like you’re being stifled and bogged down in documentation.
Jinal Shah: Any final thoughts or message you’d like to share with the audience before we conclude?
Chadielle Fayad: Just that I’m excited to meet more of you. There’s a burgeoning experimentation community in Australia and New Zealand, so I’m hoping to meet some more people who do what I do.
Jinal Shah: Thanks, Chadielle, for the wonderful insights. And now it’s time to enter the rapid-fire round, where we’d love to hear your impromptu responses.
Rapid Fire Round
Jinal Shah: If you were starting a career in CRO today, what’s one thing you’d do differently?
Chadielle Fayad: I would definitely make absolutely sure I’ve chosen the right technology and tools to execute what I’m trying to achieve — really doing a thorough check and proper research into tools to make sure they fit your business and your team perfectly.
Jinal Shah: Any newsletter that every CRO professional should follow?
Chadielle Fayad: There’s one called CRO Weekly that I’ve subscribed to — it’s quite good for seeing what’s happening in the space and getting ideas for your own tests.
Jinal Shah: Three books you would recommend a listener should also read?
Chadielle Fayad: There’s a book I’ve read called Influence, by Robert B. Cialdini. A lot of the skills I use derive from his theories around marketing principles and principles of persuasion, so I highly recommend that book. I’m not too sure of any others, to be honest.
Jinal Shah: What’s your go-to travel destination in Australia?
Chadielle Fayad: Probably Hyams Beach — it’s incredibly beautiful.
Jinal Shah: Do you often go there?
Chadielle Fayad: Not really — it’s a bit of a drive, so you can’t just spontaneously go, it does require some planning. I’ve been there a couple of times and highly recommend it.
Jinal Shah: One thing you feel AI will replace in the next three years?
Chadielle Fayad: Speaking from someone with a background in journalism, I’m concerned it will replace the need for journalists — because of the way it gets information. A big concern is that it can scrape news sites and reuse the hard work of journalists for its own benefit. I think that’s a real issue. We can already see it’s used in personalization and recommendations, but I’m also concerned it might start to move into the conversion rate optimization space beyond personalization — generating ideas for us, to the point where there might not be a need for a team that ideates, or it could remove the need for a copywriter too. There are some serious concerns with AI in our space.
Jinal Shah: If you hadn’t been a CRO specialist, what other profession would you have chosen?
Chadielle Fayad: Probably journalism. There are actually similar techniques you need to use in both professions. When I worked at Pacific Magazines, my job was to write articles that would generate traffic and clicks — once you understand the digital landscape and what works, the skills are easily transferable to CRO. I feel I could easily step back into journalism with my CRO skills.
Jinal Shah: Tell us about one CRO metric you wish people would stop obsessing over.
Chadielle Fayad: Probably exit rate in isolation — try to understand where they’re leaving to, because it might tell you a lot more than just the fact that they left the page altogether. It’s less about the metric itself and more about not unpacking it further — understanding where they went and what they seemed to be engaging with before they exited, as opposed to just saying “the exit rate’s high, we should probably remove this step.” They might actually be exiting to another sales-focused page — you don’t know. I think exit rate is one we need to unpack more, instead of taking it at face value.
Jinal Shah: A dream or goal you’d want to achieve in the next three years?
Chadielle Fayad: I’d like to lead my own dedicated CRO team. Having been a member of a CRO team alongside digital analysts, UX designers, and other CRO specialists, I’d be really interested in leading a team I’ve already been part of, and putting together my own power team in CRO.
Jinal Shah: All right then. Thank you, everybody — this has been another thought-provoking episode of CRO Wizards. Until next time, keep optimizing.
