Effective Personalization and Stakeholder Alignment

NOTE: This is a raw transcript and contains grammatical errors. The curated transcript will be uploaded soon.

BOTH: If you’ve ever been to South Africa, you’ve probably brought from a Woolworths store. It’s one of the leading brands in South Africa, leading retail brands in South Africa. I think almost 700 stores around the country and Amy Tell us for the global audience, people who’ve maybe not that familiar with the brand, tell us a little bit more about Woolworths so as you mentioned, um,

Amy Ellman: we’re a top South African retailer and what makes our business unique is that we’ve very broad product offering. um,

BOTH: Our departments range

Johann Van Tonder: from

Amy Ellman: to six. food to fashion,

Johann Van Tonder: beauty,

BOTH: homeware. And then within the fashion space, um,

Amy Ellman: we also cater

Johann Van Tonder: to,

Amy Ellman: um, kids, um, men and women the

BOTH: I normally describe it to people, especially in the UK, and tell me if you agree with this, is if you’ve walked into an M& S, Marks Spencer, it feels a little bit like that from a product range perspective and a look and feel Yeah. I think that that’s probably the closest comparison And you’ve been with Wool Ease for a while, a couple of years. What, tell us about your journey. What do you do there?

So I’ve been at Wool Ease for eight years. I

Amy Ellman: joined in 2016. Um, when I joined, um, I joined as the UX and design manager. And at that point in time, we only had a website. The app wasn’t yet in production.

and then obviously, um, the app launched and my role has kind over the years. where I’m at right now is that I still manage UX and design, but my portfolio also includes conversion rate optimization, um, which includes personalization and then also search.

BOTH: And let’s talk about the app a little bit because recently, what was it, the last year or so Dash Launched so there’s is that the app you’re referring to? Yes. so, it’s an integrated app Um When we launched dash originally it was a separate standalone app and that was integrated into our main app. So now Our customers can shop dash or they can shop standard or

Johann Van Tonder: or

Amy Ellman: and

BOTH: and that this is public information I’m, not talking out of the bedroom Um, recently reported 71 percent growth um In dash and profitable so that’s really good numbers. Um, let’s talk about this term, personalization. Personalization. Uh, it’s a fuzzy one.

It’s one that is hard to

Johann Van Tonder: kind of

BOTH: define and put meaning on. You speak to 10 different people, you get 10 different definitions. Do you agree with that? Very much So Okay.

How do you make sense of it? What, what, give us practical meaning to it. What does it mean to you? How do you define it?

So I, I remember the first conversation we had around personalization of goodies, where I was asked if I would be interested in taking that on in my team. And my first response or reaction was, do you mean? Like, Because I know what personalization means to me. But as you say, it can mean many different things to two different people.

And I think that’s know, one of the challenges

Amy Ellman: that we face in

Johann Van Tonder: that space. But,

BOTH: um, we still very much in the early stages of our journey, but where we have aligned in terms of our definition, is that, um, personalization to us is delivering personalized experiences to

Amy Ellman: our customers

BOTH: on information that we know about whether that’s their or their behaviors, um, something that’s

Amy Ellman: unique to

Johann Van Tonder: I

BOTH: see a lot of teams struggling with this. I have these conversations, it feels like daily and people struggle to get out of the blocks to get started, let alone building momentum. is that something that sounds familiar? is that a movie you’ve seen before?

Tell us about your journey. Very much so. I think journey. is an inappropriate word. Um, and we’re still very much on it.

Um, I think for me and for in our

Amy Ellman: team,

BOTH: key aspect was alignment. and I

Amy Ellman: think that’s, Yeah, it’s, it’s, it’s, it’s a key

BOTH: aspect for us. what we did when we started

Amy Ellman: this journey,

BOTH: we kicked think we had about three stakeholder workshops. We really wanted to take our stakeholders on the journey with And, um, I think it was our very first workshop. The goal was to align on the definition, which we just discussed. Um, and, um, And yeah, get by in on, you know, senior managers or yes.

Yeah. Senior managers in the online space. Um, so get alignment on that definition and then, yeah, so that, we were aligned in terms of a business units in terms of online, what

Johann Van Tonder: does personalization also

BOTH: what our goals were for the, for the program.

Amy Ellman: Um, and

Johann Van Tonder: then

BOTH: we had aligned on those, we could kind of take a step further and look at, you know, our vision, our mission, our strategy. um, Um, yeah, I think alignment very much key, Um, What is the detail of that? So you get together in a room with senior managers and people, colleagues, and, you say you, you get alignment, which means you suck the definition out of 20 people around the table or whatever. Did you find that people had different views of the world, different definitions of personalization and how did you make sense of it?

How did you? How did you? get to a point where you agreed about what this is meant to be? They did have different, different definitions.

I mean, obviously there were a lot of but I think at the end of the day, um, there is a common understanding that, personalization is the way of the future, that, that that’s where we need to go, that

Johann Van Tonder: that that’s where we can demonstrate value

BOTH: to And I mean, That’s what we’re here for. So I think that’s kind of what brought everyone, you know, back into the room. Um, and for us, that was a very successful

Amy Ellman: exercise.

BOTH: If you started a new organization today, so you walk into the building and your job is it’s a retailer, your job is to build personalization from the ground up.

Johann Van Tonder: What do you do?

BOTH: it’s day one. So I think. The first step would be to build a team. Um, and part of that is understanding what skills you need.

Um, so in my team, we’ve got a really broad been very beneficial Um, we’ve got some team members who have a usability background, some with a more merchandising background, some of the data And I think that’s really helped um, in, in having those, those diverse skills. So, so building, um, building that team and then clearly defining those roles and responsibilities. And I think for us, that was a little bit of a challenge in the beginning. Um, you know, who does what, um, and yeah, so.

You know, that, that took a little bit of figuring out, um, and then I think putting a bit of structure in place, so defining the processes also an important one. Um, until you’ve got that structure in place, things kind of feel a little bit, you know, ad hoc, Once we had our processes, um, set yeah, we were like in a really good place to kind of hit the ground running and then again, you know, that alignment piece. taking the stakeholders on the journey, Um,

Amy Ellman: you know, I guess

BOTH: figuring out, What level of stakeholder engagement you need, because obviously every business is different and you know, how frequently you need to engage those stakeholders, and your particular stakeholders engagement and involvement do they so I guess finding that balance, and then also, For us, what we started doing in the beginning was setting up our personalization, stereo co So again, taking them on that journey, um, and, you know, always communicating the That has been really key for us. If you are clearly communicating the plan, it’s a lot easier to have buy in. Um, and, yeah,

essentially putting together your first of experiments and experiences that you want to try out, you’ve, you’ve got to

Amy Ellman: to start

Johann Van Tonder: somewhere and very

BOTH: a learning process and you can’t learn until you start. So it’s kind of like you’ve, just got to start with those first few, um, and, you know, launch your first, first few experiences, get them live, um, collect your data and just kind of take it from there. But I um, key is always, you know, like maintaining that feedback loop, Um, feeding back to your stakeholders, letting them know how things are going. Um, Um, and you know, when you conclude a test, closing that up properly, sharing those results, sharing recommendations, you know, what the learnings were, Um, and yeah, I think,

you know, as time goes by and as your team becomes more comfortable

Amy Ellman: with those processes and with

BOTH: those engagements, you see how quickly, you know, things pick up and, um, how things start churning I think the quote so far for me is Just start somewhere. you got to start somewhere. um, make mistakes, get it wrong. That’s how you learn what it is that you need to get right.

What it is that you need to fix. Would you agree with that, May I set up? definitely. I of our learning has been, you know, not rooting for particular results and just, you know, letting that experience run seeing what happens.

And then of course, iterating on it. I think that, That’s also a really, really important that,

Johann Van Tonder: um, you’ve

BOTH: a winning result. It doesn’t stop there like how how much more can you push And, um, you know really that concept um, Exhausting your and um, just pushing that as far as you possibly I think that’s a sign of A& mature mindset

Johann Van Tonder: is, um, call it

BOTH: not rooting a result? Um, I, one of the, I remember my first test. So this is way back when Optimizely

Johann Van Tonder: first started. And,

BOTH: at night, waking up and looking at the graphs inside the experimentation platform. and You know, watching the A B, but it was in the middle of the night. So there was no activity on the site. Like there’s nobody.

and I’m so invested in the outcome of this and my life easier once I actually took the opposite view which is, can I prove myself wrong so almost expecting the result to be not what I wanted and um I mean it applies to personalization but it applies experimentation in general. Um, no. how do you one, two more things about the alignment, because I think this is really important. It’s, it is, this is something I see in a lot of organizations, and I think this often is the unlock, is that alignment

Johann Van Tonder: that you spoke

BOTH: So, That meeting, what does it look like? How do you put it together? Who is invited to the meeting? Um, what is the agenda?

And what is the or questions that you ask them to break through this impasse, you know, where people really don’t know. They have this term personalization, they have this instruction to do personalization, but they have no clue how to do it, what it that meeting look I, think we’ve learned,

Amy Ellman: um,

Johann Van Tonder: So, we launched

BOTH: CRO team a couple of years back and we’ve had a lot of learnings that process. when we kicked off there, um, we weren’t as engaged with our Um,

Amy Ellman: and I think

Johann Van Tonder: I’m

BOTH: grateful for that experience because it’s really helped us in, um, improving the experience of personalization now. Um, so really including them. all. At the very early stages.

So, we’ve got a hypothesis.

Amy Ellman: Um,

BOTH: include them in that before we even, you know, take it to the next level. Um, whereas previously we would have a hypothesis and we would be putting it into design and we’d be presenting a design to them and already a number of hours would have gone in before we had engaged. Um, so really, you know, sharing those ideas with them, understanding Is that aligned with their strategy? Um, and I think that’s another important point.

Obviously, we’ve got our strategy as a team, but we’re part of a, you know, a broader business, a broader product also need to align and you know, with their strategies. And, um, as I explained earlier, you know, with our business

Amy Ellman: being, um,

BOTH: the offering being so broad, there’s so many different strategies and so many different moving parts. So we would essentially connect with, um, our colleagues side and on the FBH side, you know, to ensure that, um, what we are proposing. supporting, you know, where their strategies at if there are any concerns that they have, they’ll share that with us. Um, so I think, yeah, starting at the basics, um, you know, taking them through those ideas, Um, keeping them updated.

Um, also, you know, the initial sessions, treating those as, as workshops. So really, um, asking for input. Um, and also made a big difference. and then, you know, like later on when it comes to our Sterecos and playing back there, it’s more of a, you know, a presentation, but What we found is, you know, because we’ve had that buy in early on, because we’ve engaged with them right through the process, when it comes to those other, you know, Um, you know, engagements, like the playbacks, they’re very much on board and very supportive know of, what we’ve done regardless of what the result

Johann Van Tonder: um

BOTH: So So it sounds like the different ceremonies, you’ve got the steer code. what is the steer So the steer code essentially we we try and do one every month and it’s just a playback of, what we’ve done in that month. Senior management or what sort of level? Yeah, senior management within online.

Um, Just playing back what we’ve done, the results, what’s coming up. So, yeah, same concept of taking them on the journey. Um, closing that feedback What does the rhythm look like? So you’ve got the stereo code, monthly, and that’s what leadership and then terms of weekly or daily rhythms.

What does it look like? So as we part of a greater product um, they work in

Amy Ellman: Sprints,

BOTH: So we’ve adopted that as well because it just it works for us to align with what the rest of the team is doing. And as I mentioned earlier Um, we’ve got a backlog and obviously it’s important to have quite a quite a solid

Amy Ellman: backlog.

BOTH: And what we do every

Amy Ellman: two weeks is

BOTH: refine that. on the backlog and we determine of these items priority. And you know, in, in determining priority, we look at a number of different We look at value, we look at, you know, things that might be aligned with what’s going on in the business. Um,

Amy Ellman: and then

BOTH: we have our spring planning every um, two weeks. So, you know, whatever we’ve marked as the highest priority that might go into our spring planning. And then those tasks are assigned to our and then the team members will ultimately take ownership

Johann Van Tonder: of tasks. know,

BOTH: they need to log a ticket for it, they’ll do so that include the hypothesis, that include the metrics, um, you know, the audience, all of that detail. And, um, You know, they would reset into design if a design is required. Sometimes it’s not. Um, and then we would go

Johann Van Tonder: through that

Amy Ellman: of,

BOTH: um, you know, briefing it into design. There might be a bit of back and forth and some feedback. We would then move it into development. Um, and then again, a bit of back and forth QA, and then we put it live.

but during that process, we stakeholders in the loop. Um, so obviously they are involved in the discussions with regards to what’s on our Um, they.

Amy Ellman: You know,

BOTH: when it comes to the prioritization that that pretty much rests with with my team and we’re well aware of what the strategies um, you know, in online, but, but, they see the designs before they get put into development, um, you know, you know, they’re notified when we’re putting them live. When we do put something live, we also send a mail out to everybody, Um, you know, so that everyone’s clear and obviously, um, yeah, we, we constantly monitoring, um, our experiments and experiences to make sure that there’s no impact on performance. and if there is any issue, again, just like that

those lines of communication are really open and sharing with

Amy Ellman: everyone. Um,

Johann Van Tonder: The

BOTH: The backlog, So that’s effectively experiments, the list of experiments. And, you say you reprioritize them based on new information that comes to light And also strategic direction from leadership. has an impact

Johann Van Tonder: on it. You, you

BOTH: spoke about the audience there. So you said you, you come up with your hypothesis and one of the things that you define within that, hypothesis is the audience. And you know, this brings us towards personalization. Give us a sense of what those audiences might be.

um, I mean, obviously it differs according to the business that we’re in. us, you know, there could be many audiences. So, when we refer to an audience, we might be referring to a food shopper, or we might be referring to, um, a fashion shopper, or a homeware then, when you dig a bit deeper within those, then you have

Amy Ellman: Other audiences. So,

BOTH: within fashion

Johann Van Tonder: you might have

Amy Ellman: someone that shops

BOTH: Country Road or, you know, you might have someone that only shops at Willys for their kids. Um, so, country road is one of the brands. right? Yes.

one of the clothing brands. Um, so yeah, um, can be many, many. brands. many audiences.

do you, so in your case, you’ve got the different categories, the different product categories, and you’ve got food, you’ve got fashion, you’ve got homeware, you’ve got beauty, and so on. And within each of those categories, you’ve got subcategories. Um, and so fashion, you might have women and men and kids. Um, how do you, that’s easy to kind of see at a, at a very broad level.

How would you think about Defining those In your business, but

Johann Van Tonder: generally?

Amy Ellman: so,

BOTH: I mean, we we look at our users um, and, um, yeah, the, you know, the, the data that we have available to us, um, that’s

Amy Ellman: essentially how we, how we define our

BOTH: And this is around analytics, quality, what, what sort of data? Yeah, we have, we have many data sources. So obviously we’ve got our analytics and our data that comes from GA.

Amy Ellman: Thank you. Um,

BOTH: got qualitative data. So data that might come through from things like user um,

Johann Van Tonder: interviews,

BOTH: focus groups, um, you know, session recordings. There’s, there’s so much data available Um, obviously shopping behavior. Um, much of these do you do? you spoke about usability testing and interviews and surveys and, um, is this Done on a continuous basis periodically?

done on a continuous basis So all of that’s actually lives within my team. and As I mentioned earlier, um, UX design, um, research, CRO, personalization, all forms part of my team. So we’ve got researchers and, um, they assist

Amy Ellman: with things like, you know,

BOTH: surveys, single question surveys, user Um, obviously we aim to validate a lot of our, Um, a lot of our projects, through user testing, we want to make you know, that, they work for our And, um, what we have found really interesting is that our customers love engaging with Um, there’s something about the Willy’s brand that our customers are so connected to. you. And it’s really interesting how, when we are in these sessions with them, a lot of them come with these stories, these nostalgic stories about you know, they were shopping in Willys with their mom when they were a kid. And, um, so our customers are really open to sharing with us.

And I think we, we in a really fortunate position in that respect that we, we, we can learn a lot from them. And they very openly share. must say I’ve experienced that as well. If you don’t mind me saying this, um, having worked closely with you for, for many years and I remember one session, usability testing session, it was, early in the morning and it was a lawyer, from a big firm and she came into your office and we had a usability testing session with, a and it made me realize just exactly what you say, how much value people attach to this brand, that a lawyer would give up a morning to come and give a view know, her experience with the brand.

to say that she wanted to share. Do you find that you go to parties, for example, and people hear that you are Amy and you work with Willie’s online and the app and so on and they they start giving you their views of how it should be better or or. you get that? All the time.

I mean, that’s funny. I mean, I can rely on my mom for some of that as well. The mum test, right? There is a thing like that.

I think there’s a book written, the mum test, uh, do usability testing using your mom, uh, Um, the, the process

Johann Van Tonder: you

BOTH: are you able to share with us some of the frameworks and the,

Johann Van Tonder: you know, the, the, the practical, the, the practical

BOTH: the day to day, What is a, there even something day? How do you make sense of

Johann Van Tonder: all this data

BOTH: coming in? sure all the opinions. and then through frameworks or whatever methods or processes

Johann Van Tonder: you have,

BOTH: at the end of it spits

Johann Van Tonder: out the test

BOTH: What do you do? Yeah, so that’s quite Um, and it always starts with research So,

Johann Van Tonder: you know, as

BOTH: as we’re talking about now, data, there is so much data available.

Johann Van Tonder: So

BOTH: at what’s available, looking at customer behavior, um, also looking at sharing with um, you know, feedback post purchase, um, what do they want? What do they need? What has disappointed

Amy Ellman: them? Um, you

BOTH: know, and taking all of that into consideration. And then, um, you know, we, we touched on being part of a broader team and obviously that team has got

Amy Ellman: its, you know, It’s strategy.

BOTH: So, you know, looking at all the data and looking at, okay, can we align? How can we add value? Um, and in looking at the data and doing the research, um, it might spark an a prediction. Um, we would then typically write a hypothesis put that in the it’s the first thing when you look at the data, do you try and identify what comes first or maybe there isn’t an answer to this, Um, Is it as simple as an idea jumps out of the data and then you think who is the audience that this idea relates to?

Or is it the other way around that you look at the data and you go, ah, here is a segment, a sub segment of our audience. and what are some ideas that we can,

Johann Van Tonder: bring to them?

BOTH: it’s both. I what data we’re looking at, how we’re looking at it.

Amy Ellman: Um, Yeah,

BOTH: you’ve got no specific lens that you go and look into the data and try to define audiences or try to define doesn’t sound

Johann Van Tonder: like think lens aligned

BOTH: our strategy. So whatever, strategy is what you know, whatever our are we obviously looking for opportunities

Amy Ellman: in that

Johann Van Tonder: space.

BOTH: Um, and then also obviously voice of customer,

Amy Ellman: and if there’s a particular pain point

BOTH: Um, Perhaps we are looking for you know, like, common, a common pain points? Um, perhaps we can resolve that, um, you know, through personalization or, um, yeah. And I think also because, um, You know, we were talking about the skill set in the team earlier, um, and how we’ve got a broad skill set of people who are coming from different areas. I think sometimes people tackle it in slightly different ways, quite interesting.

Um, does that, is that good? Does it cause conflict? or I think it’s great. Um, You know, we’ve, we’ve got people with merchandising backgrounds, project management backgrounds, usability backgrounds, and I they intertwine so, so, nicely.

Um, and I think what I’ve noticed is, and obviously our team is still very young, is how each person is really finding their place in the team and, and adding value. Um, and each person’s

Johann Van Tonder: contribution is

Amy Ellman: different.

BOTH: Um, and I, I think it’s really beneficial and, and I would actually recommend that. um, that might not have been my view going in. Um, but now that we’ve tackled it in this way, I’ve really seen, you know, the value in. it.

your initial stance would have been that you want a similar team with similar views and what is it that brought you to this Different view of the world that welcome different opinions different So I think there’s around personalization. Um, whenever I meet new people at, at Woolies, you know, if

Johann Van Tonder: it’s like

Amy Ellman: at, um,

BOTH: a seminar or, or anything like that, and you’re exposed to other teams and, know, you tell them what you do, they’re like, Oh, you’ve got you know? Um, so there’s, there’s definitely a lot of excitement around it. And there’s a lot of people who have in, um, getting involved, which I think is amazing. And is this personalization specifically or online in general, CRO?

What is it that,

Johann Van Tonder: Personalization

BOTH: I think personalization and CRO, obviously there’s a lot of excitement in online as well. You know, we’ve got Dash, we’ve got a lot of growth, but specifically within my team, I think, are being opened you know, the possibilities with CRO and, and personalization. um, they’re interested and they want to get involved. They kind of like pie.

And

Amy Ellman: you know,

BOTH: learning and growth. And, um, Even for me personally, I’ve learned so much, moving into the CRO space a few years ago. And, um, you know, a couple of years ago, I would have probably looked for, for someone who was, you know, um, or, or, or a team who is very analytical, um, you know, had, an analysis background, a BI background, um, and UX and, you know, merchandising and, um, that side of it would have perhaps been Um, so now I kind of look at it in a very different You want a little bit of each or how do you make it up? Again, you, you’re back, it’s day one, you’re in a new organization, you’re told to build personalization from the ground up.

What is the, the minimum team, not the ideal team, obviously you need people who, who, who are strong with data. You know, we’re working a lot with data, you know, with experimentation, with metrics, with GA, um, but I think usability also plays a significant role. And, um, you know, when you have ideas and you’re briefing in designs, if you can um, from a usability kind of standpoint

Amy Ellman: and

Johann Van Tonder: understanding the

BOTH: customer, plays a really big role. Um, and I All of this is connected,

Amy Ellman: you know,

Johann Van Tonder: UX,

BOTH: UX, CRO, personalization. Um, it’s very hard to, to, to separate them. And, um, you know, when we started this journey of personalization, one of the things that we needed to make sure that we aligned on was

Amy Ellman: how

BOTH: personalization is,

Amy Ellman: not a

Johann Van Tonder: a feature, like

BOTH: way of doing and a way of being almost more of a strategy. Um, and You know, when you take a step back and you look at it in that way, you can really identify how personalization can weave itself into different areas of your teams and of your business. And, you know, I think in my space, um, personalization is in recommendations and it’s in search and it’s you know, CRO and, um, yeah,

Amy Ellman: uh, it’s,

BOTH: really to, take that step back and see how it all intertwines and, um, and UX obviously is also playing a role in, in all of those things. and um, but I think in terms of skills, data and usability and just having that understanding of customer empathy, um, obviously also being able to analyze data. Yeah. Um, I’m going to come back to the customer understanding of the data a little bit, but, Um,

Johann Van Tonder: Let’s

BOTH: and color it in a little bit. What, What, might the personalization initiative at Woolworths look like? So we spoke about the audiences and food and women and know the different sort of product categories. Does that define the initiative or can you mention some?

So, I, mean any, any shape or form it’s um, the possibilities are endless. could be something as simple as reordering your navigation, um, for each user based on, on their behavior based on the departments that they shop. Um, right. so, So I, I come in and I buy by men’s clothing and you come in and you typically buy food and we, do we then see a different menu navigation?

How, would it, how would it, how would it be different for you and me in this case? so so yeah, a good example of something that one can do in the retail space. Um, you know, it’s really simple and it’s almost like a no brainer. is this an actual example of the, company?

Yeah, yeah, that’s an example. Um, and yeah, it’s really simple, and sometimes it’s the really simple opportunities that yield the better results. So that’s an interesting point as well.

Johann Van Tonder: Let’s move on.

BOTH: Move there for a moment, because what I’ve often heard, and this relates to personalization, but also CRO in general, is there’s a there’s a hunger, there’s a desire, especially from senior stakeholders to do something more complex, right? To be a little bit bolder. Um, and what I’ve often seen is actually the big results, the big swings come from simpler ideas. It sounds like, I don’t want to put words in your mouth, but it sounds like you’ve seen something Yeah, well, I mean, like, Our

Amy Ellman: our

BOTH: experiments are very broad. We’ve got some more complex ones. We’ve got some simpler ones, really interesting to see which other ones. that perform, better.

And I think, you know, that of value. Um, when we are, you know, working through and prioritizing

Amy Ellman: what’s

Johann Van Tonder: in our backlog, really important

BOTH: aspect be able to identify Which ones are going to have the most value both for the customer and you do that and for the business. How do you determine that up front? Because that’s why you run a test. hard.

It’s hard. So your hypothesis. Um, but you know, we obviously look at the data and we look at you know revenue and traffic and um, we obviously make certain Um, right. So it’s the exposure of this particular, experiment to say revenue generation or generation.

So the more people, I’m oversimplify, but the more people to see this experiment, the higher it will rank, the more money flows through this experiment. Yeah.

Johann Van Tonder: I know,

BOTH: know, like value also in terms of we want to create value for our

Johann Van Tonder: customers.

BOTH: We, want, and how do you put your finger on that? Because value is quite a, it’s hard. It’s hard. Obviously like experimentation is.

is, one way and we, um, we obviously learn a lot through each experiments and then iterating on them. Um, but, Um, yeah. So in a moment I’m going to ask you about value some more because obviously senior stakeholders, they’re going to want to see a return on this. So we need to talk about ROI.

Um, but back to just a few more examples personalization initiatives. You spoke about the menu on the navigation. Some other ones that you can think of? Um, gosh, I mean, we’ve done so much, but like, an example of, you know, for a retailer like ours could be, um, you know, we spoke about the different So, you know when you are going to a area of, of the site the app, um, you know, making sure that it’s, As relevant to you as can be or even something like a special occasion Um, you know, I think for all retailers special occasions are like valentine’s valentine’s christmas, you know, Um father’s day father’s day all that kind of thing.

Um, and I think christmas is the one that pops in my mind first because um You know, there’s like a food, aspect Um, so obviously like gammon and You bubbly and all those kind of things are. really big for all retailers. But, we’ve got a really diverse customer base. Mm-Hmm. .So,

um, yeah, I I

Amy Ellman: I guess

BOTH: something like knowing who your audience is, um, do they eat gammon? Or

Johann Van Tonder: I

BOTH: maybe some of us do some of us don’t. Yeah. Some of us are Hello. some of us are ous.

Exactly Right. So. If I’m a LOL and I came to the site, I would see a different experience in the food environment compared to somebody who is not a LOL. Well, that’s what we would strive to do.

And I think that, um, yeah, I mean, that would be my recommendation. And I can see how that would be a benefit to me as a customer.

Johann Van Tonder: Frankly, don’t

BOTH: want to see

Johann Van Tonder: gammon because I don’t eat it. Um,

BOTH: but then how do you so that would be another good example of How Do you know?

Johann Van Tonder: That I eat

BOTH: gammon a lot. Have I told you this

Johann Van Tonder: or is it something else? So

BOTH: haven’t told obviously, um, based on your shopping behavior, the things that you buy, you know, for

Johann Van Tonder: example, if we can has never from walrus,

BOTH: you know, we can make an assumption. that You know,

Johann Van Tonder: perhaps they don’t drink alcohol. Um,

BOTH: obviously there are other ways also. Um, you know, depending on your business, you might have data as well as online data. Um, but, yeah, I mean, that’s just one way, just behavior um, you know, what their repeat purchases are. Um, you know, like

Johann Van Tonder: the departments that they are. They’re

BOTH: shopping Let’s do one more these ideas are, are nice. and it’s, uh, Let’s take another, business maybe. if you well, up. You can choose one from Willy’s or you could, um, maybe one of your favorite, um, shopping experiences or another meaty example of personalization

Johann Van Tonder: that you

BOTH: Where do you like

Johann Van Tonder: shopping?

BOTH: The thing about personalization, while you think about is it if it’s done, well, it’s invisible. Yeah, you wouldn’t know you wouldn’t know you’re being personalized exactly and we had this very conversation this week with my team and how, um, yeah, it’s, it’s actually the art is in the subtlety. And, um, I remember many years ago Um when I was working in advertising, maybe like 15 years ago, maybe more. Um,

Amy Ellman: And,

Johann Van Tonder: first experience of personalization

BOTH: was we were working on a and, um,

Johann Van Tonder: I was a designer many years ago,

BOTH: and I was told to design something based on the weather, um, served, um, You know, different suggestions. Oh, that rings a bell.

Amy Ellman: yeah.

BOTH: And, um, you know, like, like that kind of thing comes up quite a lot. And I, I, often, um, you know, like say to the team. cool. But like, Actually needs to be subtle.

Like, you don’t need to say, Hey, it’s, it’s raining.

Amy Ellman: Um, it, it, it

BOTH: can just you know, how you are serving the product, you know, without actually spilling it out. .Um, yeah. But, yeah, I

Johann Van Tonder: mean,

BOTH: But it’s also to your earlier point about where the idea comes from and, you know, the way you explained your process is you look at the data, you look at the voice of the customers, the way you put it, what are the themes that jump out? And then based on that, how do we solve this? And if relevant, how do we solve it for this specific audience, As opposed to, I heard at a conference this cool idea about, um, weather personalization, And so let’s try that. Absolutely.

Amy Ellman: I mean, like,

BOTH: really pride ourselves as a team on being data um, what does that mean? So using have available to inform the

Amy Ellman: decisions Um,

BOTH: so to inform our strategy, to inform, um, you know, enhancements, improvements, um, ideas

Amy Ellman: for

Johann Van Tonder: personalization,

BOTH: um, really knowing what our customers need, what they want.

Amy Ellman: Um, yeah.

Johann Van Tonder: I

BOTH: find being data driven really hard thing

Johann Van Tonder: and a really humble thing because often data disagrees with you. Do you find that?

BOTH: I think, you know, it’s harder in bigger organizations because you’ve got to find a balance between your business needs and your users needs. And, um, yeah, but I think, um, I think we are learning to do a pretty good job at that.

Amy Ellman: Um,

BOTH: But I think that’s also a journey and it can be really tough, especially when you’re starting out to, you know, have the confidence to have those hard conversations. Um, what I’ve really found is when you’ve got an idea and you’ve got the data to back it up, to argue with that. data. Um, I think where we’ve had pushback has been where we haven’t had the

Amy Ellman: data.

BOTH: the big question then ROI

Johann Van Tonder: back to this thing about value,

BOTH: how do you show

Johann Van Tonder: value around personalization to who write the

Amy Ellman: So.

BOTH: obviously, you know, as part of our process. We write our hypothesis and we create a ticket for, um, you know, for the task. Um, and part of creating that ticket is obviously defining the audience, but also defining like what are, we measuring this.

Amy Ellman: um, you

BOTH: What does success look like for, this particular piece? so we know upfront how we’re going to be measuring it. So the standard metric that you apply to all the initiatives, it’ll be different depending on the Yeah, Different depending on what it is. And then we’re very intentional with how we feedback

Amy Ellman: on that. Um,

BOTH: So obviously while an experience is running, you know, we monitor that very closely. um, if we see any issue, you know, we’ll stop it immediately. So there’s no negative impact. Um, but you know, once we conclude.

Amy Ellman: a

BOTH: or experience, um, we obviously go through a um, analysis And, um, once that

Amy Ellman: is complete

Johann Van Tonder: play that back.

BOTH: Um, and we obviously share those results based on those um, You know measurements that

Amy Ellman: we So

BOTH: got a, hypothesis. We believe doing A will lead to B. And this is how we’re going to measure it and then you report that metric um And that metric is different and is it always related to money or sometimes related to money or never related It’s not always related to money. Um, Yeah, um, revenue is just you know, just one of them.

Um, but I what we do is the next step is often we iterate on that. And I think that’s been really interesting us is, you know, we’ve had a winning test with results that we’ve been like, wow,

Amy Ellman: okay,

Johann Van Tonder: cool, that that’s of

BOTH: it a step further and then we get an even better result. um, and I think that’s really exciting, you know, for, for our, for our leaders to see and also for our teams, um,

Amy Ellman: to see.

Johann Van Tonder: uh,

BOTH: yeah. What about negative experiments? So we talk about the winners and those are always the nice ones to talk about. Um, In My experience, some of the biggest wins I’ve seen started out as negative experiments and the reason for that is there’s no other way you could have, stumbled upon this insight other than running a negative test.

Have you had a similar experience?

Amy Ellman: How,

BOTH: you feel about negative experiments and you try to forget about them as soon as possible? Actually, no, we’ve had a number of

Johann Van Tonder: negative

Amy Ellman: and

BOTH: I think it’s just, I’d be worried if you say anything else it goes And I think what we’ve really learned to do in my team is to kind of welcome those as learning opportunity to collect more data and to, know, build that deeper understanding of of our customer of how they engage with things. know, that’s easy for you How easy is that to communicate up the layers? Uh, how receptive are senior leaders that kind of message? Do you find it, have to massage it or does it land in the same way?

I think it doesn’t land exactly in the same way but I think you know, you can frame it well and you can frame it as even de risking you know, um, that’s feature that maybe intended to, to go into production. I think anytime there is a learning, it kind of softens that, if it was a case of, oh,

Johann Van Tonder: you know, this

Amy Ellman: a lesson,

BOTH: you know, throw away and, um, maybe it’d be different, but, every experiments, even if, you know, the test doesn’t win, there’s something to learn from

Johann Van Tonder: that. And

BOTH: perhaps that leads to an iteration

Johann Van Tonder: is

Amy Ellman: successful.

BOTH: What do you think people? in other organizations or generally mess What do you think goes wrong And it might be something you’ve experienced yourself think that, Um, I think in a lot of organizations fear plays

Amy Ellman: a part.

BOTH: um, Fear of? Fear. So if, if, if you look at a big retailer, as an obviously that’s my context, So that’s. to that, but um, and you look at a physical um, you’ve got your shop windows and you’re able to, you know, display any big deals there or, you know, something new, you know, like now summer’s launching, you can put your new summer items there.

Um, are walking into the store and they are seeing what’s in Um, and if you compare that to a digital experience, so a website or an app, your, your landing screen. is, um, is much the Um, and I think, I think fear definitely holds people back. Um, with regards to personalization, because fear, fear of what? So, because if you are delivering personalized experiences at that level on, on the landing page level, um, you are showing, you know, audience a, What they want to see an audience be, what they want to see, et cetera, et cetera.

So they might not be seeing What you as um, you know, a manager in a certain area wants them to what your strategy you know, what your KPIs are or, or what’s the focus in the business is at the Um, and it’s almost like relinquishing control and, um, you know, giving ownership that landing page to the user them what, what they actually want to see

Amy Ellman: and trusting

BOTH: that you are still going to have The results that you’re you know, the audience that you’re targeting still going to see that. So I think that that definitely plays a role. Um, because you know, like I can see how, you know, in some organizations people might be reluctant to kind of like take that leap and be a little bit scared, you know? Um, Okay.

So if, if, if I’m the leader responsible for category A and category A is always featured well on a certain page, let’s say the homepage, and now, because we started to do personalization, The audience might not see category A, They will see category B and C, Is that what you’re talking about? Exactly. But the audience who is going to buy that category would still see Um, and I think it’s just

Amy Ellman: that,

Johann Van Tonder: shift in been

BOTH: of the you know, one of the And, you know, that’s where I can see people, you know, like struggling a little bit That’s um, making mistakes,

Amy Ellman: kind of like holding back.

BOTH: Um, and I think just, you know, like I keep hopping on about?

Johann Van Tonder: the

Amy Ellman: It’s

Johann Van Tonder: just

BOTH: for me, like

Johann Van Tonder: the most

Amy Ellman: important

BOTH: um, you need to have buy in from your Um, and you know,

Amy Ellman: That’s that

Johann Van Tonder: that trust.

BOTH: And, um, I think also those stakeholders trusting the team that is running with personalization, you know, that’s, they’re doing the job Um, you know, they’re monitoring where they need to monitor. Um, and that’s everyone’s on the same page. We’re all trying to, um, you know, deliver value to our customers yield better results. I’ve heard it said that stakeholder management

Johann Van Tonder: is not the sideshow,

BOTH: is the show. 100%. your experience as well? Yeah, I think, and for us it’s been great to see how our stakeholders are on board, um, but I think definitely taking them on that journey.

Yeah, Um, and there’s a debate in the industry and maybe we’ll, we’ll, um, we’ll geek out a little bit. But, um, velocity, so the number of experiments versus, what’s commonly called quality. I don’t really think they sit on the same, um, scale, but back to the question about value and ROI, In your organization, is it about is value

Johann Van Tonder: about

BOTH: the number of experiments or the number of initiatives that you produce in a given time period? So the input, or is it about Regardless of the number of inputs, the number of experiments, what you can measure on the other side of that. How do you think that in broad terms? So for us, it’s far less about

Amy Ellman: velocity than it is about the quality and

BOTH: actually choosing those experiments based on what we believe is going to drive value. Um, yeah. So is that quality? 100%.

percent Yeah. So, quality? for you is choosing experiments that you believe are going to drive value? Yeah.

think, you know, when we started out, um, we were perhaps doing more, but being less critical of, um, of the value of each of them. Um, and I think now we are looking at each with a much more critical lens and really, you know, looking at our backlog and um, prioritizing. we said value earlier, but also based on aligning to you know, the strategy of the business units, um, And what executed in our product teams. What does that mean in practical terms?

So aligning to the strategy um, do you, so you look at your corporate strategy or your business unit strategy, your managers KPIs,

Johann Van Tonder: KPIs,

BOTH: then. Determine how Cascades into the work that you do or, do you make, how do you so, yeah, so obviously, you know each business unit would have their strategy and then within that we’re part of

Amy Ellman: the,

Johann Van Tonder: the broader product

Amy Ellman: team. And the product team has got their

BOTH: strategy, and

Amy Ellman: you know, what’s.

Johann Van Tonder: our opportunities are

Amy Ellman: as a product team.

Johann Van Tonder: and then what

BOTH: we do is we look at, you know, how can we support

Amy Ellman: that strategy? Um,

BOTH: because we’ve all got to ultimately have the same vision. Um, we’ve got, a, How we are contributing to that vision is Um, so, we look to that strategy and determine, okay, How can we help the Is that monthly, quarterly, annually?

Johann Van Tonder: What sort of

Amy Ellman: Um,

Johann Van Tonder: cadence?

BOTH: so, I mean, well, you know, we look at, our strategy annually, but, um, we plan for each So, um, when we plan for each quarter, we look at, teams working on? How can we add the most value? If you’ve got two personalization initiatives on a list, and you’ve got to decide which one to do, and the one,

Johann Van Tonder: um,

BOTH: based on your judgment, you think might generate more revenue. But it’s not as clearly aligned to strategy as the other one that has less exposure to revenue, which way would you lean?

Amy Ellman: I’d

BOTH: probably lean to, the strategy. Interesting. Because ultimately

Amy Ellman: that will lead to you know,

BOTH: that revenue

Amy Ellman: that you’re after.

BOTH: Um, I mean, obviously if there’s an experiment that’s doing well, but it’s very much aligned to strategy, I wouldn’t you know, recommend that.

Johann Van Tonder: but of,

BOTH: okay, a little bit more revenue

Amy Ellman: you know,

BOTH: the one side. Um, I think that it’s really important to have that holistic vision of, of, you know, the way forward and how we want to kind of show up as a digital product. Is there anything you thought about personalization years ago before this, that view now that Yeah. Um,

Amy Ellman: I think, you

BOTH: like I mentioned earlier, the skills, Um, me, evolved quite a bit, um, that it doesn’t need to be an analyst who’s got years and years of

Johann Van Tonder: experience,

Amy Ellman: um, that

BOTH: can welcome different the team and, add equal value.

Amy Ellman: um,

BOTH: think, I mean, tools obviously evolve, um, all the time. And

Amy Ellman: it’s interesting to see,

BOTH: see the You know, the changes in that Um, That’s an interesting one too, because I I hope I’m not misquoting you, David Mannheim, but I remember you saying something like the problem with tools in personalisation, you know, the question of the, the tail wagging the dog, you know, people starting with a tool and then the tool kind of dictating the strategy. Um, that’s how do you think about without mentioning specific names broadly? So, um, what do you use? What’s, what sort of what, what categories of tools do you use?

And I mean, there’s obviously

Amy Ellman: so many tools out

BOTH: and I think what’s really important for, for each team in business is to find the tool that that works best for them. So for us, it’s very much a

Amy Ellman: case of, you know,

BOTH: best of need. Um, as

Amy Ellman: opposed to best of best

BOTH: breed. Um, and, you know, um, we’ve worked with a

Amy Ellman: a couple of

Johann Van Tonder: different tools and

BOTH: I don’t think there’s bad options Um, think that, you know, if you are in a bigger organization and there are other teams using tools, um, and maybe you can, you know, piggyback on that and use those tools, you access to, to, to, their data. I think, you know, That’s obviously gold. Um, you know, there’s, there’s, no wrong. Um,

Amy Ellman: I think that,

BOTH: um, you need to have your, your strategy defined. You need to, you know, know? what your vision is, be aligned in your vision. You need to have your backlog.

The tool is, is very much secondary, you know? Um, yeah. Somebody has been watching this, but not really paying attention. I kind of been, uh, reading the email and, uh, just listening with one ear.

and they’re suddenly tuning back into the end of the and they’re in your position, run personalization or they’ve been told to set up personalization. What is the one should take away from this

Johann Van Tonder: What do you want to is really

BOTH: stakeholder

Amy Ellman: stakeholder

Johann Van Tonder: alignments, just taking

BOTH: them on a journey with you.

Amy Ellman: Um,

BOTH: And, you know, like communication open and just reinforcing the, the, the idea that, that you’re part of the same Um, I think often, you know, with personalization or any form of experimentation, there can be some heated debates. Um, you know, you spoke about opinions. earlier. Um, We’ve all got different opinions, but, um, obviously relying on the data, but just making sure that you’re taking journey and, and, and and having their buy in.

And you can only do that by, by building that that trust is built through cap, through collaboration. Um, so yeah, I, I think that would be my, my number one and you know, that, that’s something that we’ve definitely learned, as I mentioned earlier. When we started CRO, we did less of that, and we’re now doing more of it. And, um, yeah, we’re seeing really, really good won lessons from the trenches.

Stakeholder management is the show, not the sideshow. Amy Allen from Woolworths.