Putting the Person(ality) Back Into Personalization: A Behavioral Scientist's Journey

NOTE: This is a raw transcript and contains grammatical errors. The curated transcript will be uploaded soon.

Divyansh Shukla: Hello, everyone. Welcome to Convex 2024, the annual virtual summit by VWO. Thousands of brands across the globe use VWO to optimize their customer experiences by gathering insights, running experiments, and personalizing their purchase journey. I feel honored to have Florent on stage with me.

He’s the former experimentation principal at cars. com and commands more than a decade of experience in consulting behavioral science and experimentation teams. Hi, Florent. How are you doing today?

Florent Buisson: I’m doing very well. Thank you very much for having me.

Divyansh Shukla: Awesome. That’s great to hear, Florent. We’re delighted to have you here to discuss such a pressing topic. In a world where clutter often overshadows clarity, personalization emerges as one of the most potent tools.

But without giving too much away, Florent, would you like to give us a sneak peek of what you have in store for us today?

Florent Buisson: Yeah, so the I mean, as the title mentioned, I’m a behavioral scientist by training, and so today’s presentation is really going to show how I’ve been approaching personalization in my different roles and kind of the challenges I’ve encountered I address them.

Divyansh Shukla: That’s awesome to know. Folks, do keep your pen and notebook ready. There will be a large share today. Florent, please take it away.

Florent Buisson: Okay. So hi, everybody. It’s a pleasure to talk with you today. So yeah, today I will talk about my personal experience with personalization.

And so to put it a bit more into context, so I come in originally from France where I did a PhD. Behavioral economics. And after a short, I mean not that short, after stint in management consulting, I got back to my career in the us uh, where, and I had the opportunity to lead the behavioral science team and work on experimentation. And so really for today’s presentation, I have anonymized and blended a few.

of the experience that I’ve had with personalization across my career in order to also make that more understandable and more applicable for you. So, how am I going to approach the credential implementation? First, I will talk a bit about how to frame and understand the problem because I feel that even though we talk a lot About personalization, it’s not always clear what we’re talking about and clarifying what we are talking about can really help us be much more effective and avoid getting lost in the process. Then I will talk a bit about how to get started and how to kind of Get your toe into the water at the early stages for personalization before moving on to personalization at scale.

And that is where we really have to use an algorithmic approach much beyond what we can do manually. And here I’ll talk about multiple approaches. So using creative, using data. And then I’ll touch on using psychometrics under the data perspective.

So when we talk about personalization, what do we talk about? And the challenge I was originally given is probably going to resonate with a lot of you today. And so what I was Told by my bosses and business partners At multiple points in my career Was some variation on this explicit challenge we have a lot of data on customers characteristics and behavior. How can we use that data to successfully?

personalize that experience. And obviously I say this was the explicit challenge I was given because in many circumstances, I mean really in all of those circumstances beyond this explicit challenge, there was also an implicit challenge this idea that, well, personalization is really cool, really powerful, and it’s really going to give us a competitive advantage. And I think it’s important to Recognize those implicit expectations that our business partners and stakeholders might have because I feel that even though personalization can be powerful, it is also complex and takes much longer than stakeholders might expect.

And so I think it’s really important to clarify the expectation of our partners in order to be able to manage them and be more realistic. But given this challenge, how should we think about personalization? What we had at the time We already had some element that felt a bit like personalization. And so from the marketing team, that would often be given personas.

And so, those personas would be qualitative. They would be kind of list of features. Demographic characteristics and a kind of archetype of customers or potential customers. And so the most classic, classical one obviously is the soccer mom.

But really those personas that can have sometimes very rich details. They’re not necessarily very useful for personalization in the sense that it’s, well, how do I know that the person I’m, that is on the other side of the screen, the person who is on my website or sending me an email is a soccer mom or a basketball dad or whatever, what have you. Um, on the other hand from the product team, I would often get this idea of the stages of purchasing a journey So again, those data would have some variation on kind of awareness, interest, and then getting into the purchasing funnel. So here we’re

moving from demographic characteristics such as age, gender, income, etc. to more attitudinal and behavioral characteristics. So what is this person’s attitude? Are they aware?

Are they interested in our product? What are their behaviors? Are they actively shopping or are they still just browsing? But again, that is not necessarily very useful because we may not know where the visitor on the website is inside, within that journey.

So again, when we talk about personalization, there is often a lot of material that we’re given from our business partners. But in my experience, I found that it was quite hard to take that material and build a personalization program successfully from it. And so the approach I used instead was inspired from a philosophy professor I had in undergrad. And so when we were given a topic to analyze for an essay, he had this advice which is If this was a chapter of a book, what book would it be?

So whenever you’re given a problem, try to think, well, what is the bigger picture? What is the domain of the field or the field that this problem is? belong to. And so it gives us, that gives us access to kind of first principle and a broader perspective.

And so because of my training as a behavioral economist, behavioral scientist, I was able to recognize like, well, personalization, really that is, we can think of it as an application of psychology and more specifically personality psychology. Then next step. Figuring out, okay, how can I learn about psychology? Uh, I asked publicity, ai, which is now my favorite search engine slash um generative ai tool.

And so what is a good introductory book? And the top recommendation, Personality Puzzle. So I read the Personality Puzzle, which is definitely a book I would recommend to people who have to deal with personalization, but don’t have some grounding in personality, psychology, or in psychology in general. So now, obviously, to get to the short version of what this book is talking about, it’s, well, what does personality psychology tell us?

Personality psychology tells us that by and large the best way to understand people is to use a model that is called the BX5 or also called the OCEAN model based on the five traits it refers to. So openness, conscientiousness extroversion, And neuroticism. And I find this visual pretty helpful in recognizing like, well, when we’re talking about neuroticism, what are we talking about? What does that look like?

What does that feel like? Well, it, and the people who are low on neuroticism. are unflappable and don’t get easily unnerved, whereas people who are high on neuroticism are anxious and easily stressed, and so on and so on. And so, once we have this model of the Big Five, Then that gives us a way to understand our customers and classify them and think through Oh, are our customers like high on openness low on openness and then we can use that to have a different message or to take a psychology term a notch and so in this case we have Advertising For makeup cosmetics, that is kind of designed around extroversion.

And so here you can see we have two different visual for to advertise cosmetics that are addressed. So one is addressed to a more extrovert audience. And the second one is more addressed to introvert audiences. And so that is.

Basically, to recap, our understanding of the problem is, well, we have business partners often come to us with this kind of big expectation of like, personalization is magic, can you do it and solve all of our business problems at once. And I think it is important to manage those expectations and to clarify like, well, how are we going to think about personalization? We’re going to think about personalization as this interaction between segmenting our customers in some way based on their personality and having creatives that match those segments and those traits. That’s really the way I like to, I’ve come to frame the problem because I think it’s how we can get good results.

Now, how do we do that? and practice. And so my recommendation is definitely to start manually. Even if you have tools that allow you to do this at a broader scale, I think that doing things manually for really gets you a better feel for how things are working in your specific context, in your specific software.

And so, the perfect opportunity, a perfect starting point for personalization is to use an A B test, so what would be normally treated as a failed A B test, where we find that the treatment is not better than the control overall, But it is better than the control for some specific segment. And so to visualize what that would look like is, oh, if you think about, okay, we have the control treatment A, that overall has a better conversion rate than the treatment B, the treatment. But we find that, oh, if we look at segment two, we see that treatment B is outperforming the control A. And so really we can apply that treatment B To that specific segment and so the way we would do that is to have some sort of system that said hey by default Serve the control a to everybody But if we have information that tells us that this visitor this user this customer belong to segment two then show them could want to be instead.

And so, using that perspective, that very simple if else loop, we are started, are getting started on our journey with personalization. And really, I think personalization does not have to be very complicated, simply having one personalized experience for a specific segment can already teach us a lot and show us a lot. And so then the way you would know if that personalization worked. would be to have an A B test, so ensure that you’re showing the different treatment to the different segment at random.

And then confirm that, yes, in segment one the control, the default experience is outperforming. But in segment two, we find that our personalized experience is working better. Now, one challenge, and this is a broader challenge with personalization at large, is that measuring statistical significance becomes more complicated. And so, if you are used to calculating, t test or z test by hand, you will need to do a slightly more complicated math.

So to be able to measure the effect size by a segment, you will need to Uh, use heterogeneous treatment effect, and if you’re starting to making multiple comparisons, you have maybe several segments, then you’ll need to correct your p value for that purpose. Now moving pretty quickly over that because in most cases your experimentation software will provide those measures for you. But if you don’t have that, I will provide some resources on how to do those maths at the end. And so this is really Uh, how I would, how I did and how I would recommend doing personalization, really starting with manually try to understand what’s going on and really build awareness and understanding across the organization of what’s happening.

How does it happen? Because once you get to scale. Things get much more complicated, because scale creates complexity. And the challenge is, once you try to scale, is things that were manageable, but a bit difficult at a small scale, will become unmanageable at a large scale.

You need to have a very smooth process before you are able to personalize at scale. And one of the reasons for that, one of the things to keep in mind is that personalization creates what I would call forever permanent technical debt. Because if you think about it, personalization means having Multiple code snippets that can be served to different customers or different visitors. And so now that means that if, say, a product team wants to introduce a new feature, they have to Consider, or maybe they don’t, they should, consider, well, how will that new feature play out with our experience A, how will that new feature play out with experience B, And they might have to debug those how that features work with a variety of experiences in a variety of contexts.

And changes that needs to be made will need to be made across those various experiences. And so that means that you’re creating technical debt that you will have to pay essentially The interest on forever. And finally, you will have to be more cognizant of how the this fits with the other processes in the organization. So now, for example, who you have to think about.

Well, we have those prints, for example, the agile print. And how are you gonna ensure that engineering processes or the way, like I mentioned, that product managers think about their work or how are you going to enter compliance if you have some needs to keep track of the data and keep track of the experience that were served for regulatory purposes. So really, you need to be very careful and smooth out the kinks before trying to scale. Now, there are several ways to scale personalization in ways that are successful.

And, but broadly speaking, to achieve scale, you need to be disciplined and you need to use what I call algorithmic, but in a simpler word, which has to be automated approach, creating an individual experience for an individual segment by hand that works at the beginning that will not work at scale, trust me. So, of those different approaches, how can we use, how can we create those personalized experiences at scale? So, as I mentioned when I was discussing the problem, personalization can really save. We start about oh, we have this figment, we have this creative, and we’re trying to map creative and figment.

And so there are two approaches there, one where we start with the creative, one where we start with the figment, and finally psychometrics is kind of a slightly different, more advanced approach that I do want to touch on a little bit. So starting first with the creative. Now, I would call the creative to even approach throwing spaghetti at the wall, knowing that it is not necessarily a bad thing. And if it’s, well, if you are able to throw a lot at the wall and you get enough things that stick to the walls, then you’re good.

You’re surviving. And so the way that would work is that you have some CMS and experimentation solution and system can handle multiple variants. And so instead of having an A B test they can really handle a lot of different creative and then algorithmically, automatically over time, identify which characteristics of customer. predict a good performance of a creative and then Determine like oh, we are finding that people in california are responding better to this creative So we’re going to serve it more to california customers On the other hand, we’re finding that this other creative is resonating better with customers, say, in Ohio, and so we’re going to serve this creative to, say, Midwest customers and people more in the central states.

Uh, now, the benefit of taking a creative driven approach when you’re using a CMS or an experimentation solution is that You don’t need to have, quote unquote, that much discipline. It’s that whenever you’re thinking of new creative, you can just add them to the mix and the system will automatically figure out if this new creative is outperforming some other creative for some segment. And then they will usually, usually tell you, oh, this creative is not outperforming for any segment. And so then you can just remove the creatives that are not useful.

Uh, a second benefit of using this approach is that if there are shifts in customer segment. And so for example, your customer base was originally in the East Coast, but then you are opening or advertising more in, say, Texas, then the algorithm and the solution will automatically adjust to shift and say, Oh, we’re having this new segment that we need to, Personalized for and so they will it will work on that and similarly if you have changes in customer attitudes and behaviors like just trends and fads and fashions and The solution will handle them. So These approaches are pretty popular because they have a lot of benefits.

And I think that is really the way that a lot of marketing people in particular approach them, but you also have to be aware of some of the disadvantages or the cons of this approach. One thing is that because it is creative driven, there is no segment that are created in a creative way. In general, that reflects some deep customer behavioral characteristic. It is just, oh, this creative performed better in this state, or for customer of this gender, we’re going to show it more to them.

But then for another creative later in the website, the two options, the segmentation there might be by age instead. And so you’re getting essentially Segmentation at each point in your website where you’re having multiple options, and you’re not having kind of those consistent, durable segment that will tell you a lot of information. about customers. And because of that, that makes a creative driven approach more of a black box, which can make debugging and data analysis more difficult in the sense that if someone encounters a bug, you have to first Kind of figure out what was their experience through your website or your project to be able to identify And replicate and hopefully fix the bug which is why another approach is to start with segment and typically the way that again more marketing, but also data science or analytics people will think about creating the segment is by using clustering algorithm or simply by using segmentation that is splitting young, old, male, female, west coast, east coast, and, and so on midwest and so on.

And so, Really splitting the customer base across characteristic and decision trees. So, to building those segments can be again, really easy to implement which is really appealing. So you just say, again, you can just split by age, gender, location, and so on. And as opposed to the black box of taking a creative approach, here we’re having more of a glass box where every segment has a very clear definition and that makes it much easier to understand what’s happening when an experience is served.

Now obviously some of the disadvantages of this approach is that you can build those segments, you can build however many clusters or segment as you’d like but it’s not necessarily clear. If they are at all relevant to your, um your company and to your website. So, for example, you might say, well we have some male customers and some female customers. And so we’re going to create a separate segment, but maybe it turns out that your brand or your products are really not, um separated by gender and For example, it would be much more relevant to segment by introvert, whatever their gender on the one hand, and extrovert.

whatever that gender on the other hand. So when you have this clustering or cluster or segment, it can be unclear if they are relevant and how to engage with them. And as you are breaking down your customer base into more and more characteristics and like segmenting by age, gender, by state, by attitude, by behaviors, then you’re getting Uh, complexity increases geometrically, and so you are really, you can really end up with a large number of a large number of segments. And also, you might have some difficulty to generalize across segments.

So, again, to stick with my example, you might create a creative form. young male customers, but now you’re like, well, if I have this creative for young males how do I create a creative for older male or for younger female? And so you’re having a harder time generalizing across segments, and really, you have to make new creative segment by segment. Which is not to say that this approach cannot work at scale.

And so, one example of taking this approach at scale was Build A Hat, which Did a segmentation exercised. And so one of the benefit of the situation was that they had information on some of their customers that was not just what they were purchasing, but also in their online account. What was what preferences did they express and what behaviors did they show online? And so that really.

Um, the creation of a large number of micro segment. So the numbers are large few people in each segment, but them using the, an algorithm approach was really successful at defining different engagement approaches. timelines and so on in for each of those micro segment. So again taking a segment first approach can work even at a large scale, but it gets more complex and you have to be much more disciplined because there is still a lot more manual work and judgment involved versus the what I have found to be the case is that with the creative first, it is easier to just put things into the system and see what works.

Now, these two approaches I have described, starting with creative and starting with judgment. Those approaches are more traditional and you’ve probably encountered them in one way or another. But I did want to talk a bit about one part of my journey that was much more novel, which was to try to use psychometrics to develop a really targeted Personalization. So, when I talk about psychometrics, what is psychometrics?

So, etymologically, psychometrics means the measurement for psychology. And so, it is how to measure psychological trait. And so to connect the bits and dots, the question of like, well, you have the big five model, how do you know where someone falls in the ocean characteristics? You would ask them a bunch of questions like, I tend to worry about things.

They disagree, disagree, and so on. And then what you’re getting is using a psychometric model such as the, the graded response, model. And so if you look at the curve on the right of the screen, really the idea is on this the, the implicit scale for the, the bottom of the setter here is neuroticism. And so if someone said they strongly disagree, I do not worry about seeing.

This makes us infer that they are very low on neuroticism. And then as they their response get stronger and stronger toward agreement, our expectation of their level of neuroticism increases up to strongly agree. And I mean, obviously a single question does not allow us to fully determine where someone falls on a scale or on a trait. But as we have more and more of this question, then the psychometric model allows us to be more and more precise and assign a specific value for this trait to this customer.

Now, you might think, okay, this is interesting about the application of the Big Five model. But what does this have to do with personalization? Well, the, I one of the challenge I had was to think about automated online payment. And so, in that case, there was really nothing in the big five or the existing literature.

That could help me think about, well, if I want customers to sign in for automated online payment, should I target extrovert or introvert customer? There was really nothing in the literature to that effect. And so, what I did instead was to think about what would be the relevant trait. And so, The trait, the psychological characteristic that we created was digital ease and so trying to assess based on customers behavior and stated preferences how comfortable they are with digital technology and using the information we had about them To kind of assign them a value of, in terms of digital index.

So one of the examples of the variable that we used, it was in terms of their email address domain. So if you have someone whose email address is aol. com, that tells you something about them that is different than if their email address is gmail. com protonmail.

com or any other email provider and so now you can use those behavior as categories for in a psychometric model to build that index. So one of the benefits of this approach is that it provides a good interpretability and so you can see for instance, oh this person has this operating system on their phone And so that that’s what it means what it is correlated with in terms of digital ease And another benefit is that it is easy to reuse and so in the case of the digital ease index we were able to You Uh, score all of our customer along that index and then have that variable in our customer data system and to be used as for example, as a variable in data science model and so on.

However, obviously this is a much more advanced and technical method, so it means that it is really much more something for a data science team to implement, and there are fewer And resources available. I mean, the resources I used, which I found to be very helpful, even though it was not targeted for business users, was this book. And so in this case, From, Hey, I have this data frame or this data table and build fitting gm. So a greater response model in our, it’s can be as simple as two lines of code.

Obviously there’s a lot more work and thinking that goes into. Which variable should we include? Determining the fit of the model, determining the validity of the model. But yeah, that was kind of an interesting and pretty novel tech I had on personalization to help when using the creative first or the segment first approach was not enough because the problem was You too far away from personality psychology.

So to recap, personalization can often be seen as kind of a magic wand and it’s really important to be very concrete and practical and to think through Which is the right approach in our specific context in terms of the amount of data we have in terms of the type of business problem we’re trying to solve And what term of the technical skills we have available especially because as you scale complexity can really broke down your personalization program and really interfere with it. And so that’s that has been in my experience, the big driver of like, well, maybe using the creative first approach is not the perfect approach for our problem.

But it is very easy to use with the tool we have at our disposal. So, let’s start with that. And my final piece of advice regarding personalization is to always keeping in mind business value. Because it can be really easy to go down a rabbit hole and to think, oh, what if we add one more segment?

What if we had one more creative? And you can really get into diminishing return and start working harder and harder to get less and less additional business value. Personalization can be really powerful, but you always need to keep track of, well, is it young? improving, is it still better?

So thank you very much for your time today and for listening to this recap of my journey through personalization. If you want to follow me, I’m linked in and I also write a post about on Medjam about behavioral science and Behavioral Data Analysis. I also have a book about Behavioral Data Analysis with R and Python, which is available on Amazon, as well as the publisher’s website. So, thank you very much for your time.

And I will be available if you have any questions.

Divyansh Shukla: Thank you so much for that amazing presentation, Florent. It honestly felt like just, you know, We just took a deep dive into the fascinating world of psychology. Today’s discussion highlighted key elements, in fact, that are essential for running a discipline and an impactful personalization program. While I have many questions swirling in my mind, I’ll focus on a few.

So from your experience, Florent, what are the ethical considerations that arise when implementing personalization at scale, particularly in balancing user autonomy with targeted interventions? So what are your thoughts on that?

Florent Buisson: Well balancing, I mean, It’s a tough question. And again, I think it really depends on which team is driving the effort and what tools they have available. And so, for example, I have found that for the same problem, a marketing team with certain tool with approach it very differently from a data science team with a different tool. And so I, I think that it’s really more of a case of, well, what is the best approach in your particular case and what you have available and trying Again, as much as possible to go from the specific of the situation to the broader concept of like, hey, how are we thinking about this?

Is this adding a value? Those are the two principles I would try to apply to every situation, but then just go with the specifics of your situation.

Divyansh Shukla: That’s true. Indeed. There’s no one size fits all. Uh, but it depends upon the, the context, the user story, and also what the segments are telling about the audience.

So definitely. Thank you for your advice there. Uh, Moving on to another popular question is around measuring the long term impact of personalization, especially on brand equity and customer lifetime value. So how do you measure the impact of personalization beyond conversion metrics and in fact, traditional conversion metrics?

Florent Buisson: To be honest, I’m not really sure you can. I think that and I mean, to be fair, it’s also hard to measure brand equity in general. And so I think it’s more a matter of keeping track of your broader metrics. So, for example, the.

One of the very common metrics that is tracked is the net probability score, NPS. And so, keeping an eye on how your NPS is trending over a long period of time, and when possible, if your platform allows you to do so, tracking is how is our MTS trending in this segment and that segment, but again, there will be so many factors impacting these metrics and those segments over time that I would not obsess too much over that and just keep that in mind as like, Hey, Our NPS is lagging a little bit with that segment. Maybe we should So to speak put a bit more gas into the engine Or it’s like well our mps is really good with that segment What can we learn from that?

But yeah, again, that is it is, this is not an exact science and it’s more a matter, there is still always a lot of judgment called involved when making those measurements in the long term.

Divyansh Shukla: Yep. I think it’s important to keep a close air to your segment throughout the life cycle, be it from the get go towards the very end. Uh coming on to the next question and in fact, we like, I’ve heard this a lot. How important is the organizational culture in cultivating a successful personalization initiative?

Is the buy in from leaders can foster a different environment? And how do you ensure that the continuous innovation in this space is, is happening? I’m really, I really know that your experience here would be valuable for our attendees.

Florent Buisson: Yes. And uh, I mean, one of the big challenges again is as I touched on, is expectation management in that by and large, uh. In my experience business leaders often have unrealistic expectation in term of the timeframe where something will happen. And so I think a really good approach when possible is to start, as I said, a manually under the radar and just talk to your boss and say, Hey, I have this idea for this particular case.

Maybe we can try the low key low stakes and see how that works. So that Hopefully, when six months or nine months down the road, one of your VP comes back from a conference and is like, I heard this guy talking about personality psychology and personalization and it was fantastic. We need to do that tomorrow. You will have done your homework and you will have some understanding of the problem.

So I think the really trying to do things under the radar can be really helpful at first and then I think the one of the key pieces to keep your senior leader engaged and motivated is to provide regular update and say, Hey, we went from two segments to five segments last month. And here is the added value that this has created and just capitalize on that. And also, Communicate, communicate, communicate, talk to people across the organization to let them know what you’re doing because you never know what Team will come up with a great idea and that’s how you can also build allies and people who will support the program.

Divyansh Shukla: Thank you for such a detailed answer, Florent. I totally agree that often the best results are unexpected. And when there’s not that big a shebang surrounding the buildup of a personalization program, then actual results will really be helpful for a lot of folks to kick off the process in general. Uh, coming on, in fact, to our last question of the day Florent.

It’s become a tradition, in fact, at Convex we do ask our speakers to share a few book recommendations. I do, did see that there were already a few great recommendations shared throughout the presentation, but building on top of it would you mind to share any further reads that the audience can browse up to?

Florent Buisson: Yeah Yeah, I, I, I mean, I, I’m all the way really, really big book nerd and always sharing books. And so I think two books I would call out that because people may not be familiar with them. So the first one is Working Minds by Beth Crandall and Klein. Uh, who can’t remember his first name, but Klein.

And so this is really a deep dive into cognitive task analysis and understanding how customers think about your product. And this has been a fantastic resource. And the second one, and I’m going to Paul is from here to get the name of the author. This is The Art of Game Design from Jesse Schell, which is not just for a game designer, but it’s really a fantastic resource to think about how to engage your customers and, yeah, be a bit more playful and really get them to drive behaviors in your website as you attempt new applications.

So those would be my two recommendations.

Divyansh Shukla: Thank you, Florent. I think I’ve sorted um my weekend read. for this week. So thank you so much for those recommendations.

Uh, thank you for all the listeners who tuned tuned to join us today. And thank you, Florin, for such an amazing presentation. Your advices were really crucibles in running a disciplined, structured personalization programs. And do write to us, folks, if you’d like to build up on this conversation and would like any questions answered by Florent.

With that thank you so much and uh, see you shortly probably on another session.

Florent Buisson: Thank you for having me. It was a pleasure.

Divyansh Shukla: Awesome. Bye. Bye

Florent Buisson: Bye.