Building a Culture of Experimentation in the Times of AI

NOTE: This is a raw transcript and contains grammatical errors. The curated transcript will be uploaded soon.

Welcome, everyone. Really excited, to talk to you today, with Andy Boyd about building a culture of experimentation in the times of, AI. Obviously, a lot has changed, and where I think we wanna go with this session is introduce how to bring that culture into your existing teams in a way that has positive impact up, down, and side to side.

So that will be what we are chatting about today. Myself, I’m I’m Matthew Mammoth. I’ve been doing growth for twenty five years. I started as a entrepreneur and successfully exited two SaaS businesses earlier in my career. Moved on to scaled organizations at TripAdvisor where I helped grow the hotels division of TripAdvisor, first, by a hundred million increase in their price metasearch project, and then eventually building a new ecommerce product from zero to two hundred million run rate business. And since then, have replicated that growth work at places like, EnergySage, where I was most recently chief product officer, and also at Spark Networks, a collection of online dating sites where I was the interim chief growth officer.

Lately, I’ve been a fractional growth leader, and I’m excited now to apply all that I’ve learned to a whole host of companies.

Also with me today is Andy Boyd. Andy, why don’t you go ahead and introduce yourself?

Andy is author of the enterprise growth playbook, ex IBM and Watson, and and AppFire. But without further ado, Andy, why you tell us a little bit about yourself?

Well, it’s it’s great to be on here, with you, Matt. I enjoy our conversation outside of the podcast, so it’s fun to do this in a virtual recording. It’s great to collaborate. So my background really quickly is it’s always been in product.

So I started my career working in startup companies. Actually I started doing digital marketing and product management, really focused on launching new products. It turns out if you do those two things, you’re probably gonna be well positioned for growth. So that’s where I started my career.

I then was fortunate to be part of the team at IBM that brought Watson to market. As Matthew, as you said, one of the things that I did at IBM was I launched the first growth team for IBM Watson. That was kind of borrowing from my earlier career.

That was a very much a new new thing at IBM Watson. We didn’t have growth teams. So it was a lot of fun. We had tremendous success. I had the opportunity to do that then with about twenty other teams inside IBM, and that’s where that book that you mentioned came from. It’s just industrializing that process of how do you build a growth team, and more importantly, how do you create that culture in which growth can thrive?

And most recently, I was the CTO at AppFire, a business that provides apps that allow you to customize and extend platforms like Jira, Salesforce, Monday dot com. And we put in place many of the practices that we’re gonna talk about today in building a growth team.

Awesome.

Andy, I’ve always enjoyed chatting about product and growth with you. I’m really excited to bring it to the to to the session here today. And one of the things that I I thought about when I when I instantly thought of you to to join me here is we’re both we’re both old guys. Right? We’ve we’ve seen it. Right?

I don’t I don’t wanna overstate that, but I also don’t wanna sugarcoat it either. And our experience both in product as well as growth together is is really what makes this particular session, I think, interesting. And one of the things that I think both you and I talked about in the past that is what we’re gonna really lean into today is that tools don’t really build a culture of experimentation.

It is the people and the processes and the disciplines that are installed that help people really grow into a true culture of experimentation, help those organizations grow.

So let’s start by going way back. As I mentioned, we’ve, you know, we’ve both seen some things in our career, and and I wanted to start with the basics of this this, like, what are the roots of experimentation here?

Tell talk to us a little bit more about what you specifically learned while growing some of those growth teams at IBM, and then I’d like to share a little bit of how I learned how to do that at TripAdvisor back in the day.

Yeah.

I mean, it’s just like you said. It really it’s not about, like, the tool stack that you’re using. That’s important. You need that. But it’s really about the culture you create in the process. And I think one of the things that was unique for us at Watson when we built the first growth team was just this idea of learning velocity.

You know, I think the real magic in building a growth building a growth team that’s successful is compounding interest. Right? Getting a little bit better every day. And so I think for us when we first started, that was always something that was very core to my belief. What you need to do to build a successful growth team. I I experienced that in the startup days of my career.

A company like IBM, they certainly understand how to get a little bit better every day. But by and large, the way that a company like IBM grows is they’re a huge company, and so they’ve gotta make big bold bets because those are the things that are gonna move the needle when you’re operating at, you know, billions of dollars of And so it wasn’t necessarily in the culture. The culture was if you wanted to grow if you wanna grow the business and therefore grow your career, you had to like kinda sit here today, get a forecast years into the future and say, I’m gonna do this thing that’s gonna be ten x better by such and such time frame and then you go and deliver that.

But as as you probably know, just simple math, I sit here today and I say I’m gonna be ten times better off of a base of one by the end of the year, I’ll be ten times better. But if you say off of a base of one, I wanna just get one percent better every day. At the end of the year, I’ll be thirty seven times ahead of where I was. And so that’s kinda to me the mental model is like, do we get better every day?

And so I think really a a big part of building that culture, and I know we’re gonna dig deeper into it, but a big part of building that culture is how do you set up a system to just get a little bit better every day?

So some of the things that we did was faster cycles. Like we met weekly, we would review what we were doing, and we were launching multiple times per month. Building that kind of weekly cadence to just keep releasing things. I think some of the other things that were really important were building culture of of data driven and transparency.

Let’s So that was something there because the I I don’t I don’t wanna cut you off any, but we’re you’re really hitting on something that I think is super impactful, and I think we want our audience to to just pause on this and and really, hopefully, internalize it. Because I had a similar revelation earlier in my career around the importance of learning velocity and always testing.

And I I like to tell a little a short little story here of how I learned how to do that at TripAdvisor. I I joined the business as a young growth guy, growth product guy in in I think it was May or June of twenty thirteen.

And and I remember four to six weeks later, it was middle of July, and and Steve Coffer, the founder and CEO, swung by my desk and said, hey. I saw the the this last week’s learnings report, and there were no tests run.

What’s up? And I said, well, it was it was it was the holiday. It was July fourth. We we all kinda, you know, took a few days off here and there, and and we’ll just pick it up again this week.

And, you know, Steve kinda paused and thought about it for a second and was like, yeah. No. I don’t I don’t think we’re gonna do that. I think we’re gonna test something every week.

And so, you know, that was his polite way of really coming from the top down, instilling this culture of always learning and always, yeah, increasing things by one percent. In a in a scaled business like a TripAdvisor or an IBM, one percent over that base is meaningful. If you can log in one percent wins every other week, you’re gonna be a hero. You’re gonna be a hero.

Absolutely. So I think I think that just embracing the discomfort that comes with not really being sure if the test is solid or if it’s gonna win or what will happen if, you know, the marketing purse the CMO sees it and doesn’t like the approach. All of that stuff, I think what I learned and I think what I’m hearing you say is you put that to bed, and instead you just focus on the learning. And then the business can decide what to do with each of those, you know, results.

If if it if it worked and everyone’s happy with the the treatment, great. If they’re not happy with the treatment, great. We know we could always go back to this treatment in the future if we decided we wanted, you know, that lift.

So I I appreciate you taking a pause there because I think that learning velocity really is step one. It’s rigor of testing something every week or whatever your your cadence is and not missing a beat. Yeah. That that’s that, I think, the foundation for all good experimentation systems.

Absolutely. And I think we should probably talk a little bit about, like, KPIs because I think you have some good stories to tell about that. But I think as a growth team, doing those incremental experiments, like, real magic is if you’re in the growth team and in the day to day, you’ll see that result, and you’ll get excited about it.

But when you show other people a business, they may not realize kind of the magnitude of what that could become. They don’t get excited about one small improvement.

What you really have to be able to do is show the compounding improvements on a metric over time, and that’s when the story gets really exciting. Because I think you have some good stories about KPIs and incentives about really making your growth team successful because that’s what the business is really gonna care about.

Yeah. I I think the the if one of the pillars is the rigor around testing, then another pillar is aligning incentives and and focusing on KPI improvement that matters.

If we’re if we are moving a number that matters to the product management discipline or the engineering function, but really doesn’t matter to the other functions, those are often viewed as, like, pet projects that are easy to stop. Yeah. The the if if we’re starting out with trying to build a culture of experimentation where none exists, focus on conversion and revenue. Right? That’s the KPI that will get everyone’s attention.

And if you’re if you’re able to make a meaningful impact on that, which by definition you should be if you’re starting with not a culture of experimentation, that really means you have an unoptimized surface. You haven’t run any tests. Yep. So there’s probably a lot of low hanging fruit to so pick the things that will help you drive the KPI that is the leading indicator to revenue.

At EnergySage, it is a lead generation model. And so every homeowner who filled out the form that said, hey. I’m interested in going solar. Please give me a quote.

That had an immediate impact to to to the revenue line. And coming in to to EnergySage as the CPO, not a solid culture of experimentation to begin with, wanting to tell that story, I laser focused on that. I laser focused on let’s get more homeowners to raise their hand with us and get put into the quoting system.

And that was a good use for for AI as well. So once you have your kind of target of this is the surface that I wanna change, this page, this mobile experience, this whatever, here are the KPIs I wanna move, you can use, an AI tool like a ChatGPT or Perplexity or or what have you, Gemini, to help brainstorm some of the tests that you might wanna run.

So if you don’t know where to begin and and you don’t have a good, thought partner or team of thought partner yet because by definition, we’re trying to build from the ground up. Use ChatGPT as your thought partner. I would stop short of saying, hey. What do you think I should do next, ChatGPT?

And then, you know, you’ll you’ll be you’ll be led astray. But if you can use that tool as a extension growth hacker, growth marketer that’s on your team, it will help you accelerate the job of defining tests, launching tests, and analyzing tests, which will help you with your learning velocity. And and so I know, you know, Andy, you’ve been a big part of the AI ecosystem from from day one back at the Watts in in Watson. How have you started to see growth teams use AI toolkits or copilots or or prototyping tools to reduce the burden of coming up with tests and running a rigorous high velocity learning system?

Yeah. So I great question. I’ve really seen two key areas where teams are using AI.

So, like, in the macro sense, if the if the magic for building a growth team is getting one percent better every day so you can get those compounding gains. There’s really, like, a couple places where teams can get a little battle act. The first is just sometimes it’s the resources to be able to deploy those experiments.

So, actually, I should say it differently. It’s probably the ability to deploy an experiment or a test, thing one. And I think sometimes it’s teams are using AI to do, like, two parts to that.

When we’re a product or we’re growth for a marketer or whatever, we’re having to describe in words, hey, design team. This is what I want. Hey, engineering team. This is what I’m looking for. And things get always sort of lost or elongated as you’re communicating in different languages.

So the the first thing that the AI team’s doing is they’re actually communicating in the next team’s language. So if I’m a product person, maybe I can prototype a design that makes it faster for an engineering team to implement. Or if you’re working with a growth type designer, they can actually write some code that makes it easier for the engineering team to implement, thing one. I think thing two, sometimes as you work in the businesses with massive volume, like really small changes can have big impacts.

Like the heretical example is changing the color blue on Google homepage. Like I didn’t have that experience. Our businesses didn’t have that kind of volume. Yours might have.

But you could use AI to make those small treatments and small changes really quickly. It can actually write production code. So that’s thing one is just using AI to communicate the language of the person that’s implementing or even sometimes implementing.

And and let me pause. Let me just jump in if that’s okay. Because I you know, it’s such a powerful observation given that teams that are trying to create a culture of experimentation don’t have a dedicated pod. Right?

Most likely. They do not have a growth mindset group of individuals. You mentioned some roles like the growth designer or the growth PM. Those are different than your core product designer, your core product PM, your your your core engineer team who are trying to build things the right way Yeah.

For scale. Whereas your growth team is embracing the discomfort that comes with testing a whole bunch of stuff, most of which is gonna fail from a perspective of moving the KPI. Right? And so getting bogged down in these long communication cycles of, like, well, I think it should look this way, or I think it should act that way, only to find out that eighty percent, whatever, sixty, eighty percent of the time well, it didn’t move the KPI anyway, so we’re throwing the whole thing out.

Yep. And it’s incredibly wasteful for both growth and core. And that actually kind of, like, it dampens the excitement around building this new muscle called growth and experimentation. So if you can take these tools and reduce the resource need and produce results, suddenly, you’re creating this beacon that everyone starts to follow.

Yeah. That’s the so we’ve talked a little bit about the science of this. Now now we’re kind of getting into the art of it of Yeah. Showing your work, showing how you got there, showing that you do it every week, and suddenly resources start to, like how how do we make this process go faster, better because it’s it’s yielding the results they that we want?

And you would prefer not to, like, irritate the entire design and marketing team with with, you know Yeah.

Matthew’s out there in Lovable all the time. And that’s how you start to start to use AI to accelerate your own efforts, show the impact, and then start thinking about scaling and sustaining this. Right? So, you know, kinda growth hacking the entire culture development is is a big is a big part of it in the early days. What are your thoughts on that one?

Oh, well, I I actually wanna go in, like, a slightly different direction because you hit on some Yeah. That was really important.

So you you talked about, like, experiments fail That’s something we haven’t talked about yet, but it’s really important in that culture.

Yeah. I think I don’t know if this was your experience back to you, but in a lot of companies, if you’re a a product manager, not growth, set Brent over to the side for a second. But if you’re a product manager, you wanna declare what you’re gonna do, and you wanna drive business results and show success. Like, there’s only winning. There is no losing.

However, if you’re really doing growth the right way, you gotta look at the wins and the failures because that’s ultimately you got both sides of the equation for learning. That’s the kind of culture that you need to create. And I remember during my time when we were doing this at Watson, you know, that that was very much the corporate culture was declare what you’re gonna do, do it, win, and that’s way you grow your career. You didn’t really wanna talk about failures.

And but, like, that’s core to being able to drive growth. Right? The wins and the losses. And I remember one time we were going into a meeting with somebody who’s very, very senior, and we were presenting some of the work that we did.

And I had some slides where I was like, this one won, this one lost.

And one of my men was like, don’t put on there. I’m like, but but you really have to because that’s the way that you’re gonna drive. Girl, now I knew that this very senior leader came from a digital background, so kind of understood the winning and losing.

But my manager at the time is like, don’t put that on there. And I remember in the meeting, I’m like, I’m gonna put it on there. And I remember in the meeting, I’m like, these ones won, these ones over here were crap tastic. I actually said craptastic.

And then afterwards, my manager was like, I can’t believe you used the word craptastic with that person. I was like, but it was pretty funny. Right? You know?

Yeah. I was trying to create a culture. I mean, it was fun, but I was trying to create a culture where we celebrated the wins and the losses, and that was really important to being able to build that culture. And I know you have to have some stories about that as well, Matt.

It’s all about optimizing for learning. I think that’s that what that’s what that story really drives home for me is that we want to to learn with every experiment that we make. Sometimes we will learn how to have an immediate positive impact on revenue and everyone, you know, celebrates. If it’s a big impact on on revenue, then, you know, maybe there’s a, you know, for for for for dinner and and drinks afterwards and celebrate on the company’s dime. But but you’re right. Those happen rarely.

And what what often is the day to day is missing on the KPI, but then learning why why you missed. Yes. And so bringing that that mentality into the meetings when you’re managing up, I I gotta figure Andy was, like, critical to installing this, culture of experimentation, because that that that that that is what you’re doing. You’re just saying, we are gonna create a culture within some component of the business where it’s a lab, and we’re gonna run experiments.

And sometimes those experiments will be eureka moments, and sometimes they won’t. And so when you’re talking to me, be prepared to to to learn, and learning can be hard and and ugly at times, craptastic even. And so for cultures like TripAdvisor where that was really baked in, I I didn’t often have to explain that, which was which was great. For other cultures and teams where I was bringing this in, I I think the first of all, the the describing things as winning or losing can can kinda backfire a little bit Yeah.

At you because of the all the reasons that you described. And sometimes it’s better to say, these were the results of the tests, and this is what we learned from the test.

Great insight.

You know, the the the results were conversion dropped by twenty percent. And what we learned was by moving the button or by introducing a video playback, more people clicked on the video than actually did the thing we wanted to do. Yeah. So that’s what we learned.

And That’s a great One of the things that Steve always taught us at TripAdvisor is it’s okay to make a mistake, but let’s not make the same mistake twice.

So take that learning, bank it, share it. You know, a scaled business like an IBM or a trip probably has a few different growth teams, nascent teams out there.

So what we learned from our hotels testing, we always made sure the tours and attractions, the vacation rentals team, all these teams had that information so that we could, yeah, increase the win rate for all of us. So learning, documenting those learnings, and sharing those learnings helps you build this culture of experimentation.

I I so great insight. First of all, great insight on sharing it as a result versus the win and loss. Like, that’s people are listening out in the audience. They should write that one down.

That’s really good. I always told my team, you know, when they would present an the result, I would say it’s okay if it didn’t hit the hit the goal. Like, it’s okay that it won. It’s okay that it didn’t win.

Sorry. I’m reverting back to some of that old language, but but what is never okay is you can’t sit there and not have an insight or an idea of what to do next. That is the thing that’s unacceptable. Right?

I think one of the other things that we really institutionalize in a couple places that I’ve been is this idea of shared learning. And I think that’s also real a really important job that the growth team is doing is to institutionalize that learning. You know, I think about, as you said, your CEO told us, we can never make the same mistake twice.

Well, like, that’s really hard because if you’re the person that took the action, you’re gonna learn the result.

But sharing it is really hard in different sized organizations. Now we had a process that we used during my time at IBM and facilitated that in other roles where we had playbacks and we had documentation and kind of vignettes and snippets of here’s what we did. This was the result. And we made a repository for people to learn from. I’m wondering, Matthew, if you had other techniques of how you shared those results, particularly in a company that was really that was part of their operating system.

Yeah. You’re right. And and you’re building those learnings in a way that are digestible will increase the chances that people engage with the content. Everyone’s very busy. Right?

Yeah.

And one of the things that I I think was a simple a simple change that had a really powerful impact was those those those meeting cadences that you described, whether you call them a playback or a learnings review, make those be a very broad invite, and everyone’s optional.

Right? So there would be meeting requests that would sometimes go out at at this company, TripAdvisor, that had something like a hundred people on it, and they were all marked as optional.

And then then the the meeting would would happen. And, you know, to start, maybe you’d have ten people show up. Just curious, eating their lunch, half listening while they’re while you’re sharing your your learnings.

But if we’re consistent with that velocity of learning and that rigor of learning, eventually, you’ll learn something that’s really meaningful and impactful, and it gets sent to a hundred different people. And then suddenly, others will will say, like, maybe we should be paying more attention to what’s going on over there. Like, these guys, these people, these folks seem to be onto something. And if at that moment, you have this repository that you can share with them that’s, like, easy for them to get up to speed on on what you’ve learned over the past six weeks, definitely increases the probability that the culture starts to to scale and and and get baked in.

And these days, this is a great use of AI. Right? You you you you do all this work. You you run the test.

You have the data that comes back. Packaging this up to, like, what did we test? How what did it look like when we tested it? What were the two different variants or whatever?

And what were the results? And why do these matter? And and clipping, all of that stuff, that’s a lot of work. There are tools out there that do this for you now.

And so the the long story made short, the the creation of the agenda for your weekly learnings, meeting can either be, we’re gonna talk about test one, test two, test three. That’s how it used to be. And then you’d have a team that’s, like, creating all the materials. Or you could say, here’s the here’s what we learned last week.

And bite sized, you know, learnings or playback packages that are already created by AI tools, and then you use that meeting to discuss. So the discussion is where now you’re starting to really scale and sustain, this culture of experimentation because people have jumped over the hurdles of what are these guys doing? What are these people doing? Why are how are they doing it, and what are they learning?

You kinda jump over all of that and provide a lot of transparency around it.

Yeah. I mean, I think to me, one of the signals that I’m looking for, I’d like to say, is is the culture changing? Are we becoming more experimental? Is like, I’ve seen this in a couple places, but for a culture again, TripAdvisor for you, that was just part of how the team worked the teams worked.

You’ve been in different situations. I’m in a situation where that wasn’t the way that it worked. It was we’re gonna create a growth team, and they’re gonna start to do some things and drive some experiments, drive some results. And but what you really want is everybody starting to think like a growth person, like thinking about definitely, you wanna think about longer term goals and strategy.

Absolutely. But then you also wanna think about what are those things that you’re gonna do incrementally to just keep improving. And so to me, one of the big signals when that starts to take hold is really this. You have your broad team.

They’re starting to do some work. They’re running some experiments, having some results. You start to have the weekly learning interviews that you described. People start showing up.

But then the when it starts to switch, there’s really two things that you’ll see. The first is when some of those other teams that are not doing quote, unquote growth, like product managers, marketers, they will start pulling in those growth people and saying, hey, here’s something I wanna do. Can you give me your thoughts on this? Because that that growth team now is really expert on how do you pull levers and move the needle.

They have a lot of subject matter expertise. So the first thing is the organization starts to pull them and say, give me your insight. They start to be a subject matter expert, which is awesome. The more important signal that I start to see, and I think I see you nodding your head, I think that I’ve seen is then it goes from tell me what you’re doing to actually show me how to do this.

The real the real transformation of the culture happens when those under product managers or marketers start saying show me the tools, show me the things, and actually start to work. And that’s when really the growth team can start to, in some ways, disbanding, infiltrate other parts of the business. But it’s I think growth is definitely there to drive a result. There’s no question about that.

But I think it’s also there to drive an organizational type of transformation.

Absolutely. I think that inflection point, I I’ve seen it, as you’ve seen it too. And and that is the moment when, I think you’ve you’ve started you’ve you can almost declare victory around creating experimentation. Sometimes I see a middle step that that comes in. And so for folks who might be seeing this step, that middle step is a sign of progress, which is before the other, you know, non experimentation squads start asking to be taught, they start lobbying in ideas. I know. You know, it’s like, I want you to test this next.

I want you to test it for me next.

You are the testing team. You know how to do experiments. I don’t, and I wanna learn, so you do it. So this middle step can sometimes it’s like you go from grassroots or or skunkworks to center of excellence, I call it sometimes, or like a almost like a mini agency model. They’re our testing guide or people.

That’s that’s a sign of progress, so we can we can celebrate that if that if you’re listening to this right now and you’re feeling like, jeez, this I’m getting all these requests. But you don’t wanna live there. Right? To your point, Andy, the the growth team needs to recognize that moment and shift their their dynamic to be more about, instead of mercenaries just running the projects to missionaries and saying, hey.

Happy to happy to test that. With the two or three people I have on my tiny little squad, I’m gonna test it, like, two weeks from now or two years even. But instead, I could take that time to show you how to set it up, how to how to run this experiment, how to do it like we do. And that is typically on the growth team to to take that initiative.

There will not be someone coming from above to say, hey. I want you to stop running tests and spend more time teaching people. Because by definition, at this point, you’re onto something, and you’re getting this attention from other parts of the business, which is a good thing. And now you wanna replicate your success across multiple teams.

What, from your experience, Andy, have you seen as, like, the best way to identify, like, the next team to start learning how to do experimentation? How do you how does one one kind of do that? You know, the amoeba has to split, right, kind of thing. Like, what’s the first split that you can that you you’ve seen work really, really well?

Yeah. I mean, I think there’s there’s two ways you could do it. The first is like, if you’re if you’re in a company of any size, they likely have a portfolio whether it’s it doesn’t have to be a huge one. It could be like three products and couldn’t be relevant.

It could be ten to fifteen. You know, who knows? But generally, if you’re in a company of any size, there’s some amount of portfolio, and there’s gonna be some products that are just the go forward products that are gonna drive more growth for the business. Could be they’re the biggest products.

You need them to drive more growth. It could be like a new initiative that you’re trying to scale up or maybe other parts of the portfolio are winding down. So the first thing is you’re really just looking for where does the business want the growth to come from. The second way that I’ve seen it again, it depends on the dynamics of the business, is you look for pull.

So there are certainly different teams that are more inclined because of the the person that’s leading that team, that product, whenever. They just think more like this growth mindset, running experiments, incrementally improving things every day. So if you don’t have the luxury of somebody saying, this is where we need you to focus and have that top down support, then the other path is you go find those people that are naturally like minded, and they’re like, I want what you’re selling, air quote selling. Right.

And they’re gonna pull you in and say, these are the things we’re trying to do. I need your help. This is this is the challenge, and they’re gonna just lay out what the goal needs to be. They’re gonna give you the support that you need, and they’re gonna help make you successful.

So those are the two things. But I I wanna hear your input on I think another thing that comes to mind is if you’re in one of these companies that in some cases you might have a budget or resource to go hire some people. But oftentimes, you’re not hiring. Oftentimes, you might be re repurposing is a bad word.

You might be pulling somebody in from another part of the organization to take on this mission. And so I’m you’ve worked with a lot of teams, Matthew. I’m curious. Like, what are the kinds of people that you’re looking for either hiring or if you’re gonna bring somebody in that’s already in the organization?

What are you kind of looking for for those great growth people?

Great question. And and you you said a lot of things that I’d love to unpack, but I’ll I’ll stay focused on the on the question at hand. It is it is difficult to build an a culture of experimentation quickly if you are relying on folks who have been voluntold to join the growth team. Yeah.

So that is a difficult challenge and one that I think, you know, the folks at the tippity top of the organization don’t always think of because resources are fungible most of the time in their mind. Right? You know, engineers here not busy working on a project. Let’s have that engineer work on this initiative.

The culture clash that comes from that can be it actually can be I’ve seen be incredibly damaging to everybody involved.

Yeah.

And and if we’re if we’re asking someone let’s pick the engineer persona for a second, who has spent their life building their craft around creating code that doesn’t break under heavy usage across a zillion edge cases, and we say, okay. There’s a new game we’re playing, and it’s called just ship it today, whether it’s ready or not. It’s it’s more than just very hard for that person to, you know, to do that.

You’re you’re speaking to their DNA, and you’re asking them to change their DNA. And in my experience, when I’ve seen that happen, there is a there’s a curve, and it starts with, I’ll give it a shot. You know? This is what the business wants.

I’m I’m a good team player. I like you, Andy. I like you, Matthew. Let’s let’s make it work.

And then there’s that that chronic fatigue that sets in, and the there’s like a cliff and a crash where, you start to hear things like, you know, Sally doesn’t wanna be part of this team anymore.

It and and maybe we should shut it down. And so at that point, we’ve conflated two problems.

One problem is we just didn’t find the right person who’s culturally a fit with this other potential problem, which is you’re not driving revenue for the business. And if you conflate those two problems, you can be dead on, you know, on arrival sort of thing. So getting people who love to move fast is incredibly important.

High motor is is I think how this is sometimes described in the organization.

You know, they’re they’re typically, I I think, sometimes, you know, a little bit more on the junior side because they don’t know what they don’t know, and it’s all very exciting. And they have lots of energy, and they’ve never been through a a, you know, a a three hour, you know, code review of of a sloppy code before, so they they have they have a little less scars.

And they’re a little more like, yeah. We can make this work. Let’s figure it out. I I’m excited. So, high motor, no scar tissue.

Know, if if the if the if the organization has been the the kind that’s penalized mistakes in the past and the person that’s been voluntold to join your team has been with the organization for a long time, they probably are gonna be reluctant to put them put their neck back out on the line.

So sometimes, it is easier to get started with a set of AI tools that can, you know, be an extension of yourself Yep.

With interns. An engineering intern or an MBA intern on the product management side ticks a lot of these boxes. And and so that’s a way to start in the skunkworks mode. As you wanna scale now, that’s when you go out and you I would find someone to join the team either on a fractional basis or a full time basis who is specifically an experimentation designer, experimentation engineer, because they will they will be not only used to this level of speed and and and and, you know, not winning all the time, but they’re actually super psyched by it.

They’re the types of people who look at the test results every single day even though you’re not supposed to until the experiment’s over It’s oh, it’s with we’re it’s up. It’s down. It’s up. Yeah.

They’re they’re engaged by the the spirit of learning, right, and and the unpredictability of of each experiment. But what what have you seen?

That’s a great question.

What I know we’re getting close to the end of our session here, I I won’t try to go too long. But I agree with everything you said. I love the insight about, like, how the organization could potentially penalize people in the past. That’s a great insight.

The real quick answer is I agree with the attributes you’re looking for. I also used to look for people that were an entrepreneur in some form. Like, one of the questions I would ask when I bring people to my team is like, have you ever tried to start a business? There’s something about people that’s like a just a there’s something about people that try to start a business even if it failed that they’re just they’re willing to roll the dice and try.

I always felt like that was also really important.

But you said something else that I really wanna touch on, which was, like, you talked about the organization penalizing people that made mistakes. And I think that I think our job as leaders is to reinforce certain behaviors from our level. I mean, if we are sitting at the top, you enforce them down. If you’re sitting somewhere in the middle, you enforce them up and down.

But our job is to really reinforce certain behaviors and make it acceptable because if we’re gonna build a system of high velocity learning, we have to change certain behaviors. And so for me, like I said, I was always focused on two things I would penalize you for two things. Like, you you should have an idea of what you should do. So I’m penalize you for, a, doing nothing.

That’s a problem.

B, if you do something and it doesn’t work out the way that you didn’t want, if you didn’t have a learning or a next step, that should be penalized. But if you’ve got you should always have an idea. You should always be willing to try something moving towards that. That’s the kind of behavior I never penalize people for trying something that didn’t work. But I think that’s part of creating that culture. But I’m curious for you, Matthew, as a leader, what are the kind of behaviors that you had tried to instill and create to create that environment to make those people feel really safe operating in a gross type of role?

Yeah. Great great question. And and I’ll I’ll give you my my first blush answer to that, and and then I I think we’ll have to leave it there and kinda move to to to wrapping this one up. Although, I think we could talk for another, like, two hours.

I I think that first and foremost, every mistake that is perceived that the team has made is is your fault as a leader. You did it. Got it.

Even if you didn’t do it, even if you were privately saying, like, oh, I I don’t like this. I think, you know, my experience or my gut is telling us not to do it, and the team convinced you to to do it, and lo and behold, it didn’t work or it caused some kind of kerfuffle. I did that. That was me. I’m I mean, I’m ultimately accountable to everything this team does, especially when something is perceived to be a mistake.

Two is the corollary to that. Everything that worked, well, that would team did that.

Jeez. We never would have thought to to to to go in this direction and describe our product in this way, and it’s resulting in all these extra benefits and leads and revenues. Who who came up with that? Oh, that that was a brainstorm that we had with the team.

I I didn’t. I don’t I don’t know much about it. So, I think that is important in in both directions. The team needs to know that you’re out there at the the tip of the spear.

If someone, you know, running the the complaint department, go see Matthew. Matthew is the chief, you know, of complaint department. And internally, you’re you’re the chief cheerleader as well. So that’s what I’ll just say there.

I’m sure there’s a lot more we could talk about. But, hey, Andy, this has been a great session.

One thing that I wanted to ask in closing is what what are you reading or watching these days, whether it’s related to experimentation or, product or growth or whatever? I I or even even, something that you’re doing, outside of, you know, professional, reading and and watching. What’s what’s on the playlist these days in your in your world?

Great question. The two things that are on my playlist, there’s a book I just finished reading. It’s called endurance. This is on the personal front.

Endurance. It’s about Shackleton’s attempt to cross the Antarctic, and it was just an amazing inspirational story. Like, the thing that’s like that I loved about it is, like, they did this so this is US based. Sorry.

But they did this so many years ago, and it was I won’t spoil the whole story, but they didn’t make it. They didn’t make their end goal successful, but they they had a bunch of hardship, and they survived in the face of unimaginable odds.

And, like, the story that’s so amazing to me is there’s a a part where they’re, like, about to kinda get to the point where they’re rescued, and they have to, like, steal this mountain with, like, tools, just basic tools, like, totally ill equipped, and they did it.

And then years later, people, like, in this day and age with all of our modern technology and climbing tools, etcetera, they tried to attempt the same summit, and they didn’t succeed. And I’m like, so talk about a story of grit and perseverance. So if you’re in the growth world, speaking of people that are, like, gotta do things different and do the hard things, like, this is an inspirational story. So I’d highly recommend it.

Second thing that’s on my playlist is, you know, this day in AI, like, I don’t think we yet really know what is possible with AI. So one of the things I’ve been personally doing is getting hands on and actually writing code again. And so that really helps me understand what we can do with this new age of AI. So those are the two things that are on my playbook.

What about you, Matthew?

Yeah. On the professional side, anything related to AI tools, I I wanna try out. I’ve I spent a good amount of time and a little bit of money signing up to all kinds of trials for for tools, and I I try to make that part of my learning agenda every day. But what’s really driving a lot of my attention is on the personal side, completely unrelated to experimentation and growth.

It is it’s just a few weeks after Halloween, which is one of my favorite holidays, favorite time of year. And so, we’ve been watching, I think it’s on Netflix, Midnight Mass. If you like horror, creepy, creepy, stuff, I would I would encourage you to to check it out. Sometimes it’s good to put down, you know, your your your culture of experimentation and rigor and velocity and and recharge and refresh with a little bit of escapism.

So highly recommend midnight mass for that if you’re into vampires.

Andy, this was great. As always, I I enjoy our conversations. I think we covered a bunch of topics that hopefully are are helpful to to watchers who are starting from square one. How do I I know where I wanna get to, but the organization is not you know, does not yet have that operating system that’s easy to plug into. We’ve covered a lot of ground on how you could build that. So thanks again for your for joining, and where can people find you outside of this?

Great. Well, first of all, I wanna say thanks for the great conversation. Always enjoy these. We’ll have to do this again soon.

People can find me on LinkedIn, of course. We’re also on my personal website, which is just Andy f Boyd dot com.

Awesome. And if you’re interested in getting in touch with me, I’m also on LinkedIn and Matthew Mammoth dot com.

But with that, we’re gonna wrap, and best best of luck to everyone starting the the long journey to create a culture of experimentation. It’s worth it, and you’ll you’ll really have made a meaningful difference when you get to the other side. Thanks all.