Design for Impact: Boost Business by Enhancing Customer Experience

NOTE: This is a raw transcript and contains grammatical errors. The curated transcript will be uploaded soon.

Hello, everyone. Welcome to Convex two thousand and twenty four by VWO.

My name is Vipul Bansal, and I’m the senior marketing manager at VWO. Thousands of brands across the globe use VWO to optimize their customer experience by gathering insights, running experiments, and personalizing their purchase journey.

Day two at Convex two thousand and twenty four is here.

I’m sorry.

And we have, amazing and interactive workshop, by Erin Weigel.

And, I’ll just invite Erin on the stage.

Yeah. Hey. Hi, Erin. Hey. How are you?

Going. I’m good. How are you doing?

Doing well, and I’m quite excited to be an audience to this to this workshop. I might be a moderator as well, but, you know, I really don’t have a job as a moderator. I’ll be mostly just, you know, replying to comments here, answering questions, taking care of it. But most importantly, of course, I’m going to be an ardent audience, to your to your workshop as well.

Excellent. But no trolling. Okay? No trolling in the FigJam board later.

Yes. And I think, we did not mention it explicitly, but now that you say, guys, there’s an interactive activity that is planned for you using the FigJam, board. I’m not sure how many of you are already aware, but it’s, you know, very similar to a Miro board. And we will be pasting the link to join the FigJam board where there’s, where Erin has prepared an activity for you guys.

And if any of you, were an audience to our workshops in the two thousand and twenty three edition of Convex, You may know from I mean, some of the workshops used my robot, so it was very interactive.

I see a lot of a lot of people have actually joined the workshop yet again. I did ask this question during, before actually going live, but I see a lot of highs and hellos.

I’m interested, very interested in knowing where are you guys from. I see there are people from Munich, Brazil.

So you can just let us know. There are people from Austin, Texas. Hi, Dorian. Hi, Ben.

Amsterdam, UK, London, NYC, Poland, Colombia, Spain.

From across the world. Across the world, Erin, there are people watching you right now attending your workshop, ready to learn the nitty gritties of designing for impact.

We have Alice from South Africa as well.

Hey. Hi, Betty. There of course, there are a lot of names that I see in the chat scroll, over there yesterday as well. Thank you so much for joining the workshop yesterday and joining in again today as well.

Thank you so much. So while the responses are pouring in, Erin, would you like to give us a sneak peek of what this workshop is going to be?

Yeah. So today, we’re gonna talk a little bit about research and how you can do the most impactful research in your organization to make sure people are working on the things that actually matter.

Yeah. So let’s just if I give away too much, then, like, what’s the point? Right? So it’ll be amazing.

So that’s that’s guaranteed. That’s my guarantee that it’s going to be amazing because, of course, Erin Wagle is delivering this workshop. So, of course, who better than who better to deliver this workshop than Erin herself?

Thank you so much.

Alright. Can everybody see my screen? Alright.

So just in case you haven’t heard everyone, user research is in a pretty big crisis crisis right now. And I wouldn’t be an experimentation person if I didn’t have data to back up, my my claims. Right? And the data that I have to back up the claims is that researchers are three point six times more likely to be laid off than engineers.

This is something that my friend, Bob, who leads the CRAP community, conversion rate analytics and product community, shared with everybody on LinkedIn the other day. So if you look at the data here, you’ll see yeah. Maybe in in raw absolute numbers, more engineers have been let go. But when you look at how small the, the researcher section is, the whole way on the other side, they’re a relatively small team that was disproportionately affected, by layoffs.

So we all know we need research to make an impact in our organizations today. But what we’re doing wrong and why we’re I think we’re disproportionately affected is because we need to actually draw a direct line from the research insights that we uncover directly to the impact that we’re making on the business. You really have to spell it out for business leaders so that way they can understand the activities that we do on a daily basis really impacts the bottom line of the business. Today, what we’re gonna cover is we’re gonna take a look at all the different types of evidence that you need to collect as a researcher to make sure that you’re being impactful and getting the full picture.

I’m not I’m going to share with you an impactful user research approach. We’re gonna learn a little bit about how to ask good research questions, how to pick the right research method based on the question that you need to have answered, and then finally, I’ll share a quick tip at the end about how you can most effectively communicate with stakeholders and people around the business to inspire action on your impactful insights. Alright. So let’s dig into the first section.

First is about evidence collection and research. Now research has pretty much started in science and medicine. It is the art of observing what’s going on in the world. And it’s very useful to steal this mental model from the craft of science, because it’s a very useful framework for us to think about what is the availability of the evidence that we have compared to the reliability of the different types of evidence that we collect.

At the bottom of the pyramid of the hierarchy of evidence, you have expert opinion, and you also have observational studies.

These two areas or these two levels are where most businesses exist.

Most businesses used to only operate on opinions or expert opinions. There’s a big difference between regular old person’s opinion and expert opinion. The difference being that expert opinions, they form their views based on having studied more reliable forms of evidence up the pyramid. Then you have the observational studies, which are things like, you know, collecting surveys and doing research reports and all that kind of stuff. And that is way more reliable than just somebody’s opinion because it’s looking at a specific situation.

And when you’re doing your research for, you know, observational studies, you can collect multiple types of data. You can collect qualitative data, which is information that is not expressed numbers, or quantitative data, which is information that you’ve discovered but expressed as numbers. And when you think about the most powerful type of information expressed as numbers quantitative data, you get to a rung higher on the hierarchy of evidence where a lot of businesses don’t tend to travel, and that is by running randomized controlled experiments. These are the gold standard of the type of evidence that we as researchers can collect. And the most common type of randomized controlled experiment is an AB test.

But once you’ve collected all the data from up and down the pyramid, then you’re finally able to reach the top rung, which is a systematic review. And a systematic review takes into account all of the evidence that has been collected both qualitative and quantitative to collect a full picture of what is actually happening so that way you can make a value add decision or make the most well informed value adding decision to maximize the amount of impact, positive impact that you’re able to have on the business.

So there you go. So our job as researchers is to make sure we traverse the entire hierarchy of evidence to collect as full of a story of what’s going on as possible to guide the business towards finding and mining value for your customers.

So now that we understand, what types of evidence we need to collect, let’s take a quick look at a research approach that would help us draw a clear direct line from the insights that we uncover to the impact that we’re aiming to have on the organization. And the way we do this is with a very targeted research strategy.

What we need to discover real problems. We can’t just make stuff up and throw spaghetti against the wall and hope that something sticks. We need to discover real problems for customers that when they’re solved, they’re likely to impact important business metrics. This is a very important frame.

So you’re probably wondering, well, yeah, how do we do that? How do we make sure that we’ve solved the right problems that are likely to impact the business metrics? Well, we do that by looking through the lens of business goals. So we do all of our research through the lens of important business goals.

So, for example, imagine you have a business goal to reduce customer service calls by ten percent by the end of twenty twenty five. This is a very clear goal. It is concise. It is specific.

It is time bound, and it’s actionable. Right? So, hopefully, your business comes up with very clear goals like this because then what you do once you have a business goal that becomes your lens through which to focus your research, you’re then able to examine customer top tasks. So then you start to focus in on the top tasks that my customers are doing on a daily basis that is likely, to be tied to the business goal that I need to achieve, which is reducing customer service calls.

Right? So when you start to think about it, then you start to say, well, maybe a top task that a customer has is to check order status. Maybe they need to make a return. Top tasks are the clarify warranty policy or perhaps most people are calling customer service to update a failed payment method, for example.

So, again, you’re looking through the business goal, and then you’re focusing up on the top customer top tasks. Now let’s imagine that you wanted to check on the orders you wanted to focus your research efforts then, on checking an order status. So then what would you you would do is say, well, that seems like a really interesting path to go down. How do we help people check on their order status?

And then that’s when you start to, define a research question to further focus your line of inquiry.

And from there, once you have a clearly defined research question, you can choose an appropriate method to answer that question. And then finally, at the end of all of this, once you’ve gone through all of these steps, you will be gleaned with a number of actionable insights that, again, if you have stayed focused on following this process, you’ll now be able to draw a direct line from the business goal that you’re aiming to impact to the actions that you are inspiring people to take from the insights that you have uncovered.

Alright.

So now what we are going to do is look at how to ask good research questions. Right?

So there are four useful types of questions. There are a lot more different types of research questions, but I’m only gonna focus on these four for the purpose of this workshop and the exercises that we’re gonna cover in a little bit.

First one is exploratory questions. Second is descriptive. Third, explanatory. And the fourth one we’re going to look at today are relationship based research questions. And regardless of what type it is, good research questions are clear, specific, and goal oriented. So good research questions are clear, specific, and goal oriented.

And you can often decide what kind of question you need to ask depending on what it is that you need. So do you need to have ideas because your product is relatively low maturity and you need to discover, a little bit about how people are using your product first before you can dive deeper? Or do you have a lot of discovery work and now you need to find answers and ways to solve the problems that you’ve uncovered. Right? So depending on whether you need ideas or answers, that also impacts the types of questions that you ask. So if you need ideas, I suggest that you use exploratory or descriptive questions. These are a nice preliminary type of question to get you started off in the right direction for your research.

In terms of exploratory questions, they’re typically broad questions, and they help you learn about something where you have little existing knowledge. So exploratory questions are broad, and they help you learn about something that you don’t already know about. Right? So you’re really just dipping your toe in for the first time.

So an example of an exploratory question would be, how do people decide what refrigerator to buy? It’s very it’s very open. It’s very, you know, amorphous. It’s kind of fuzzy.

You don’t even know where to begin most of the time. So you’d start with how, do people decide which refrigerator to buy. And you can start to to understand, like, there are little clues in how you write your questions that give you, an idea of what type of question this is. So you’ll see that there are words like how and which, you know, in this exploratory type of a question.

Now in there’s descriptive questions as well. And a descriptive question is one that seeks to explain when, where, or how, or how many times something has happened. Right? So it a descriptive question explains or describes when, where, how, or how many times something has happened. So just it’s just describing. Right?

And then, again, if you take a look when, where, or how, you can see that these types of things that you would be looking for within how you’ve written your your question as well. So you take a look at an example. We have what marketing channel, right, led to the most, customer purchases?

And what and most are the clues within your question because it’s how many, again, descriptive, and what. You know, it’s just defining, describing something. Those are two just two words that kinda give you a clue about what kind of question it is.

So these questions tell you basically what happens and how it happens. Right? Again, just very, just very open. What happens and how? And they give you foundational knowledge upon which that you’re able to build other lines of questioning on top of. So now if you need answers, use these types of questions, an explanatory question or a relationship based question.

Now explanatory questions are focused questions. You know, they really start to narrow down in as opposed to being broad like an exploratory question. It says, okay. Now that we’ve explored a little bit, we really wanna focus in on this specific part here.

So it’s a focused question, and it’s aimed at explaining why something happens. So the previous two kind of tell you what’s going on and how it’s happening, but an explanatory question really helps you understand the why behind it. An example of this type of question is why, again, that’s your keyword, your clue right there, why do people drop off in the address field of a check of the checkout form? And again, looking at the words in the question give you a clue to what type they are.

And then finally, the last question type that we’re gonna look at today is a relationship based question, and that is a question that seeks to understand if one thing impacts another thing. So what is the relationship between one thing to another, and are they connected in any particular way? And if so, in what way are they connected?

And then the example of a relationship based question is, will more people make a purchase if we add a product comparison tool? Right?

So and then will more people, so that then then the keywords are purchase and comparison tool because you’re trying to see if there’s a relationship between purchases completed and the ability to make a comparison. So those are the two things you’re trying to find a relationship between. And if you can take a look at how the questions are formulated, they start out really broad and very fuzzy. And then you start to build on those broad and fuzzier questions so that they become more narrow and focused over time to lead you towards the answers. So the summary of this section before we go into an exercise, because I’m gonna have you get all hands on with question types next, is first, you have to start your research with a clear and focused research question.

And some research questions tell you what and how something happens, while others tell you why it happens. And then finally, you have to use the right question type, for what it is that you need to learn. So, some questions tell you give you numerical data as an as an answer, and other ones give you more, like, attitudinal type qualitative or quantitative data. So the type of question that you ask determines what type of information or quantitative or qualitative data that you get.

So now thanks for hanging with me all that time. We’re gonna do a little exercise. Are you all ready for this? So please go to the FigJam board.

People, can you, put this link? You can go to bit dot lee forward slash convex twenty four, and that will take you to a FigJam board that I have created for all of you. Just a quick note, you cannot you cannot join on mobile phone. So if you’re trying to, like, scan the QR code with a mobile phone, it should work on, like, a larger iPad screen, but it will not work on your mobile device. So you’re gonna have to open a new tab in your browser window and then copy and paste or write in this URL. And I see people are joining in. There are forty five of us so far.

Let’s see. Spotlight me. Oh, I’m seeing people names. Oh, another thing you’re gonna have to do is you’ll probably have to, if you wanna participate and collaborate, what you’re going to do is you’re gonna need to log in.

Okay? So take the time now. If you don’t have an account and you want to collaborate and make edits, then what you’re gonna have to do is log in. Yes?

Hey. Hi, Erin. We we noticed that there’s an audio issue on your side.

Okay. I’m gonna turn the camera off because they say it’s better without the camera. I’m gonna do that. Thank you.

Alright. So let’s go ahead. Yeah. We have what he said. So camera is off. That’s great.

Alright. So now I’m seeing everybody in the FigJam board.

For those of you who have never used Figma before oh, hello. Hello.

Hopefully, you have logged in by now. If you wanted to collaborate, there is a really fun cursor chat. Right? Possibility here. Let me just go over here.

This is Erin’s. You can see you can actually follow me if you want.

If you click on my face in the top right hand corner, you’ll be able to follow me around the board so you don’t get lost because there’s lots of little cursors to follow. So you can chat with your neighbors and your friends.

And if you want to, later on, we’re gonna have a thing where you can, like, vote or react on things. So in order to do that, you go to this stamp tool down here. You’ll see if you are logged in, you’ll have access to stamps as well as different types of emojis, so you can be using those a little bit later. And then finally, we also have sticky notes down here.

So you can see you should have a toolbar here, and if you click on the sticky note, you choose which color sticky note you’d like to have. You know? It’s really great. Lots of color choices.

I do like classic yellow sticky.

And then all you have to do is just drop a sticky down, and then you can start typing if you want. Cool? Alright. So I’m just gonna delete that for now, but that’s just a little bit about how you use it.

Another thing is I like to give a cheat sheet, for all the shortcuts. If you type forward slash, so it’s this little thing right here on your keyboard. It’s typically on the bottom right hand side of your keyboard. You can access your cursor chat.

Yeah. So that way, if you need to say anything or if I’m asking questions, I see Maria. Awesome. I see Ashley also. Excellent.

So perfect. So now you know how to do it and just kind oh, yeah. And then you can also high five people, which is really fun.

You can see I’m waving. Okay. Cool. And then you hit s if you wanna get a sticky note and e if you want to access the stamps and emojis.

Now that we’ve done that oh, wow. Everybody’s already moved on to this. I love it so much. So the first thing that we’re gonna do just to get you acquainted a little bit with the tool is some quick intros.

Grab a sticky note, and then put your name, where you’re from. And if you feel like it, you can add a little picture of yourself just so I can get to see, some of the people. Mariana Pessoa from Brazil. That’s very cool. I see Jenna from Denver, Colorado.

It makes it feel like there’s actual humans, which is wonderful because I feel like I’m losing my mind talking to myself by myself in a room. Odyssey Kennerly is in Philly. Very cool.

I’m from Harrisburg.

That’s a fun fact about me. It’s like the middle of nowhere in Pennsylvania.

Nice.

We got some people from Chicago, Indianapolis, again, Chicagoland, Burlington, Vermont, Indonesia. Very cool. So it looks like we’re really covering everybody. So now everybody I can see, you’re typing those. I’m gonna check these out afterwards as well too. So thank you so much for dropping stuff on there.

Oh, yeah. Here’s more pictures. How very fun.

Yay.

Cool. And if you feel like dropping a funny meme or an animal picture too, that’s also totally valid. Right?

Okay.

I’m gonna move on so y’all can finish doing that, or you can do it later afterwards too because the board is just gonna be oh, I see Joel has used a circle using artistic expression. Excellent. From Florida. Of course, Florida man would be creative.

Fun fun fun fun. Nice.

Alright. So our first exercise that we’re going to do is, like I was just talking about, we need to write clear research questions. So this exercise is to refine these user research questions. And remember, research questions and how you write them should be clear, specific, and goal oriented.

So the first step I’d like you all to do oopsies. I just got pulled away somehow. I’m not sure how that happened.

First thing I need for you all to do is it is it spotlighting me? There we go. First thing I need you all to do is read these business goals and the associated questions and these research questions. So the the first one is, the goal is to increase small business partner sign ups by five hundred thousand by the end of twenty twenty five.

Again, a good, clear, concise question that’s time bound and goal oriented. And a question that you might have, which is vague, is how can we get more partners to sign up? That’s not a good question, but it’s vague and we’re gonna refine it. Another goal is to improve customer satisfaction score, NPS, by eight percent by the end of twenty twenty five.

Again, it’s very specific, it’s measurable, and it’s time bound. So a question that you might have is, like, well, what impacts customer satisfaction? Right? So it’s just, like, what what can we look at?

What would impact that to begin with? The third one is to reduce customer service calls by ten percent by the end of twenty twenty five. So the first question that might come to your mind is, well, why are we getting so many customer service calls? And then finally, the last goal is to increase product subscriptions by five percent by the end of twenty twenty five.

So a first question out the gate might be, well, what affects product subscription rates?

Nice. So what we’re gonna do now that we’ve gone through those things is everybody pick a sticky note, right, and then copy and paste the question onto the sticky note, and then you can see what you’re going to do then is write a refined version of the question.

So how can we get more partners to sign up is the original question, And then you have to take, let’s say, five minutes to think about how you would refine that question.

And something to consider is that it’s okay to make stuff up. Like, imagine that you’re in a business, These are your business goals and that you’ve got some kind of insight so you could look at data. So how might you refine that research question? And you can pick any of the four that you want. I’m gonna put five minutes on the clock. Okay?

And, then you can do one or two questions or whatever, and then we’ll go through them. And then I’m gonna ask whoever wants to come on stage.

So what we’re gonna do is then, oh, hey Craig.

Then what we’re going to do is I’m gonna invite whoever wants to come up on stage. So if you want to explain how you have refined research question, please go ahead and request to get on the stage, but but I’ll just go through. So I’ll put a couple of minutes on the clock, and you can do as many as you want. So let’s see here. Let’s do let’s say plus one minute.

We’ll give you, like, about five minutes. Alright? So here we go.

Okay. Great. So let’s take a look at some of the questions and how you all have refined them.

And while I’m looking at some of them, would somebody like to join the stage and explain to me and the rest of the group how you refined your question. I’ll read a couple of them.

If so, if somebody was to join the stage, Vipul, I will let you let people on one by one. And in the meantime, what I will do is take a look and read one of the refined questions. So here’s the first one that Lei has.

How can we get more partners to sign up?

The refined question could be, how can we find out the reasons partners do not sign up? Can we run some sampling survey or feedback? What does the data say?

So that’s that’s good. They want to go collect more data.

Why are we getting so many customer service calls? Could be refined. What are the most common issues customers are calling about, and what are the root issues? Yep. Jenny, that makes a lot of sense because there you have, you’re you’re specifying to say, like, well, what are the reasons that people are calling and looking for the actual root of those things?

Another refined question is, so the first one is let’s see how how can we get more partners to sign up?

Where do we lose customers during the acquisition and sign up process? That’s a very good one. It’s very clear because then you’re able to go and take a look probably in your data analytics, to see where people are dropping off in order to plug that hole. So that’s a really good way to refine it.

So here the here we have increase product, subscriptions by five percent by the end of twenty twenty five. Then the broad question is, what affects product subscription rates?

And then the more refined question is, at what point in the sign up process do we lose the most customers?

Correct. Yes. That’s good to know. Like, where do people drop off? Is it the first day?

So the sign up process is, like, the first day after they’ve subscribed. Is it during a free trial? So it’s giving you specific points in time where the customer journey to dig deeper in. Would somebody like to share, get on stage?

Yes. Hey, Yash. Hey. Hi, Yash.

Hi.

So I did have a question which I refined, and it was that I assume that it is a clothing brand or clothing site. And if we are getting a, if we want to reduce the customer service cost, then the problem that I have assumed is that they are having some problem understanding the sizes based on brand. It could be that some T shirt brands do not have the same l size as some, sizes are, let’s say, medium on some brands, and they just don’t get the correct fit depending on the brand. So how can we ensure that, someone can get the right sizes without having to call the customer service or without having to return the product while the returning customers aren’t facing the issue because they are able to understand the platform?

So, yeah, that’s the the fine point that I have given.

Nice. Cool. Thank you so much, Josh.

Yeah.

Nice. Would anybody else get on stage and explain what their how they refine the question?

I see a familiar name. Craig, Sullivan has recently raised his hand to join.

Let me just Yay.

Hey, Craig.

Craig, I’ve accepted a request.

And Hey, Aaron.

Hi, Craig. Hey. Great workshop.

Nice to see you. I couldn’t resist.

My my refinement was around the NPS thing because NPS can hit it’s an average. Right? So it tells you what the average is of customer satisfaction, but you need the sublevel drivers. What are the service metrics or the elements of service that make the NPS score go up or down? Once you understand those, you can get a much, better idea of what you need to improve to make the average score go up by x percent. So you need to kinda take it all apart, basically.

Nice. That’s excellent point. Like, figuring out whether granular things within the score that you can examine more closely to figure out where you can make the most impact. That’s awesome. Yeah. Thanks, Craig. It’s so nice to see you.

Thanks, Craig.

It was nice to see Marcela’s name on there too.

Nice. Okay. Cool.

So now what I need to do is to finish off this exercise before we move on to the next section is and also thank you whoever said to lock or unlock the board. That was a great suggestion. I need to unlock the board really quick right now, though, because I have to, move something around really quick. Awesome. Cool. So what we’re going to do now is we are going to identify the question type based on, I guess, the quick overview that I gave you at the beginning of the of the session. So your question type determines if the data that you collect is qualitative, quantitative, or both because, questions can lead to different types of data output, sometimes qualitative, quantitative, or both.

What you need to do now is review the different question types. Okay? So, again, exploratory questions are broad to help you learn about something that you don’t know about. Descriptive questions are questions that seek to explain where, when, or how many times something happened.

Explanatory are the big, are the big questions that answer why and help refine a little bit more. And then you have the relationship based questions, which are the backbone to AB tests, hypotheses because you’re examining a relationship or how one thing impacts another thing. So what we’re gonna do now is for step two, I want you to put a stamp on the question, based on what type you think it is. So, how might so here we go.

And find, through the product. Okay. So a question, that has been refined is how might we increase small business partner sign ups by five hundred thousand by the end of twenty twenty five through sign up flow and product improvements?

So if you think this is a, exploratory question, a descriptive question, an explanatory question, or relationship based question based on the definitions, go ahead and take up the stamp.

Pick up a stamp, whatever you want, and then, like, post whatever you think it might be in whatever column here. Oh, I guess we need to lock the board again. That element wasn’t locked. Give me I’ll give you all three minutes to think through it. What kind of question is this?

And post a stamp, only one stamp per person in the column that you think that the question type is. Okay?

Alright. Looks like there’s still just, you know, one or two getting moved around, being removed.

Alright.

It looks like most people put, put their votes in the exploratory category, which is indeed the most correct answer.

This is a very broad and open question. It’s trying to understand, you know, like, what are all of the possibilities, that we could look into to increase the partner sign ups.

But it also is a little bit more refined because it tells you a little bit about, like, the sign up flow and product improvements. Right? So I think there is a relationship based assumption within this question that is about the sign up flow and product improvements. It’s assuming that you can make, an impact on partner sign ups by looking at the sign up flow and making different, improvements to maybe, like, the the product itself or the value proposition, all that kind of stuff.

So awesome. Would somebody like to, get up on stage and give any example of, like, say anything about what they thought it was and why? Cool. So let’s do one more of these, but we’ll only take, like, a minute or two so we can get to the next section and finish the next exercise.

So on question number two, we have refined it to say, what is the top predictor of a customer rating their experience a seven or less in the NPS survey? So this kinda gets to what, Craig was saying about there is more granularity, in the NPS average score. So now it’s saying we’re only gonna look at scores that are seven or less. And then what is a predictor?

Right? What is a what is a thing that happens before that that makes them give a seven or less? Right? So I’m gonna put I don’t know how to take the time off of this.

We’ll give you two minutes to think about this and then to add your vote. K?

Alright. So it looked like most of you had voted, if I can remember correctly, that it was a descriptive question with a lot of votes also in the relationship based category as well too. And I think those are both really great, great votes because it is descriptive because it’s looking at, like, you you’re describing what predictors, but it’s also kind of relationship based too because a predictor is something that impacts the score that somebody gives.

So it’s a it’s relationship based, but it could also be just it could also be kind of descriptive in nature as well too. Alright. Now what I’m gonna do is, yeah, control z for anybody.

For MacBooks, command z. Yep. Just in case.

Yeah, everyone just try to undo.

Alright. Well, you know, sometimes things go as you plan and sometimes they don’t. We’re just gonna go with it. So the next step, what we’re going to do is after you write your research question, the thing that you want to learn, then you choose your research method, what you’re going to do to, to learn, the answer, which you will need to learn.

Okay. So what I’m gonna do now is I’m gonna share my, my presentation again. So I’m gonna share a different tab. So just call it.

So, now what we’re gonna do is we’re gonna dig into how to pick the right research method. And like I said before, the question your question influences the research method that you choose. So some useful research methods that, are not the only ones available, but ones that I rely on a lot are these. First one is usability testing.

It’s very popular. It’s basically watching people complete tasks.

The next one is data analytics, so things like Google Analytics or Hotjar.

Those things like click maps and heat maps are actually a visualized form of data analytics. Then you have AB testing, which is a form of randomized controlled experimentation to understand how one thing impacts another. You got shadowing, which is a personal favorite of mine. It’s just creepily watching people to see how they behave in their natural environment.

And then you have surveys, which is just getting people to answer questions, but in written form. And then you have user interviews, which is asking a line of questioning in person to gather as much information as you possibly can. So it’s a little bit more, fluid because you’re actually able to guide discussion as you go along. A really useful tool to pick the appropriate research method is something called the landscape of user research methods by a dude named Christian Rohrer. At the end, I have a list of resources for you, so you can actually see all of the different user research types and then where they fall within this landscape.

As you can see, it’s a matrix. And on one side of the matrix, it defines what type of data you get from the research method. Does it provide you with qualitative data, which helps you understand why something happens and how to fix the thing? Or does it provide you with quantitative data, and it answers questions like how much and how many? And as you can see, depending on what type of question you ask, in surveys, you can get a mixture of both qualitative and quantitative data. So some methods give you a mixture of both types of data.

Then, on the other part of the matrix, the axis, you have, attitudinal. So it tells you what kinds of things will you learn. Will you learn what people think, how they feel, what they say? So that’s attitudinal type of data.

Or will you get behavioral information? Will you be able to see what people do, and and and the actions that they perform? So when you take a look at when you’re able to look at your question to say, okay. Here, I need to understand what people do or what people say, and then, and I need to learn why, then you can kind of traverse the matrix or take a look in the matrix to see, okay.

If I need to, let’s say, understand customers’ behaviors, but in a very clear and unbiased way, then you can take a look at a b test or do data analytics because that gives you, it gives you very clear behavioral data in a quantitative way. Alright. So now we’re gonna go back into the board and do one more exercise before we finish everything up. So if you can all go back into the, board.

I see a lot of you are still in there, and you can see that I have let me see. Let me just change my screen again.

Stop sharing.

I’m going to share my screen again.

Go back.

Here we go.

So you can see in our board right now no. Sorry. Not at all. I’m super glad for people interacting and giving suggestions.

We’re on the same page, Greg. Sometimes I feel like we’re we might be the same person. Okay. So here you can see this is your landscape to help you decide which, refined research question to use. So let me go ahead and grab the question. So let’s see here.

Alright. So what I’m going to do now is I am going to put some text here.

So what we’re gonna do now is we’re going to look at these refined research questions. This question is, why do people call customer service post purchase instead of using the self help features? Can somebody so this is the question. Why do people call customer service poach per purchase instead of using the help self help features?

So if you take a look at the question, right, this is the first question here.

If you follow me here, why do people call the the questions? Put a stamp on what you think what kind of research methods you could use to answer this question. Okay?

I’m gonna give you a couple somebody has a wrecking ball.

That’s great. Nice.

Yep. So I’m gonna put, let’s say, two minutes on the clock for you to read through the question and put your vote on what you think research method you would do based on the question.

Okay. Cool. Awesome. Perfect. Everybody can hear me. Thank you so much for answering. Alright. Cool.

So if we take a look at what everybody voted for, what method they would choose to answer this question, it’s looks like some people answered usability test.

I would probably not do that one, but it’s a good thought. Shadowing could be interesting to understand.

You have to have very specific targeting of the shadowing person, so it might be kind of hard to answer that question with shadowing.

Data analytics. I see oh my god. Almost nobody voted for data analytics is a is a good way to answer that. But I would like to think that data analytics is actually a really good way because depending on what you capture, if you capture, let’s say, customer service calls and the customer service agents tag what the call is about, then you’re actually able to identify that in the analytics for your call center data.

So I think data analytics is a good answer even though many people did not ask, put that in there, but data analytics is one. And then we also have surveys and user interviews. Yes. You could do a nice little micro survey, after somebody has maybe called the customer service line to say, hey.

How was your recent customer service? Can you help us understand why you called today? You know, just to give some more information.

And you can also maybe target specific user reviews to ask people to have a conversation who recently called the customer service line. So, yes, I would say data analytics, a well targeted short survey or potentially targeting specific customers who’ve called for some user interviews. Nice. Since we’re running out of time, yeah, web call tracking.

Exactly. Yes. People should. If you don’t already track why people are calling your customer service line, you should really do that.

It’s great data to collect.

Now, who do we want do we wanna do one more? I’m happy to stick around. Or did you wanna get into the final thing?

Did you all wanna do one more, or did you want me to just keep on? You wanna do one more? More. Okay.

Cool. Yay. Okay. We’re gonna do another one. I have a whole bunch of them. So let’s see here.

Descriptive.

Okay. So now this is your second question.

Now if this is your refined research question, what are the top three reasons people don’t subscribe after a seven day free trial is over? What would you possibly what kind of research methods would you use to answer that question? Okay? So what are the top three reasons that people don’t subscribe after the free seven day trial is over? So this is actually a descriptive type of question. You can tell because of the word what.

And you’re trying to identify reasons. So what reasons, so it’s descriptive.

I’m gonna give you two minutes.

There we go. And, you know, I’ve locked this board so many times, but for some reason, it’s just not locking. I’m really sorry, but I don’t know why it’s not locking.

So go ahead and put in your votes. Two minutes on the clock.

You should be able to join. Hey. Hi, Shilpang. How are you?

Hi, Shilpang.

Can you hear me alright?

Yes. We can hear you.

Thank you so much for the very fun workshop.

I wanted to say behind the what, there’s a why question. So it’s individual persons would answer why am I not subscribing.

So I’m guessing it’s surveys and user interviews would be interesting. And then we kind of put in more of the top three based on what users say. So I think it’s a quantum call together.

Very cool. So you’ve already taken the question and you’ve dug down one level further, and made it into a why question, which is great. I really love how you’ve done that. Very cool.

Nice. Would somebody else thank you so much, Shilpa, for, answering that. Would somebody else like to explain which one they would like to do, which research method they would like to use to answer this question and why?

Hey, Peter.

Hello. Can you hear me?

Yes. We can hear you.

I think it will be something of, a mixture again between surveys and maybe interviews. Although interviews might be a bit hard with lots of, guests. I think more a survey and, again, if they would be willing to answer that and then, obviously, analyzing that data.

Yeah. And when might you ask that survey? What kind of question would you ask?

Sorry. You’re breaking up. What?

Sorry. When might you ask that question?

Like, if you think of, like, a targeted moment that you could send the micro survey, for example, when might you probably try to do it before the actual, trial ends so we don’t lose them completely.

Very interesting. Cool.

So you would try to preempt them Preempt them before before it’s happening because, then it’ll be more of a recovery, which makes it a little bit more harder.

Yeah. Very cool. Thank you.

I love that. Case. So working in hotels, obviously, you’ve got your customers, your clients there. So sometimes you want to kinda capture what’s going on before they leave. Otherwise, once they’ve left, you lose that kind of access and we all know it’s hard to get surveys out of people once they’re not there and engaged.

Nice. Yeah. So it’s like anticipating their need and then and addressing it before you lose them. That’s great.

I love it.

I used to yeah. That’s true. I used to work customer service, and that was always a very, you know, that you anticipate their needs before they even know what they need.

Doesn’t always work, but it’s a good, I think a good way to tackle the problem before it’s happened if you can. And then if not, then use that data.

Once it’s happened to make sure it doesn’t happen again. Automatically improve Excellent.

Excellent. Thank you. Cool. Thank you so much, Peter.

And it looks like, Denise has a question in the chat. What’s a good example to when we cannot use data analytics at all? That’s a good question.

I think I need to think more about that if I’m completely honest. When can you not use data analytics? Data data analytics doesn’t give you why. It doesn’t tell you the why. It only tells you what happens. So I think anytime you have a why question that you need to answer, you data analytics numbers will not tell you why, in my opinion.

For diary studies, analytics will tell you how big and where, not why. Yep. So Craig, who we all love, also, backs up the answer of numbers don’t tell you why. Yes? Awesome.

Cool. Does anybody else have any final questions on this section before we move to another, the last section?

Yeah. Why not trigger the survey? That’s also, that’s also a really great point. That’s another good moment where you could trigger the survey.

That was my initial thought. So why not the survey, on the unsubscribe action so they can share their thoughts while active? That was my first thought as well too. But I really like the idea of trying to preempt saying, hey.

Do you intend to subscribe? If not, why or to to con to confirm your subscription or to keep on? And if not, why? And then that way, you could potentially save them based on their intentions.

But I think either before or after is a great or maybe even both if you wanna do a double whammy, to get all the information. Nice.

Right. Yeah. I think I was on mute for a few seconds there.

I think we we, have time for just one more person, or do you want to move forward, Erin?

Yeah. No. We can do another person. I would love to hear. Yep. I mean, if people don’t mind staying on, I love doing this.

So I’m more than happy to stay on and do it. So who else would like to come on and talk to us about what they’ve chosen and why?

Alright. This is fun. I love it. Okay. So let me get my question for my little questiony doc.

So here’s the refined one of the refined questions that we had, which is okay.

Huge. Let’s make it a little bit smaller. Let’s do large.

How might we increase small business partner sign ups by five hundred thousand by the end of twenty twenty five through sign up flow and product improvements?

So what kind of research method would you use to answer that question? Remember, this is a big fuzzy question. It’s an exploratory question.

So what type of research method might you use to understand, how you can increase partner sign ups? And, also, don’t forget, you can go up to the landscape of user research methods and see what are you what are you trying to understand. You wanna understand, you know, what people think, how they feel.

Do you wanna learn behavioral, what they do or they do not do? And do you wanna understand, how to fix something? Do you wanna understand why, or do you wanna understand how much or how many? So kind of think about it through the lens of the landscape of user research methods, and then say which, method you would do.

Nice. Cool. Looks like a lot of people have put their votes in so far, and it looks like we have a lot of votes for AB testing, data analytics, and fewer for usability tests, almost none foreshadowing, and a few for user interviews. Who would like to join on stage and say which user research method they would use to answer the question, how might we increase small business partner sign ups by five hundred thousand by the end of twenty twenty five through sign up flow and product improvements? Who would like to get on the stage?

Yeah. It’s so nice to be here again. I just wanted to say that the way I would choose would be AP testing. And I would say that it is a very open ended thing, and there could be lots of variants that we can make, And we can have an unbiased testing that we can, let’s say, fix a time.

And unless and until that deadline has has indeed, we do not just stop the test or do something that would create some type of bias. Just practice a cognitive distancing for it and allow us to, let’s say, change the user flows, maybe, add some sort of small tokens of appreciation, like, they’re almost there or this, flow is almost finished. Something like, to add something to these steps, how many steps are remaining or some type of hints to the user, how they can fill it. Something that would actually make it more interactive, make it easier for the user, and let’s try out, in various ways that we can.

So, yeah, that’s fine.

Nice. So it sounds kind of like you would do a bit of a UX audit on the sign up flow, and then do some research to understand what are maybe some interesting UI patterns or interaction patterns that you can use to make it easier, for people to actually get through the flow.

Nice.

Very cool. Thanks, Yash.

Super interested to hear what you would do and why. Would anybody else I see Neil has answered that he would choose data analytics as he would like to see the numbers to understand how many sign ups, we have at the moment, and are they new or return returning users? Also, usability test to run through the task. I love this because he goes in he’s using, like, mixed methods. Right? He’s taking a look first at the data analytics to understand if there’s any difference between the user type, and whether or not they’re signing up.

And, also, looking through usability tests, do a usability test to identify some of maybe the blockers that people run up against in the sign up flow.

And, yeah, the survey prize. And what did you say? The survey prize.

I think you said five or He’s asking about the quiz.

They’re asking about the quiz.

So, I did see a new member actually, request, to be on stage. It’s Nate Branson.

Nate, I’ve Nice. Given accepted a request if you can try coming up on stage.

Yep. Nate’s here. Hey. Hi, Nate. How are you?

Hi, Nate. I think it’s an audio problem for you as well. Can you try speaking something, Nate?

Testing one two.

Can you hear me now?

Yes. Sorry. It’s all good. Yeah. Ah.

Please wait.

I had to mute and unmute just for future reference. Did the same thing for my camera to get it to turn on.

But I chose everything except for shadowing.

I would this seems like a bigger question than some of the previous ones in the sense that we would need similar to what Neil was saying, that we need some exploratory studies first. So, like, data data analytics was the first one that really screamed out to me as far as, like, where users actually dropping off or where are they actually having issues in their sign up flow if they’ve already started doing Gage. But then following up with, like, usability testing and user interviews, if you’re really trying to find the the why in there necessarily. And then once you start to get, like, how is it actually going to work, then really hammer in on, like, quantitative usability test and some AB testing afterwards to get it into the prod environment before pushing everything live.

Absolutely. I love it. Because, again, you started really, really broad to say, hey. Where do we need to look closer?

Like, is there any interesting happening in a specific area of the product for where this is happening that something interesting is happening? And a great reason to start with data analytics is that most companies usually have these. Right? And they’re pretty cheap.

Right? It’s not like you have to go out and recruit users. You can just start poking through the numbers from behind your desk to see if something pops up to say that’s interesting. I wanna learn more about that specifically.

And then like you said, once you see where somebody dropped off, for example, or the highest drop off rate, like you said, do some usability studies, to understand, you know, what is actually happening there. Or if you have screen session recordings, like, if people are dropping off at a specific form field, what is it and why?

Why are they dropping off?

But, yeah, I think it’s great. That’s awesome. Awesome, Nate. Thank you so much for sharing. I would do all those things too.

Cool.

Thank thank you so much, Nate.

Fabulous.

Alright. So we’re gonna y’all wanna get to the quiz. We’re already thirty minutes over. But you know what? Time is a human construct. It’s just an idea that we have in our heads that we agree to.

Absolutely.

So let’s go ahead and I’m gonna finish up the time with just a quick thing about how we can present our findings, the insights that we, glean in a way that people will be interested in listening.

Let me turn my camera off.

Alright. So, now what we’re gonna do, cover in the last couple of minutes, is how to communicate your findings to inspire action.

So, what I’ve noticed is that researchers who are trained academically tend to like to, write really long research reports because academic writing, that’s typically what you do. You write long sentences. You’re very detailed about every specific finding that you’ve come across, what they could all mean. But what I’m suggesting and proposing for you all is to shift how you write your research reports in a business environment. And you start with writing a research report name that is concise but describes the research findings or the research area.

Then what you do is you write a brief executive summary or a T L E R at the top, no more than a paragraph or two, that really distills the main messages that you want people to take away from the research findings that you got. So they that way helps people quickly understand, is this research important for me to read or not? And if so, what what are the main takeaways that I’m going to get? So give them the goods upfront to pull them in to read more. And then what you can do is create bulleted lists.

Bulleted lists are a great way to get people to scan through things, again, to figure out what is most interesting to them. And then what you do is you have a list of the insights that you’ve gleaned, like, the top insights from your your research study that you’ve done, and then you always put the research insight next to the goal that it aligns to with the business. So that way people understand why it is immediately relevant to them. So the customer insight or the research insight should always accompany a goal so people know what to do.

And then finally, to make the insight actionable, really spell it out for people. Give them a place to start. Help them understand that you could try doing this. You could do that.

You could do this. This is a solution. To get them thinking about all the different ways that they could act upon insight, in a way that would move the associated business goal. And then finally, everybody loves a listicle.

So another great way to present your research findings is a top customer problems list. And this is great. You can do, like, a top ten list or a top three list depending on, you know, how focused you wanna go. But, like, print these out, hang them around your office, make them be your desktop, your your background to your desktop, and all your calls or whatever.

But, like, people love a listicle. Give them something short and sweet that they can read and refer to. Make a list. So for example, on this list, you’ve got preferred payment method is missing.

That’s a problem that people have, and that would directly impact sales if you solve that problem. Maybe the warranty terms are something that everybody constantly misunderstands.

Right? And then make sure that you put in that top problems list that it’ll if you solve it, it’ll impact customer service calls, which ties directly back into the business.

Perhaps there’s unsuccessful delivery. If a lot of people are not getting the packages that they’ve ordered, again, put in how it impacts refunds or customer service calls, and also the wrong size ordered, and let them know that if you solve that problem, it’ll likely impact returns. So always put the insight alongside the business metric. So, hopefully, what you all got from today, from the workshop is what you need to do is align the research that you’re doing to customer problems, but looking through the lens of business goals.

Make sure that you’re asking good research questions that are clear, and specific in some cases, and then make sure that you’re using the right research method to answer the type of question that you’re asking. And then finally, summarize your insights in a clear and concise way in a way that people enjoy digesting information such as lists or brief reports that help align them with the business goals to inspire people into action. Does anybody have any final questions to close out the workshop today?

Any questions?

Let me see. So Craig has been waiting, to get on stage. So I’ll just accept his request again.

Yeah. Awesome.

Hey, Craig. Yeah. Oh, he’s there.

I thought you’d moved on. It was fine.

Yeah. That last question was really great, because it exposes an underlying truth here, which is you need to use often multiple research methods for ways of getting perspective on a problem. And the more perspectives you have, the more of a three sixty degree, like, fleshed out view you have the problem. So sizing it with quant stuff is great at first because that tells you what is the whole machine and how does it work, how many people come in front of the machine, and how many go out the end, and where do we lose them in between. So that gives you focal areas, then you can use all your qual and quant techniques on those focal areas to discover what the problems are at both qual and quant level. So session replay as well, but people mentioned user testing, surveys.

There’s device testing too to see that people aren’t getting short changed on the popular devices that they use. And then that will give you tons of ideas for changes to make and a b test that you can, run and measure how the stuff is then getting you towards that goal. So the whole thing is multiple steps and multiple techniques that need to be applied in slightly different ways. But that was why it’s such a brilliant question because the answer isn’t, oh, you just run a user test and you’ll know the answer. The answer is often more complicated to do several things.

K?

Yeah. Each you use the different question types to get closer to go from broad and fuzzy to get more focused and narrow down until you finally have the answer that you’re looking for. And then that’s why I really love AB testing like you mentioned because it usually tells you, did you quantitatively solve the problem or not? Like, did you move the metric that you were aiming to move or not?

So, yep, all the time.

Our sculptor doing doing this and the way they shape the head and refine the head is exactly like this. They start with just a blob of clay and then it’s featureless and then the features start to appear, and then it gets more and more detailed with every step. And this is the more in qual and quant that you do, it it improves the fidelity of what you’re seeing. And and when I saw them doing this with the sculptures, I thought, oh, this is a bit like my work.

You make something a pattern appear out of a lump of clay.

Extracting it. I love that metaphor. I mean, I went to art school, so I completely totally lean into any art metaphor. For there are. Love it.

Nice.

Thanks, Vicky.

So looks like Josh?

So Hi, Greg. Miss, before I have Yash on stage, there’s an interesting question actually from Lisa that I want to take up. I’ll just show it on the screen as well.

Okay. So realistically, doesn’t the company budget and product delivery limit the research methods?

Absolutely.

That is one hundred percent a constraint that we all need to work within. And sometimes, we’re limited not only by product delivery limit and the budget, but also what information we have available. There have been instances where I would love to answer a question with data analytics, for example, but we hadn’t been tracking it. So you might not have access to the data, and then you can start, tracking specific things to answer questions that you need to answer. So yeah. So it’s like, is the data available?

Is the method financially, available to you? Are you able to recruit users that would be willing to have an interview with you? Are you able to shadow somebody? Will they, like, allow you into their homes or into their places of work?

So there’s all kinds of constraints, which is often why it’s good to have, a couple of different different types of methods to use, to answer a question. So that was a great a great point. Yes. Well, a hundred percent, you have to work within the constraints of reality.

A hundred percent.

Great. So I’ll now have Yash on stage.

Yeah. So, first, I want to thank for this wonderful session and also for the wonderful book that you wrote. I really thoroughly enjoyed reading it. And there is something in the book that I really agree with, which is the deceptive patterns. There are things like confirmed shaming or there are things like undesired defaults.

And, it does happen in our companies. Let’s say, if I take example of my company, let let’s say some product manager would bond some undesirable defaults, let’s say, AI to take or train with your data or some type of confirmed shimming, let’s say, do not, purchase the default way. Try the pro pro plan. So it will just write, I will try the hard way or something like that. So how would you, tackle such situations, such real life situations at your company?

So thank you so much. That’s a great question because ethics comes at any have the privilege of making some for people. You have to consider how is what I’m doing impacting the end user at any moment in time. And when I worked in customer service, I could immediately see how my action either made somebody have a great shopping experience or have a lovely dinner or how I ruined it, you know, by the service that I was giving them.

And when we do things online, there’s a lot of space in between the actions that we have and the impact it actually has on people’s lives. So even just being aware of that fact that what we do on a daily basis can ruin somebody’s day or mess up their whole workflow for working on an admin tool or take their hard earned money, you know, that’s a big responsibility. So I think just being aware of that and having a good moral compass is the first step. Then the second step, I think, is not as as common.

It’s being in a place of privilege. Right? I really feel like business leaders, people that have the power within organizations really need to be the ones to step up and fly this flag of making sure that you’re making ethical decisions because people will follow, the examples that you set within your organization.

And I remember when I was working at Booking Prom, I had the privilege and the responsibility of being a principal designer there. So the actions that I would take, the experiments that I would run would be under pretty big scrutiny. I remember one time, somebody at the organization asked me to run an experiment for them. They were, like, a product marketing manager or something. And they said, oh, if we just I think it was like a an undesirable default. Oh, if we just do this, then we’ll get more blah blah blah. And I was like, I’m not doing that.

Because I was in a position of power and privilege. Again, not everybody can do this. But if you are, I implore you to say no. So you just say no and explain why you’re not going to do that. And because people will like to take shortcuts, they’ll change their behaviors, in ways that incentivize them to hit metrics, but it’s really up to us to hold ourselves and each other accountable to make ethical decisions.

And then another thing is just flag it to people. So flag it to your leaders, and let them know what you think is or is not okay. I’m gonna it says, I’m gonna turn my video off.

But Okay.

Yeah, that’s that’s what I would say. And then also just if you I feel like healthy environments have a place where you can have a discussion about product quality and ethics.

There are multiple places where I have worked where we have, like, a whole Slack group or a Facebook workplace group where we would have debates about, you know, is this default, ethical or no? Why not? Or and then people could chime in with what they thought and and why not, and it was an ongoing discussion that we would have with one another. So yeah. Does that answer your question, Josh?

Yeah. It it does.

Nice. Cool.

Thank you so much.

Thank you so much, Ash.

Thank you so much.

So since there are no other, request, and I think we’ve answered all the I saw one question.

Somebody had I don’t see it says this event had ended in my what I’m seeing. But I remember the gist of the question was what do you do if an AB test says one thing, but then you have user research or qualitative form of data that either contradicts that or says something opposite. Right? And I think that’s a great question.

It’s something that we, when we traverse both the worlds of quantitative and qualitative data, seeing where things contradict.

And I think that’s really important where we cannot just blindly follow numbers or swap at a specific number. So just because an a b test tells you that something happened, doesn’t necessarily tell you that it is what you should do, number one, and that it is if it’s the right thing, morally or ethically. And that’s where we have to take into account the top part of the hierarchy of evidence, which is the systematic review, where you come together as humans, and you have to make an ethical judgment call and use and weigh and consider all of the stakeholders that you have to represent and all the people that you’re serving to make the boat best and most well considered decision to move forward.

So that’s not always following an AB test. It’s about looking at all of the evidence that you’ve collected, seeing how it stacks up against one another, understanding what your business goals and strategies are, and who your stakeholders are, and then making the best decision possible with the evidence that you have. Hope that answers your question.

Yes. Great engagement engagement once again, in this workshop as well.

Erin, do you have any any parts of the presentation, left?

All I was gonna say is thank you so much for everybody for coming in your I have some resources here, but you can also see Andy Board, that I have. There’s, like, a little bookshelf.

One of those the books that Craig has suggested, surveys that work by Carolyn Jarrett, and then interviewing users by Steve Portgle. So there are some more resources there, dig in and learn some more. But thank you so much. It’s been so much fun, everybody. Y’all were wonderful.

Thanks for giving me your time.

A lot of peep there were a lot of people who actually asked for access to this, presentation deck as well. So if you have a public link, you can just drop it in the chat right now, and you can share it with me as well. And after this workshop, I will share it with, the attendees, everyone, in an email. Right? So you can share it with me.

And, yeah, great, great workshop once again. Of course, goes without saying, you, it’s all it’s, it’s evident from the kind of response, you received on your FigJam board and, you know, all the exercises that people were practicing and all the questions that poured in and all the lovely people who, gather the courage of coming up on the stage, this virtual stage, and, sharing their thoughts and their inputs. Thank you so much, Arin, for building this workshop.

I know, there’s there has been a considerable amount of time that has been that has gone into, you know, building this entire, FigJam board and this entire flow. So thank you so much for that.

Let me just pin this.