Teresa Torres - Continuous Discovery for Successful Product Teams at Product Faculty

So there was a whole bunch of questions around learning more about automating your recruitment process. When you talked about the fact that you're recruiting someone every day, that's incredible. Do you have any further tips on that? There's a lot of ways to implement this. The key to making it work is to match the right incentive with what you're asking for. And the philosophy here is to ask for something small. in exchange for a big reward. So the example that I showed, it was actually a company called Snag here in the US. They're a job board. It's always weird as an American to say this to people from other countries because I have to explain that our minimum wage is embarrassingly low. In much of the United States, our hourly workers make as low as $7.29. That's actually our federal minimum wage, which I realize is atrocious and embarrassing. And so Snag is helping people find retail and restaurant jobs, many of which are minimum wage jobs. So these are people that are making $7.29 an hour. I don't know, you may not have noticed, but on that screen, it said, we would love to talk to you for 20 minutes in exchange for $20. So that's a small ask. They're not asking for an hour. They're asking for 20 minutes. and they're offering them $20. So for somebody that makes $7.29 an hour, $20 is a big reward. All right. Good evening, everyone. Thank you for joining us on this wonderful Wednesday evening. We're excited to kick off our second event for Toronto Product Mastery Series with Teresa Torres. Teresa's blog, Product Talk, is widely read by product managers and product leaders. Teresa is one of the most respected discovery coaches in the world and we're really excited to learn from her today. I've actually attended Teresa's workshop in New York last year and loved it. I was able to apply the learnings in my work right away. Today, Teresa will be presenting on the what and why of continuous discovery and we'll learn why continuous discovery is a cornerstone habit for the most successful product teams. So with that, let's kick it off with Teresa. Teresa, I'll let you take it away and I'm going to close my camera and have you take it away. All right. Thanks, Mo. Well, welcome, everybody. It's always a little weird to do this virtually because I feel like I'm talking to myself, but I'm going to do the best I can. And hopefully we'll get to interact a little bit in the Q&A session. So today I'm going to talk about the what and why of continuous discovery. We're going to get into all the nitty gritty details. For those of you that aren't familiar with me, like Mo said, I do work as a product discovery coach. I have worked with teams all over the world. You can see on the map sort of pins everywhere and some of the companies that I've worked with on the bottom. I only share this to let you know that everything I'm going to share today really comes from seeing how a lot of different product teams work, seeing how discovery can be done lots of different ways and lots of different contexts, and then really starting to understand what are the common principles that underlie good discovery. Now, for those of you that... are not super familiar with this discovery delivery distinction we're going to start at the very beginning so don't worry um so i like to distinguish between product discovery and product delivery this is becoming much more common vernacular in the tech industry um it's not very complicated it's really as simple as this discovery encompasses all the activities that we do as a product team when deciding what to build So how do we decide what goes in our backlog, what goes on our roadmaps? How are we making those decisions? Whereas product delivery are things like writing code, creating production quality designs, the things you might be doing in JIRA to really be able to ship production-ready code and products. Now, why is this distinction important? We see a lot of companies, maybe even most companies, overemphasized delivery and underemphasized discovery. It's really easy to fall into what Melissa Perry calls the build trap. I've also heard the term feature factory, where we measure the value of our work based on the code that we ship. And we forget to look at what impact did it have. And so what we've seen over the last 20 plus years is we've seen the rise in importance of discovery to be on an equal level with delivery. It's not just about how fast we can build or how frequently we can deliver. It's also about are we making the right decisions about what to build? Now, I'm sure for most of you that are that are logged on, this is not a new concept. Discovery has become pretty trendy lately. Most of you are probably doing some discovery activities. So you're interviewing customers or you're doing A-B testing or you're running usability tests. I think usability tests are one of the most common things that I'm seeing teams do. But if you're like most teams, you're experimenting with those methods when you absolutely have to or when you have time to. You're not doing it in a structured or sustainable way. So there's a couple ways this shows up. One is teams might do it at the beginning of a project. and then they do a bunch of research, they make some decisions, and then they move into delivery. And it's this very phased approach. Another common sort of pattern that we see from product teams is you only do these discovery activities when there's disagreement on your team. So as long as everybody agrees, you're going to go ahead and build what you think you should build. But then when there's disagreement, you might experiment to figure out what to do. The challenge with both of those patterns is that As product teams, we're actually making a lot of decisions that like just based on what we know and oftentimes what we know is different from what our customers think or want to do. And so what we're seeing develop is this concept of continuous discovery where instead of doing discovery in this phase approach at the beginning of a project or instead of doing discovery just when your team disagrees. We want to always be doing discovery. We want to always be engaging with our customers, learning from them about what their lives are like, learning what their pain points are, learning what their needs are, and then constantly evaluating our solutions. Are we building the right thing? Are we building the right thing next? Are we building the right thing in the best way? So some of these concepts are new enough that I know from experience working with teams that it's really easy to think that you're already doing continuous discovery. So I want to start by defining continuous discovery really crisply so we can dive into what does it mean and you can use it as a benchmark for are we doing this today so I define continuous discovery as at a minimum weekly touch points with your customers so we're actually meeting with your customers by the team building the product so your product manager your designer your software engineers they're actually building the product or meeting with customers where they that team conduct small research activities in pursuit of a desired product outcome. Now, we're going to dive into this definition because there's a lot here. And we're really just going to go line by line and break this down. Why does each of these lines matter? And we'll use this as a benchmark for what does good continuous discovery look like. So first, weekly touch points with customers. Why does this matter? So most of us on product teams we're making decisions every single day some of our decisions are really big strategic decisions like what goes on our roadmap or what strategic bets to make some or even what outcome to go after some of them are smaller decisions like where do we expose this feature in the user interface or what do we label this button or do we prioritize this user story over another user story most of us know that for those bigger decisions, the big strategic bets, the big roadmap that we want to be doing discovery, right? We don't always remember to do discovery on the smaller decisions. And here's what happens when we talk to customers once a month or once every other week, we're making 10 days or 20 days worth of decisions where we didn't get any customer input. So the challenge with that is that I'm sure every one of you, if you pick up your phone, which I'm sure is near you, and you look at it you probably could find at least one if not half a dozen or a dozen apps that you downloaded that when you downloaded it you were really excited about that app they got the big idea right but then you started to use it and some of those daily little decisions that they made weren't quite right and so while you were excited about the big product concept it didn't work for you in practice And we see this often, right? Teams do just enough research to get a big idea and they put their heads down, they go write a bunch of code, they release it out in the world and they hope it works out. And we see a lot of products get some early traction and then flame out. So one of the best ways to guard against this is to make sure that we're infusing as many of our product decisions with customer input as possible. And so if we're making product decisions every day, We want to be engaging with our customers every week. Now, when I first read the definition, I said at a minimum every week. I have teams that talk with customers every day. Now, what this looks like is going to vary significantly based on who your customer base is. If you work at a large common retailer, you're all based in Toronto. So if you think about your... Oh, man. I know there's a Canadian retailer that has something higher in the name that's like the equivalent of our Target or Walmart. Hopefully you know what I'm referring to. And, you know, it's a big, large store. Almost everybody's been to one. Finding a customer to talk to every day. Super easy. You could just go into a store and talk to customers as they came in. You could call almost everyone you know and they're a customer. If that's the type of customer you have, if you work at Facebook, if you work at Netflix, Customers where it's really easy to find your customers, you probably could talk to your customers every day. You might even talk to your customers multiple times a day. Again, we're trying to infuse as many of our decisions with customer feedback as possible. Now, the complete other end of the spectrum is, let's say you're working with doctors and nurses. And right now, a lot of our doctors and nurses are busy with more important things to take care of than giving us feedback on our products. So we're clearly not going to talk to them every day. What we want to look for is how do we find a regular cadence where we're using as many of our decisions as possible? And it's going to look different for every team. As a benchmark, I want to encourage you to try to get to at least once a week. For many of you, that's going to be very hard. Think about it from a continuous improvement mindset. Can you make next week look better than last week? If you're interviewing quarterly, can you get to monthly? If you're doing it monthly, can you get to every other week? If you're doing every other week, can you get to weekly? For a lot of you, doing it more frequently than weekly is going to make a lot of sense. I will share that I used to be a startup CEO of a company that had two sides to the marketplace. We worked with universities and university alumni, and we also worked with employers. We helped companies hire alumni from different schools. And as a CEO where I had many responsibilities, I still talked to an alum. and an employer every single day because I really think it's this important that we stay this close to our customers. Now, I'm not going to tell you to start with every day. It's overwhelming for most people. So this weekly benchmark is a good place to start. Here's why I'm encouraging this. When we talk to our customers all the time, we start to share our half-baked ideas with our customers. So if you talk to customers once a month or every other week, odds are you're talking to customers when you think you have your solution figured out and you're saying, hey, did we get it right? So that's a validation mindset where you're doing all the work on your own and you're going to your customers and you're saying, hey, did we get it right? I want to encourage you to adopt a co-creation mindset where instead of waiting till you're finished, if you talk to your customers on a continuous basis, you can, you're going to share half-baked ideas with them. You're going to encourage them to stand in front of the whiteboard with you. Maybe in the COVID days, it'll be virtually. You may be using Zoom whiteboards or Miro or Miro or Mural and get them to draw with you, share ideas and ideate with you. Now, this doesn't mean you're letting your customers dictate what you build. It just means you're adopting a co-creation mindset. You're getting their feedback much earlier in the process. I don't know how to, I can't express enough how important this is. This is, I think, the keystone habit to continuous discovery, where the more often we engage with our customers, the more likely we're going to test our designs, we're going to test our assumptions, we're going to catch when we're wrong, and we're going to get a lot closer to building things that our customers actually want. Okay, so that's that first line, weekly touch points with customers. Let's dive into this second one. by the team building the product so what do i mean by the team building the product many of you may have heard of um i think it was canadian tired by the way hopefully that was right um okay so many of you have heard of this concept of the product trio sometimes called the triad i prefer the simpler language of the trio it's this idea of the product manager the design lead and the tech lead working together through discovery making team decisions Now, this is who, at a minimum, I recommend do discovery activities together. We know from multiple decades now of building digital products that we need all three of these roles to be involved throughout the whole process to build products that are desirable, viable, usable, and feasible. So that's our goal here. Now, I know that you have other people on your team. You definitely probably have more than one engineer. Depending on your DevOps strategy, you might still have QA folks. You probably have some of these other roles, whether it's customer success or user researchers or data analysts or product marketing folks embedded on your squad with the rest of your team. So the question is, do you include everybody? At a minimum, I want you to include the trio. Whether or not you invite other people is going to depend on the types of decisions that you're making, the types of research that you're doing. If, for example, it's early on and you're researching your go-to-market strategy and desirability questions and how to position the product, you might want to invite your product marketer to participate in that research. If you're a very data-heavy product and your data analyst or your data scientist is critical to most of your decision-making process, you might want to invite that data analyst to participate in your trio. Here's the trade-off. The more people involved in discovery, the slower your discovery process will go. So we want to balance speed with inclusion. We don't want to include everybody just because we're nice and we want consensus that will grind everything to a stop. Instead, we want to think, be critical about what do we need to learn this week? What are our discovery questions? Based on those questions, who needs to be involved in the activities? Now, some teams ask me, why do I have to do my own research? Why can't I let my centralized user research team do the research? Why can't I use my business intelligence team to do the research? Why can't I let my marketing team do the research? Here's the thing. Those organizations can be extremely helpful. And if they can provide you with research, by all means, use it. But it's rare that they can answer the small questions you have week over week. Product teams work on a fast weekly cadence. where we're marching towards deliverables, we're trying to drive outcomes. Those other teams may have quarterly goals and they may work on that cadence, but odds are they're answering longer horizon research questions, whereas the product team needs to be getting those fast answers to all those little questions. If you do your own research, it's going to be timely, it's going to be actionable, it's going to be believable. So I strongly recommend the team. learn how to do their own research and to do it quickly and to do it week over week. All right. So we've covered weekly touch points with customers by the team building the product. These last two lines, we're going to dig in a little bit. So we're going to talk about what are you doing in these customer touch points? What kinds of research activities are you doing? And we're going to talk about doing it in pursuit of a desired outcome. Now we're going to talk about these two lines together. So especially with. the rise in popularity of design thinking. We see this a little bit with lean. We definitely see it a little bit with jobs to be done. We're seeing an increase in popularity of research methods. And sometimes we see teams get bogged down in research for the sake of research. So we interview customers indefinitely. We can always do one more diary study. We can always do one more on-site visit. We forget that our job as a product team is to drive product outcomes. So we're going to talk about balancing research with making progress. The best way I know how to do this is to manage your discovery process with an opportunity solution tree. This is a visual that I created. It is not the only way to do it. The key tenet here is to externalize your thinking. So what we're going to chart out here with an opportunity solution tree is to start with defining your desired outcome. So we're going to have an outcome mindset, not an output mindset. An outcome is what's the business metric? What's the product outcome we're trying to improve? So if you work at Netflix, it might be to increase engagement, increase minutes watched. If you work at Slack, it might be to reduce the time it takes to onboard a new team. It's really what's if you work for a SaaS company, it might be increase month over month retention. It's what we're starting with for all the product work that you're going to do. How are you measuring success? What's your goal? From there, we're going to discover the opportunities that if we addressed them would drive that outcome. So what do I mean by opportunities? Opportunities are customer needs, pain points, and desires. So they're opportunities to intervene in our customers' lives in a positive way. If you think about your outcome, let's say your goal is to increase engagement. You could think of lots of ways of increasing engagement. The best way to increase engagement is to address the customer need. So sometimes we see a tension between a business need and a customer need. So for example, let's say your goal was to reduce churn. You could think of lots of ways to reduce churn that didn't address customer needs. So for example, you could remove the cancellation workflow on your website and force your customers to call you. That would reduce churn, but it's not very customer-centric. It doesn't really address the customer need. So because in the discovery process, we want to really make sure that we're satisfying our customers in a way that drives business value. We're going to focus on what are the customer needs, pain, and desires that if we address them would drive our desired outcome. Once we've mapped out the opportunity space and we've prioritized an opportunity, we're going to focus on discovering the solutions that will address those opportunities. And this is where we're going to get into how do we figure out how do we actually make decisions about what to build in a way that addresses customer needs and drives our business outcome. So there's a few things that I'll highlight about this process. I did mention this visual that I'm sharing. It's called an opportunity solution tree. If you want to learn more about it, just Google that term and you'll get some of my resources on it. One of the most common questions I get is how is this different from an impact map or whatever your favorite visualization tool is. It's not really I mean, I don't know enough about impact maps to give you the pros and cons of each. Here's the goal. The goal is how do you externalize your thinking in a way to show your decision making to help your team align? This is one way to do it. So I'm going to walk through the way that I know well. if you have another way to do it just try to map some of the principles but you know actually earlier today i did a webinar on there's no one right way to do discovery and if i've learned anything it's that each team needs to find the way that really fits best for them so i try my best to just focus on underlying principles so for and i'll try to highlight that as we go through this so for example the first thing we need to do here is we need to define a desired outcome Right. So that's the top of the tree. And I argue that this really should be a two way negotiation between the product leader. And by that, I mean, your chief product officer, your vice president of product, whoever your head of product is and the product team. So by the product team, I mean that trio, your product manager, designer and tech lead. And the reason why this is a two way negotiation is because the product leader knows what the business needs and the product team knows what's possible. And then. when i talk about trying to identify the underlying structure and letting you find the means for how to do this i don't really care how you set an outcome you might use okrs you might use um the north star metric that john cutler popularized you might use um missions visions and strategic objectives um i the key is find what works best for your team but in this context the outcome is we're starting with a quantitative metric. So if you're using OKRs, it maps to a key result. If you're using the North Star metric, it might be your North Star metric. It might be one of the key inputs. Next, our goal is to discover the opportunities that, if we addressed them, would drive that outcome. Now, I personally think the best way to discover opportunities is to interview customers, to elicit specific stories, to learn about their lives. I know some people prefer jobs to be done stories. It turns out jobs are very similar to opportunities. It's a great way to do it. I know other people that, especially design thinkers, they really want to get out and observe their customers. If you have the ability to do that, that's an awesome way to find opportunities. So again, it's less about what's the tactic. It's finding what works for you, but making sure that you're using some of your time and your customer touch points. to explore their needs, their pain points, and their desires. So why don't we do this more often? Really the biggest barrier is it's hard to find people to talk to. So the very first thing I teach my teams is how to automate the recruiting process. Here's our goal. Our goal is for you to wake up on Monday morning, look at your calendar, and already have an interview scheduled with you not having to do anything. I'm going to give you three quick ways to do this. We do not have a ton of time to dive into the details, but I will be sharing some resources on how to dig in and learn more if you want them. So let's start with recruit people who visit your website. You've probably seen services that are doing this, where as people use your product or service, you can do this on a website. You can do it in a mobile app. It's a little bit harder if you have on-prem software, but it's not impossible. Somewhere in the process of using the product, you can ask them, hey, are you open to talking to us? We'd love to talk to you. Here's what we can do to compensate you. You can then use this to funnel people in a scheduling software and have them pick a spot and they end up on your calendar. This is a really powerful way to turn interviewing into any other weekly meeting. It just shows up on your calendar. You show up. There's someone to talk to. It's easy to do. For those of you that work on enterprise products, maybe you don't have a ton of engagement in your product, you can work with your customer facing teams to set up triggers. So the way this works is you work with a sales team or a customer success team or an account management team. And you just sit with a triggers in this form. If you talk to somebody who matches this criteria, then schedule an interview. And the criteria can change. You can change week over week. You could say if you talk to a customer that wants to cancel, schedule an interview. If you talk to a customer who wants to learn about this feature or has started using this feature, schedule an interview. So this is a great way to enlist the people in your company who are already talking to your customers every day to help them schedule an interview for you. Now, I worked with a company that had a very small, actually two different companies, that had a very small market base. Their total addressable market was tiny. One of them, their customers were Canadian medical schools. So not international medical schools, just Canadian medical schools. So a pretty small addressable market. I worked with another company based in Los Angeles where their customers were movie studios and they had a total addressable market of six customers. So if you only have six customers or you only have 100 customers and your total addressable market, you probably can't recruit them while they're using your service. You probably can't. Maybe you can use your customer facing teams, but one of the best things you can do is you can set up an ongoing relationship with them through something like a customer advisory board where you say, as a board member, we require that you participate in at least one-on-one interview each month. And then you give them compensation for being on your advisory board. So this is a way to almost build a design partner relationship with some of your customers and get long-term over time feedback from them. Okay. I'm going to have to skip through a couple of things just in the interest of time. When you're interviewing, I want you to do your best to avoid speculation. I don't want you to ask people what they would do. I don't want you to ask them what they think they do do. I want you to focus on one thing only, collecting stories. So you're going to collect specific stories. This is, tell me about the last time you watched Netflix, a specific instance, not what do you like to watch on Netflix? That's speculation. You probably don't know what you like to watch on Netflix unless you took the time to inventory everything you've watched on Netflix. So our goal is to avoid speculation, collect specific stories. All right. When we've mapped out the opportunity space, our goal is to dig in and explore solutions. You'll notice in this visual, all three solutions ladder up to the same opportunity. This is because our goal is not to work with one solution at a time. So odds are many of you have a backlog full of ideas and they all solve different problems. I want you to start to think about for a single problem, how do you consider a set of solutions? The reason why we're doing this is that decision-making research tells us we want to avoid whether or not decisions. We never want to ask, is this idea good or not? That's setting us up to be susceptible to confirmation bias. It's setting us up. to really see what we want to see instead we want to set up good comparing contrast decisions so we want to say which of these ideas look best so if we're trying to solve the problem of i can't find something to watch continuing with our netflix idea we want to look at three different ways to help people discover a show not one way to discover a show one way to fast forward through commercials and another way to um get friends and friends those are three solutions for three different problems So we're really trying to compare and contrast within the scope of a single opportunity. The visual that I use to explain this is, this is Usain Bolt. He was the world's fastest 100 meter runner at one point. If you saw him running around on a track and he was by himself, it'd be pretty hard to know if he was a fast runner. But as soon as you put him on a track with other people, it's really clear he's very fast. This is what we're looking for when we compare and contrast solutions. we're prototyping we're experimenting we're looking for a clear front runner so i hear all the time from teams that say i ran an experiment these are the results that i got are they good enough i don't know how to answer that question that's a whether or not question are they good or not we want to ask a compare and contrast question i compared these three prototypes i ran these three experiments i'm comparing and contrasting my results and i'm looking for a clear front runner Now, this is only possible if we test assumptions. A lot of teams test full ideas, and it's hard to compare and contrast and experiment with full ideas. It's a lot of work. But if instead we break our ideas down into their underlying assumptions, desirability assumptions, does anyone want it? Viability assumptions, should we build it? Feasibility assumptions, can we build it? We start to look at... What has to be true for this idea to work? We can map all those assumptions on an assumption grid. This exercise comes from David Bland. You can find it in his new book, Testing Business Ideas. What this does is it helps us identify the very specific assumptions that we need to test. It's often much faster to test assumptions than ideas. So you might be able to run half a dozen assumption tests in a single week. If you're running half a dozen assumption tests, it's now easy to compare and contrast across different solutions. Okay, we went through a whole lot, so I'm going to recap briefly, and then we can jump to questions. So let's talk through this. We started with our definition of continuous discovery. We're working against the desired outcome at the top. You're talking to customers every week where you're discovering opportunities. You're doing that through interviewing, collecting specific stories, avoiding speculation. Once you choose a target opportunity, you're going to work with a set of solutions, running experiments, prototyping to test specific assumptions. Based on what you learn about those specific assumptions, you're going to compare and contrast those solutions against each other, looking for a clear front runner. I have teams that are doing all of these activities every single week, 50 weeks of the year. That sounds crazy to a lot of teams. It's really about turning big research activities into small, iterative, incremental progress, fast assumption tests, continuous interviewing that allows us to constantly chart out our best path towards a desired outcome. If you want a copy of this visual, you can grab one at producttalk.org slash continuous discovery. And that URL is on the bottom there. And Mo, I think we're ready for questions. Awesome. Awesome. Thanks, Rita. That was an awesome talk. Loved it. So there was a whole bunch of questions around learning more about automating your recruitment process. When you talked about the fact that you're recruiting someone every day, that's incredible. Do you have any further tips on that? Yeah. So for the vast majority of teams that I work with, the way that they're recruiting is through their product or service. So they're finding a way to integrate an ask in the product. That could be like that pop-up that we saw earlier where when people visit the site, you pop up something. It could be like in your mobile app. It could be you just have something that's always present that takes people to a form or a survey that they can fill out. It could be some... companies just code in dynamic text areas throughout their product so they can constantly change the ask or change where it shows up there's a lot of ways to implement this the key to making it work is to match the right incentive with what you're asking for so and the philosophy here is to ask for something small in exchange for a big reward so the example that i showed um it was actually a a company called snag here in the us um they're they're a job board It's always weird as an American to say this to people from other countries because I have to explain that our minimum wage is embarrassingly low. In much of the United States, our hourly workers make as low as $7.29. That's actually our federal minimum wage, which I realize is atrocious and embarrassing. And so SNAG is helping people find retail and restaurant jobs, many of which are minimum wage jobs. So these are people that are making $7.29 an hour. I don't know, you may not have noticed, but on that screen, it said, we would love to talk to you for 20 minutes in exchange for $20. So that's a small ask. They're not asking for an hour. They're asking for 20 minutes and they're offering them $20. So for somebody that makes $7.29 an hour, $20 is a big reward. Now, if your customers are doctors, $20 isn't going to cut it. In fact, cash probably isn't going to cut it. You're going to have to get creative and find an incentive that they do care about. And I can share a couple of examples. I worked with a company that was trying to talk to car dealership owners. And most car dealership owners or managers are sitting in the car dealership all day. They have plenty of time. If you ask them to schedule an interview, they will. But at the time they schedule the interview, if a customer walks in, they're going to cancel the interview. Right? So, um they couldn't just pop something up in their service because it was really scheduling was really unreliable so they had to get creative um instead they said instead of scheduling it on the calendar they said here's our phone number not what's your phone number here's our phone number text us when you're available for an interview and they let the dealer choose when to schedule the interview so that's an example where they're making the ask really easy because they're letting the dealer choose the time real time like hey i'm available right now Another example is a company that sells a real estate broker, one of the big online real estate brokers, offered first-time home buyers access to a webinar about how to secure a mortgage, right? So if you're a first-time home buyer, you've never secured a mortgage, it's a scary unknown process. You may not even know what bank to go to. You may not know anything about interest rates or how to evaluate them or how to get a good one. They basically said, if you do an interview with us, we will give you access to this exclusive webinar. So the key is you're trying to find something that's really valuable for your company to offer. I mean, for your customer, but that's not super expensive for your company to offer. And then to pair that with a really easy ask. And this might take a little bit of experimentation. So you might launch something and nobody responds, in which case you've got to play with making the ask smaller and the incentive bigger. Awesome. That's some great suggestions. There's a lot of questions here. The one is around, let's say that people come out of your talk and they're like, we want to do continuous discovery. How do you get your organization to buy into it? What's the way to get your organization? There's a question from Keith. Yeah, this could be a whole other hour-long talk. But I'll do my best to give you some things to just try tomorrow. So the first thing is, Look for every opportunity to talk to your customers. So don't start by saying, I got to go to my company, I got to convince all my stakeholders that we got to radically change the way that we work. That's going to be a long, slow, exhausting process. I would just individually just start to look for, how can I sit in on some sales calls? How can I sit in on some account management calls? Do I have the ability to just call a customer? How do I start talking to customers more often? I would just start there. From there, every opportunity when you're with your teammates or you're with stakeholders and you're talking through ideas, odds are you're talking about one idea. So can you start to introduce this question, what else should we consider? Right? To transition from one idea at a time to sets, there's this magical question, what else should we consider? What else might we do? So you can start inspiring some comparing contrast thinking by just encouraging people to think about more ideas. And then from there, eventually you're going to have to tackle this sort of organizational change problem. And really what I recommend there is oftentimes we don't feel the pain of bad discovery decisions, right? So we're so hyper-focused on delivery that as long as we're shipping code every week, it feels like everything's fine. What we're missing, what we don't feel every week, is that a lot of that code is having zero impact, right? We're releasing things that don't work, they don't get used, they're hidden in the interface, people don't know about them. So we're not actually creating value. So one of the best ways to make the argument for more discovery is to start instrumenting your product. So measure everything. Before you release something, document the impact you expect it to have. Why are we building this? We expect engagement to go up. Great. Release it. After it's been live for the right amount of time, measure what happened and start to have a conversation about the gap between what you thought would happen and what actually happened. And I promise you there will always be a gap. And it's that gap that opens the door to a better discovery. How do we learn about that gap earlier in the process before we build the wrong thing? Yeah, that's awesome. I think that reminded me of something which is really powerful in that. if you do a like if you do a cohort of retention of um product of your product from one year ago to five years ago and you haven't done anything new and you see retention hasn't changed then everything you've built hasn't been effective so you're really using data to show that is is incredible there's a question uh from sabrina that was plus one by a whole bunch of people um that means many people have this so um how are these teams balancing discovery and delivery and are both being done in a single sprint. It seems like it's a lot. Yeah, it is a lot. It requires a pretty big mindset shift. So first of all, kudos to Sabrina for asking a popular question. Okay, so let's talk about this. So Marty Kagan has thrown around some percentages that I think are philosophically right. I don't know if they're practically right. So I'll talk about what I mean by that. So he argues a good product manager spends 80% of their time in discovery. and 20 of their time in delivery now if most of you are product managers that probably terrifies you because it's probably the other way around so we'll talk about how to get there for designers he recommends 50 50 and i'm not sure if he makes a recommendation on engineers but i think i've heard it's also 50 50. now here's the challenge the reason why for software engineers and for designers it's 50 50 is because they have heavy delivery responsibilities right your designer has to create delivery ready design and your engineer has to code whereas your product manager today you might have a lot of delivery responsibilities but theoretically you should not if we think about whether it's scrum or kanban the whole idea of these agile methods is for engineers to manage their own work right your scrum manager master as scrum was originally written was meant to be an engineer an existing member of your team Somewhere along the line, it got merged with the product owner and product managers started doing it. Product managers are not project managers and they should not be responsible for project managing delivery work. And it's really a requirement for us to shift to having a better emphasis on discovery for software engineers and for designers to manage their own delivery work. And I know it's possible because I see teams doing it. Now, Part of what makes this hard, like even getting to 80% on the product management side, is that it actually feels really good to be the person that has all the answers. So when our engineers come and ask us questions, delivery questions, or our business stakeholders come to us and ask delivery questions, or something goes wrong, or a customer is upset, a lot of product managers are in a hero role where they get to go fix the problems and go have all the answers. And that feels good. Like every time we help someone, it feels good. But the reality is that's not our job. Our job is to be out talking to customers and figuring out what to build. So some of this requires being willing to let go of some of the things that feel good and let our other coworkers pick up the slack and take responsibility for it. Got it. Okay, awesome. There's a question from John Stedek around what are some best practices for organizing interview feedback based on customer type, persona, and customer segment? So I'm imagining and I'm going to interpret the question. in the sense that if you're automating your recruiting process, you can get any customer, but you can be working on multiple products, or how do you segment your interviews better? Yeah, so even when you automate the recruiting process, you can still use screeners to make sure you're getting the right types of people. You can also choose where to show your pop-up or your question to recruit. So for example, if you're looking for people that are using a specific feature, you can show it on that page. So there's ways to fine tune your recruiting automation to make sure you're getting the right people. But the other thing that I heard in that question was just around how do you keep track of all the stuff that you're learning? One of the things that I recommend teams do is that they create what I call an interview snapshot. It's just a one pager that summarizes everything you're hearing in each interview. It's something that I cover. I actually have a four week. course on continuous interviewing. If you're interested in learning about that, you can find it at learn.producttalk.org. But it's just, it's a class that gets you hands-on help, hands-on practice, practicing the story-based interviewing that I talked about. And in that course, you get that interview snapshot template. But really it's just a visual one pager that helps you quickly summarize all the things that you've heard in an interview so that it becomes an index to your research. So that as you continuously interview, you're building up this knowledge bank that you can constantly reference as you face new product decisions. Got it, got it. There's a question from Jana around, does this change depending on what stage of the product you're at? Let's say if you are... launching a product would you still automate your recruiting process and do the same if you are in optimization or or let's say um sunset in a product is is this practice irrespective of what stage of product you're in um broadly yes the tactics may change a little bit so for example if you haven't launched your product you don't have a product to recruit participants um but i've seen startups before they have a product do when they're doing landing page tests, use their landing pages to recruit participants. If you are sunsetting a product, you probably want to be interviewing people about how to phase them out to something else. So I do think like a lot of the process that I designed for this, like a lot of the methods that I designed around continuous discovery really come from decision making and problem solving and critical thinking research. So I think they're. very broadly applicable from an ideology standpoint. Now, every team is going to have to adapt the tactics to find what works for their stage of product and for their specific customer type. Got it. There's a question from Duncan around what can you say on beginning discovery with uncovering desired customer outcomes? Okay. So that question is confusing a little bit because let's clarify two different things. So the blue box at the top of this tree is the desired outcome that actually should be a business product outcome. So what do I mean by that? Your business has business outcomes. Like if you're a SaaS company, it's going to be things like increase monthly recurring revenue. Maybe if you're an enterprise company with a big annual contract, it might be increased annual recurring revenue. If you're Facebook, it might be increased engagement because engagement is directly tied to revenue. right so you have these actually if you're if your facebook is probably increased um ads ad dollars brought in um so those are sort of your business metrics that keep the lights on that are tied directly to revenue from there we want to translate our business outcome to a product outcome so if we work with some of those examples um if we're facebook and our business outcomes add revenue our product metric is probably something like increase engagement increase um uh Maybe hourly active usage divided by daily active usage because you're Facebook. If you're that SaaS company, you might be looking at maybe you have a model that tells you that customers that use a specific feature on a regular basis are more likely to renew. So your product outcome is to increase engagement with that workflow or with that feature. So the blue box at the top is a business need, but it's a business need framed as a product metric. The green boxes are customer pain points, customer desires, customer needs. Those are customer opportunities. So how do you define that blue box? It starts with your business needs, and then you're translating into how does your product deliver those business needs. The green boxes are going to come from your customer interviews, listening to customer stories, and listening for customer needs, pain points, and desires. Got it. Okay, that's really helpful. And the last question is my question. In your writing, you talk about the importance of product managers or continuous discovery teams being visual synthesizers. I thought that was really good. Can you talk about what you mean by that and why is that important for continuous discovery product teams? Yeah, so I think this is directly tied to this idea of product trios being jointly responsible for their products. So it's actually really hard to collaborate cross-functionally. Most of us are used to working in business silos where you work as a product manager, you make decisions, you hand it off to a designer, they make decisions, they hand it off to an engineer. When we talk about collaborating, we need a way to get all the thoughts in our head out so that we can align as a team around a shared understanding. For most people, our inclination is to talk our way through that. The problem is language is really vague. And we're going to talk for hours and hours and we're going to get frustrated. We're going to stop going to meetings. We're going to want to just get back to doing the real work, even though aligning is real work. And we're going to give up on this collaboration and we're going to go back to silos and handoffs. I've found the best way to avoid that is to learn to externalize your thinking, to really do it visually. So this opportunity solution tree is one example of how to do that, right? We're visualizing our options. What's the best path towards our desired outcome? Experience maps, customer journey maps, story mapping. These are all ways of externalizing our thinking. Assumption mapping. Ways that allow us to do it tangibly, even if it's in a virtual space, so that we can all look at it and say, yeah, that's what I mean. Or no, I meant this instead. And it really allows a team to visually align around we really do mean the same thing. Got it. Awesome. All right. I'm going to... So with that, I want to thank Teresa for coming in today and sharing with us with your knowledge. And hopefully this gets people to start to think about continuous discovery and start to read up more on it. And I really think that that cornerstone habit of automating your recruitment process is absolutely critical to you becoming a good continuous discovery team. So with that, I would like to thank Teresa for coming in. So guys, please do spread the word on linkedin and twitter if you tag product faculty that you enjoyed it we'll keep on doing this and now we wanted to head over to the networking piece which is super cool so i'm going to head over there and teresa you can head over there too for one or two sessions essentially head over to networking and when you get there you said that i'm ready to network and that point you will be paired up with someone that you can connect with and once you connect with them you can chat with them for three minutes and exchange contacts and keep in touch after so let's head on over to networking and and thank you so much for everyone so see you guys over networking

Product management is evolving quickly.

Product teams are experimenting with their way to viable solutions. We are putting our customers first, taking the time to discover unmet needs, and developing solutions that address those needs.

Product Faculty is super excited to announce Teresa Torres for the next talk in the Toronto Product Mastery Series!

Teresa was our second guest speaker at the Toronto Product Mastery Series supported by Product Faculty (www.productfaculty.com)