The What & Why of Continuous Discovery by Teresa Torres at Lean Product Meetup

Let's get to the reason we're all here tonight, Teresa Torres. I'm super excited to host my friend after far too long. And she's, you know, a top discovery coach and a speaker I really miss. We used to always see each other at the conferences, the product conferences, and we just haven't had any in so long. So I'm super excited to see you here on Zoom at least. She writes awesome content at Product Talk. She's a graduate of Stanford, a symbolic systems major, and Northwestern. When we were doing this, you were studying still, and now you're a graduate. And then I just realized... We both went to Stanford Northwestern. I went to undergrad Northwestern. We flipped it around, but we're both grads from both, which is cool. And her Twitter handle again is at T Torres. You can see it on my screen. This is her brand new book, Continuous Discovery Habits. It just came out. So she's going to share with us the what and why of continuous discovery. So welcome, Teresa. Super excited to have you. And I'm going to stop sharing and let you share your screen. Perfect. Well, thank you, Dan, for having me. It's super fun to return. I apologize. It was so long. I just had to spend a lot of time writing a book because I know you know it takes a ton of work. So we'll learn a little bit more about the book. But basically today I want to talk about the what and why of continuous discovery. That's actually what the book is about. It's where I introduce just a really simple framework for if you're a product trio, so a product manager, a designer, and an engineer. and you're responsible for a product how do you make good decisions about what to build that's really what we're going to get into and for those of you that aren't familiar with me i like to share at the beginning of this talk that i've worked with teams all over the world i've worked with teams that are two people in a startup on day one all the way to companies with hundreds of thousands of employees that exist in many countries part of the reason why i like to share that is because the framework that we're going to talk through has been used in a lot of contexts. It's been developed with B2B businesses, B2C businesses, lots of different industries, regulated industries, audiences that are really hard to reach. And the reason why I like to start with that is because I always get questions like, I get that this works for Amazon and Netflix and Facebook, but it'll never work at my company. So I want to set the expectation that these methods and tactics have worked in a lot of contexts and a lot of places. They need adaptation, right? Like just like any tool or framework that you learn, you got to kind of take it and experiment and make it your own and figure out how to have it work in your unique organizational context. But really a lot of the ideas in this talk and also we'll see at the end in the book. have been battle tested. So what I would like to encourage you to do is to go into this talk with just more of a, rather than like having your guard up. And I know it's really easy when listening to a talk to just sort of feel like that's really cool, but it's never going to work at my company. And instead to kind of flip that on its head and be like, wow, that's really interesting. How might I do this at my company? I mean, that will really help you. Think about this from an experimental mindset and a continuous improvement mindset. And I think you'll get a lot more out of the talk. So with that caveat, let's go ahead and start at the very beginning. So I know discovery has become a really trendy topic. If you've been to a conference in the last five years or you've been to meetups, it's becoming more and more common, sort of this vernacular. But I really like to start at the very beginning because inevitably there's someone in the room that this is still new for. but we're gonna quickly advance, don't worry. So I always like to start at the beginning and just talk about what do I mean by discovery? So discovery is all the activities that we're doing as product teams to make good decisions about what we should be building. And I wanna contrast this with delivery, right? Which is all the activities that we're doing to build, ship and maintain production quality software. Now, Most of us are doing some discovery work, right? We're at the point in the industry where most product teams are doing some discovery work. And that's great, right? Whether that's your interviewing customers or your A-B testing or your prototyping or you're running usability tests, we're doing some discovery. We're seeing our discovery methods get a lot better in the last five, 10, 15 years. Here's the challenge. A lot of us are still taking what I would call a project mindset to discovery. Now, project-based discovery is not bad, right? Just like waterfall is not necessarily bad, but it has limitations. And we're starting to see as more teams mature in their discovery processes, we're starting to run into the limitations of project-based discovery. So let's just talk about what I mean by project-based discovery. It means you kick off a new project or initiative, At the beginning, you do some research, you talk to some customers, you summarize it in a research deck, you go several weeks while you're designing and building, trying to figure out what's the right thing to build without talking to a customer. Maybe once you've completed the design, you do some usability testing, you hand it off to the engineers and they go build for a long time and then you ship your product. And what's happening is that we're front loading the process with research and maybe right before we push to engineers, we're usability testing. But that's it. We're not engaging with customers throughout the process. And the problem with this is that that initial research is getting stale. We're remembering to ask the big questions, but we're forgetting to get feedback on the smaller everyday daily questions. We're validating, do we get the designs right? But if we get feedback that actually we got the whole concept wrong, it's happening too late in the process, that we can't actually integrate that feedback. So I don't wanna just talk about discovery today. I wanna talk about how do we shift from a project mindset to more of a continuous mindset? How do we continuously infuse our decisions with customer feedback so that we're continuously making better decisions about what to build? And this couples really nicely with what we're seeing on the delivery side, where more and more teams are pushing towards a continuous delivery model. Continuous delivery means a lot of things to a lot of people. For some companies, if you work at Amazon, Amazon shipping code every 11 seconds or something ridiculous. If you work at a more conservative company where you're deploying maybe once a week or even once a month, it doesn't mean that you can't have a continuous mindset. So here's the idea. We wanna think about the idea behind continuous improvement is how do we continuously ship value to our customers? It's more than continuously shipping code, right? We wanna make sure that that code is valuable. It's creating value for a customer in a way that creates value for our business. And that's really what we're gonna get into tonight is how do we shift our discovery activities so that they're continuous and we're able to continuously ship value to our customers. And even if you're not continuously delivering, I want you to take a continuous improvement mindset to this whole process, right? So you're gonna learn some new things that might feel overwhelming. Next week, you don't have to suddenly wake up and be like, okay, I'm a continuous improvement team. We're continuously delivering. We're continuously discovering. Instead, you can say, how do we make next week a little more continuous than last week? How do we just get a little bit better over time? So that eventually you're finding yourself adopting a more continuous cadence. Now, i have worked with a lot of teams and not like 98 of teams that i work with they come in and they say teresa we're already doing continuous discovery and that's because they are doing discovery activities frequently but they're still doing them from a project mindset so i want to start with a really simple definition of continuous discovery so that we can really get a clear picture of what does this look like and what does this mean and then we're going to spend a lot of our time just breaking this down I do love that people are posting in the chat because it makes me feel like I'm not just sitting in my office by myself. So please keep doing that. I love it. And Dan, I'm going to trust if there's something in the chat that I should stop and respond to, please feel free to pipe up. Yeah, no, I think we should be good. Yeah. Okay. So let's talk about continuous discovery. What does this look like? So I define continuous discovery as at a minimum weekly touch points with customers by the team building the product. where they're conducting small research activities in pursuit of a desired product outcome. Now, this is a mouthful. There's a lot here, and we're going to go through it line by line because really every line is a big deal here. I spent a lot of time on this definition, and I think it includes the key components here. So we're going to start with weekly touch points with customers. So as product teams, we make decisions every day. Some of them are big strategic decisions, like what should go on our roadmap. What customers should we serve? What opportunities do we go after? What markets are we gonna play in? We're usually pretty good. That's where our project-based research shows up, right? We know for those big strategic decisions, we need to do a lot of research. We need to go do our homework before we really make big investments. What we sometimes forget is that even our small to medium-sized decisions need customer feedback, right? This is why we see products where the data model limits what's possible and companies find themselves hearing about customer needs that they simply can't meet because they didn't do the research up front they didn't work with customers as they developed the product the example this could be anything from how we label our buttons where we expose things in the interface how a workflow works these are all the daily decisions that we're making that can make or break a product And the example that I like to give for this is if you pick up your phone, I'm sure it's in arm's reach because we're all addicted to our phones, and you scroll through your apps, I'm sure each and every one of you has dozens of apps that you downloaded at one point because you were really excited about the big idea of the app, right? So the strategic context of the app, they got it right, but you stopped using it over time. Some of those daily decisions weren't close enough to how you wanted to use the product. It didn't quite work. and they didn't keep you engaged over time. And so one of the values of continuous discovery is it helps to make sure that we don't just get the download, we don't just get the sell, we don't just get the big headline, that we build a product that works for our customers over time. And all those little small and medium sized decisions, it's just as important that we get customer feedback and that we integrate them into our process so that we're guaranteeing that we're building something that somebody cares about and that somebody needs. And this is really different from a lot of what we read about when we talk about discovery. A lot of teams take what I call a validation mindset to discovery, right? So the product team chooses what problems to solve. They brainstorm solutions. They do all the design work. And then the last minute, they put it in front of customers to validate, did we get it right? And this validation research is important. We do need to validate, did we build the right solutions? But there's a lot of discovery that happens ahead of that. So what I really encourage, because we're making product decisions every day, I want to see teams engaging with customers every single week. So those small and medium-sized decisions, you have a lot of chances to get feedback from your customers. And more importantly, you're getting feedback earlier in the process so that you have time to act on the feedback. When we engage with customers, when we don't have a shiny mock-up, when we just have a half-baked idea or a pencil drawing, or we're just exploring what problems to solve, what happens is we shift from this validation mindset to more of a co-creation mindset. We start co-creating with our customers. Now, inevitably, when I bring this up, somebody is thinking, I don't want to co-create with my customers. Henry Ford said that if I asked what customers wanted, they would have said a faster horse. Or they talk about that Steve Jobs quote where he says, customers don't know what they want until we show them. So I want to be really explicit about what this means. When we're talking about engaging with customers every week, we're not asking them what should we build. We're asking them about their lives, about their context, about their world. We're learning about them. Because while we are experts on technology and what solutions are possible, Our customers are experts on themselves, their own lives, their own needs. And so our goal when we talk about continuous discovery and co-creation is how do we fuse these different strains of expertise, right? We want to engage with our customers regularly so that we're constantly learning from them about them while we're making decisions about what to build. And this is really powerful. I know some people get overwhelmed when I start to talk about my continuous discovery framework. It's going to sound like a lot. And I'll tell you this first piece is the most important step. If you're looking for a place to start, I want you to think about how do I engage with customers more often? Okay, so that's our first line of the definition. Let's get into the second line, which is by the team building the product. So what do I mean by the team? So Dan alluded to this at the opening. I really teach a trio based approach to product and to discovery. So what does this mean? A product trio is a product manager, a designer, and a tech lead. In most instances, it's your design lead and your tech lead. Sometimes we swap out other people. Here's what's key. This trio needs the authority to make decisions about what to build. So if your tech lead isn't into discovery and they don't want to be involved and they want to just focus on is the data model sound, you can have a different engineer participate. But that means the tech lead has to seed those decisions about what to build to that engineer that is part of the trio. So that's the first important thing is that this trio is responsible for making those discovery decisions. The reason why we want the team that's building the product driving their own discovery is we want them to have first-hand exposure to the customer we want them co-creating with the customer we don't want somebody else in the building doing the research and then handing it off to this team here's what we know from past experience when we hand research off it's not actionable it's not believable people don't trust it they don't act on it right it's really easy to look at somebody else's research and say that doesn't jive with my experience i'm not acting on that um also Most of the other teams in our organizations that are doing research are doing longer horizon research. So centralized user research teams are usually doing month-long research projects. Your business intelligence teams, your market research teams, their research is all valid and good and we should rely on it, but not in lieu of our own discovery. That research is happening on a different time horizon. They're going after bigger strategic questions. Product teams need fast answers to their daily and weekly questions and that's what discovery is for it's how do we get fast feedback from our customers about the daily decisions that we're making now inevitably the question that comes up here is what about all the other people on my team whether it's your squad or your scrum team or whatever name you use odds are you've got other engineers depending on your devops strategy you might have a qa person depending on how your other business teams interface with your team You might have a user researcher or a data analyst or a customer success person, or maybe even a product marketing manager on your team. Some people make the mistake of saying, let's include everybody in discovery. What happens when we have eight people in the room and we're trying to make a decision? We slow way down, right? So as a product team, if we're trying to continuously deliver value, we want to move fast. So here's the idea of the trio. We're talking about true cross-functional collaboration. We need the right roles in the room for the type of decision we're trying to make. At a minimum, for most discovery decisions, we need at least an engineer, a designer, and a product manager in the room. Depending on the decision that you're making, your trio can flex to include other people. So for example, if you're doing discovery on your go-to-market strategy, you're probably inviting your product marketing manager to be part of that trio, right? If you're working on a data-heavy product and you need your data analyst in most decisions, your trio might turn into a quad. So don't think about this as a hard and fast rule. The idea is for the given decision that you need to make, what are the cross-functional roles that need to be represented to make sure that you're making a good decision? And then no, don't... err on the side of just inviting everybody because the more people in the decision the slower you're going to go so that's this idea of the team that's building the product so if we go back to our definition we've covered sort of these first two lines but um weekly touch points with customers by the team building the product so you're a product trio you're meeting with your customers every week what are you doing with customers each week what's happening during those customer touch points our goal is to conduct small research activities where the goal of those research activities is to pursue a desired outcome. Now these two lines work together and I've seen it worked really poorly when they're not coupled, right? So I've worked with teams where they're really good at research and they're engaging with customers and they're creating experience maps and they have personas and they're interviewing customers constantly and they know a ton about their customers, but they forget that their job is to ship value to customers in a way that creates business value, right? And so They don't build anything. They don't ship anything. They just do research because research is fun. I've also seen the other side where people hyper focus on an outcome and they forget about, it actually takes research to figure out the best way to reach that outcome. Instead of doing that research, they're just throwing spaghetti at the wall, hoping that they meet their outcome or they're hyper focused on business value and they forget about customer value. So for the rest of the talk, I wanna talk about how do we keep these lines? these two goals balanced together? How do we make sure that our discovery activities are in service of reaching our outcome? And to do this, I'm going to introduce what I call an opportunity solution tree. So this is a visual that's going to help you map out the best path to your desired outcome. I like to think about it as a discovery roadmap. How do we keep track of all of our options? How do we keep track of the decisions that we've made? How do we get a big picture view? of the best path to our outcome so we can be more strategic about how and where and how we spend our time. So it starts with defining a clear outcome at the top of the tree. Your outcome is gonna set the scope for your discovery. Now there's a lot of misunderstanding about what this outcome should be. So I wanna be really clear. Your outcome represents the value you're gonna create for your business. So it's business value. That doesn't mean it's a business outcome. So this is language that I introduce in my book. So I wanna break this down a little bit. Business outcomes usually are financial metrics. Every business is trying to grow a profit. They wanna increase revenue or decrease costs. So business outcomes are things like grow revenue, grow market share. Maybe it's reduce costs. So you could reduce customer support tickets, things like that. Most product teams don't directly impact business metrics. The product impacts business metrics. So for most product teams, you need to find a product outcome, a metric that measures something that's happening in the product that directly drives that business outcome. So I'll give an example of this. You might have a business outcome. The business says we need to reduce churn. That's the most important outcome that will create business value right now. What we wanna look for is what's a behavior in the product that's indicative that somebody won't churn so maybe this is engagement we're going to increase engagement that's a engagement in the product that's a product outcome that will drive our business outcome of reducing churn this is the translation layer that's often missing so the outcome at the top of the tree needs to represent a business need But it's not that business financial metric. It has to be translated to a product outcome that the product team can influence by changing behavior in the product. So that's the beginning of this. From there, we need to discover what are the opportunities that if we addressed them would drive that business outcome or that product outcome. Now, opportunities is a little bit of jargon. It just means customer need, pain point, or desire, right? So we're trying to be a little more inclusive than just problem. There's lots of products that don't solve problems. My favorite is I really like ice cream. It definitely doesn't solve a problem for me. In fact, it creates problems for me, but it does address a desire, right? So opportunities are needs, pain points, and desires. Once we discover opportunities that could drive our outcome, we now need to discover the solutions that will drive those opportunities. Now there's a really big, there's a subtlety here that I think is really powerful. So I talked about we need to keep our research activities in service of a product outcome. And I've talked about balancing customer value and business value. Our opportunity solution tree is helping us align those. So I've met teams that are really good at creating customer value, but they forget to create business value. And what happens? Their product gets shut down because it doesn't create enough business value. I've also met teams that create business value. but they forget to create customer value. And what happens there is we actually find ourselves in ethical quandaries, right? Where we're maximizing business value at the expense of the customer. A good product team knows we need to balance these. And the way that we're gonna balance these is opportunities represent customer value. That's the value we're gonna create for customers. The opportunity space is infinite. So we're only going to consider the opportunities that have the potential of driving our business outcome. And this is how we're going to align customer value with business value. Okay, so we've got an outcome at the top of the tree. The way that we're going to set this is it needs to be a two-way communication, a two-way negotiation between your product leader and the product trio. So what do I mean by product leader? I do not mean the product manager on the product trio. I mean your chief product officer, your vice president of product. If you're a really large company, it might be the GM of your business unit. It's the executive level leader who can tell you what business value your individual product tree can create. So this is the leader that's going to communicate to you. Here's how your team can create business value. The reason why this needs to be a two-way negotiation is the product trio needs to communicate how much value they can create on what timeline, right? So if the business leader says, I need you to reduce churn, the product team might translate that to a product outcome of increasing engagement with the product. And they might say, we think we can increase engagement by 10% in the next three months. Then there needs to be a negotiation. Does that create enough business value, right? So this is a two-way conversation. From there, once we have an outcome that's setting the scope for our discovery, we want to see our product trio go out and start their weekly cadence of customer touch points. The first thing we want them doing is interviewing to discover opportunities. Now, this is a big misunderstanding in the industry. A lot of people interview and they show solutions and they say, hey, customer, what do you think of my solution? And they don't get reliable feedback. The goal of an interview is to understand your customer's context, their needs, their pain points, their desires. So we're interviewing to surface opportunities. Now, weekly interviewing is not gonna happen overnight. It's not easy, right? It's gonna take a little bit of infrastructure to be able to support weekly interviewing. And the reason why most teams don't interview every week is because recruiting is hard. Most of us have to hustle to find an interview. So what I wanna tell you, the key to unlocking a continuous cadence is to automate your recruiting process. Here's your goal. You wanna wake up on Monday morning and there's already an interview on your calendar and you had to do nothing to schedule that interview. It just magically appeared, right? This is what unlocks a continuous cadence. If there's automatically an interview on your calendar every week, It's just another meeting. You just have to show up. It's easier to interview than to not interview. I'm gonna give you several tips for how to automate the recruiting process. The book's gonna go into more detail. This is the key to unlocking continuous discovery. So if you only take away one thing from the entire talk, write this down. Your goal is to automate your recruiting process. This is the key to unlocking a continuous cadence. I'm going to share the most common ways of doing this. So for 90% of the teams that I work with in all kinds of industries, all sizes, B2C, B2B, these are the three most common ways. The first is recruit people while they're using your product. So this is a job board. They snag a job. They help hourly workers, think restaurant workers, retail workers, find a job. How are they recruiting? They have an interstitial that they show to a very small percentage of their participants. And they ask them, hey, we'll give you 20 bucks if you spend 20 minutes with our team. Here's what I want to highlight. If you're going to experiment with this, it will take experimentation to make it work. You're going to launch it and you're going to get no responses. And you're going to have to play with what are you asking for and what are you offering in exchange. The reason why I use this Snagajob example is because it's a really good example of asking for a small amount of time, 20 minutes, and offering a big reward. Now, as a tech knowledge worker, $20 might not sound like a lot of money, but if I'm a restaurant worker or a retailer worker in the U.S. making federal minimum wage, which is $7.29 or something absurd like that, $20 for 20 minutes of my time sounds really appealing. This is an easy yes. That's the key to recruiting people while they're using their service. Now, if you're a B2B company and the people you need to interview are not in your product all day every day, we have some other strategies. You can use your customer facing teams to help you recruit participants. So who do I mean by this? Your customer support teams, your sales teams, your account management teams. The way that you wanna automate this is you wanna define triggers for them. So here's how a trigger might work. If you're on the phone with a customer who wants to cancel, offer them this reward and ask them if they'll schedule 20 minutes with a product team, right? The trigger could be anything. It could be if you learn about a customer who's using this feature. If you talk to a customer who's experiencing this problem, right? And you can even change your triggers every week based on what you're learning week over week. But you can really empower your... the teams in your companies who are already talking to customers every day and help them enlist them to help recruit interview participants now i have worked with some companies where their total addressable market is tiny i'm going to give you some examples i worked with a company where they sell to movie studios in the united states there's six of them their total addressable market is six customers I worked with another company that sold to Canadian business schools. I don't know how many Canadian business schools are, but let's say dozens, maybe a hundred, right? Small addressable market. When that's the case, what you can do is you can set up a customer advisory board and interview your advisory board members. I want to be clear, I'm not talking about focus groups. Let's say you have 12 people on your customer advisory board. If all 12 of them are required to do a one-on-one interview every week, you now have enough people on your board to support three product teams doing an interview every week so you can scale this idea based on the number of teams that you have now the challenge with this method is that you're going to talk to the same people over and over again and there's some advantages of that you're going to learn how their needs change over time you're going to learn about their context in depth but there's some limitations because you might design there's only six customers in your total addressable market you don't have to worry about if they're outliers because you're talking to all six of them But if there's 300 customers in your total addressable market and you're only talking to 12 of them, you wanna make sure you couple this with some of the other strategies and that you're also talking to a wide variation. I will share, 90% of the teams that I work with, one of these three strategies works. And a lot of the teams that I work with are using multiple of these methods to guarantee that they have somebody to talk to every week. So I would say start here. We do cover a lot more ways to automate the recruiting process in our continuous interviewing course, if you want to dive in and learn more about this. I also go into more detail on all three of these methods in the book. Okay, so we've addressed, we've recruited people. We show up every week on Monday. We have an interview on our schedule. How do we ask the right questions? So remember, our goal is to discover opportunities. So we're not going to show prototypes. Yet, that comes later. We're not going to show prototypes. We're definitely, when we show prototypes, we're not going to ask, what do you think of this? So we need to talk a little bit about how do we ask the right questions. Your goal is to avoid speculative questions. Here's the challenge. Human beings, because of the way our brains work and because of cognitive biases, we're not very good at answering questions about our own behavior. So if I interview Dan, let's say I work at Netflix and I interview Dan, and I want to know about Dan's Netflix usage. If I ask him things like, what do you like to watch? Who do you watch with? What devices do you watch on? These are all the things I wanna know, right? Dan's answers aren't necessarily gonna reflect his behavior. And it's not because Dan's lying to me or he's trying to be deceptive, right? It's because Dan's brain is a human brain and it's susceptible to all sorts of cognitive biases. And so his answers aren't necessarily gonna reflect his actual behavior. We see this a lot, right? Gyms make tons of money on people who pay monthly subscriptions and never go to the gym. And that's great. If you just need to make the sale and have no engagement, then great. You don't have to worry about do their answers reflect reality. Most of us, we need to engage people past the sale. And in our interviewing, our goal is to uncover actual behavior. So you might have heard right on the internet, heard somebody that gives interview advice say, ask open-ended questions. That is true, we wanna ask open-ended questions, but a lot of open-ended questions are speculative. So if I say, Dan, tell me about your experience on Netflix, that's a great open-ended question, but it's not gonna tell me about Dan's actual behavior. It's a speculative question that's gonna lead to unreliable feedback. So here's what I want you to do instead. I want you to ask a question where you're collecting a specific story. So I'm not gonna ask Dan, tell me about... your Netflix experience, I'm gonna ask Dan, tell me about the last time you watched Netflix. And I'm gonna get into all the nitty gritty details. Where were you? What were you doing? What did you watch? Who are you with? What device were you on? How many episodes did you watch? How did you decide what to watch? And I'm gonna mine that story. I'm gonna collect as much nuance and context and detail as I can. And Dan's answers, because we're talking about a specific instance, are going to match his actual behavior. This is where we're gonna see insights go way up and the reliability of his responses go way up. So this is the key. If you wanna discover opportunities and reliable opportunities, you wanna focus on collecting specific stories. Now, I just spent a lot of time talking about interviewing and it's because it is the most important continuous discovery skill. We often build great products that solve the wrong problems. right it's a beautiful shiny solution it solves the wrong problem the heart of continuous discovery is uncovering opportunities mapping out the opportunity space making strategic decisions about where we can play so if you want to start somewhere i recommend automate your recruiting process and focus on collecting specific stories listening for needs pain points and desires when you start to collect opportunities this tree structure can help you with how do we get a big picture view of all the things we might do for reaching our desired outcome when we do that we want to prioritize i really encourage teams to work on one opportunity at a time this is key because it's going to help us hyper focus and evaluate multiple solutions for the same opportunity now most product teams are drowning in ideas they think they already consider lots of different ideas i want to be really specific here we're talking about a teeny tiny opportunity a really small customer need pain point or desire and we're considering multiple ways of solving it what this does it's going to unlock a compare and contrast decision that's going to really improve the value of your solution so let's talk through this a little bit most of us work with one solution at a time right we just work through our backlog we're looking at the top the top customer problem we come up with one solution the a b test it we get prototype feedback we're asking is this idea good or not Here's the challenge with this. Confirmation bias is gonna come into play. The escalation of commitment bias where we fall in love with our ideas is gonna come into play. Our ideas have flaws. It's gonna be really hard for us to see the flaws in our ideas when we work with one idea at a time because of the way our brains are wired. So what we wanna do instead is we wanna come up with at least three ways to solve the same opportunity. We wanna compare and contrast them against each other. We're not gonna say, is this idea good? We're gonna say of the consideration set, which one looks most promising? This alone unlocks better solutions because you're gonna, you might be doing all the right things by prototyping and running assumption tests and testing your ideas, but you might be falling prey to confirmation bias and escalation of commitment and you're not really getting value out of those activities. When you compare and contrast, it's a lot easier to see which one is the front runner. The visual that I like to use for this is I want you to imagine that you're watching Usain Bolt running around a track and I ask you, is he fast? It's actually hard to answer that question. Is he fast relative to what? If you don't know who Usain Bolt was, at one point he was the world's fastest 100 meter runner, right? So if I ask you, is he fast? Is he fast relative to a cheetah? Probably not. I'm gonna guess a cheetah is faster in the first 100 meters. Is he fast relative to a Tesla? I actually would love to see this race. I want to know who's faster in the first 100 meters, Usain Bolt or a Tesla. Is he fast relative to other humans? Absolutely, right? So the picture on the right is a clear compare and contrast where we're looking for a clear front runner, a solution that stands out from the rest. So when we set up a compare and contrast decision, we see the pros and cons of each of our ideas. Whereas when we work with one solution at a time, We fall in love with it. We don't see any of the flaws. We just see all the pros. So keep this image in mind. We wanna be comparing and contrasting so that we're truly evaluating our solutions. Okay, so we're gonna, how do we do that? We've got three solutions. It's already hard enough to prototype one solution. We definitely can't build all three and AD test them. So here's the key. We wanna unlock fast discovery cycles so we can't prototype the whole idea. We can't build the idea and A-B test every idea we have. The key is to take the idea and break it down into its underlying assumptions because assumptions are faster to test. That's what's gonna unlock fast iterative cycles. So what do I mean by that? Every idea is built on a set of assumptions. Some people talk about this as risk. I like to talk about it as assumptions because I can draw a direct line between an assumption and the type of assumption test I'm going to run. So let's start with this. Every idea, we make assumptions about why people want it. So desirability assumptions. Desirability is bigger than do people want this assumption. It also includes are they willing to do the things we need them to do. So if the solution requires that they set up an account and they remember a password and they can figure out where to go and they're willing to do all of those things, those are desirability assumptions. Viability assumptions are this solution is gonna create value for the business. And this is where your outcome can really help because this solution is gonna address an opportunity in a way that's gonna drive our outcome. So it's gonna create value for the business. Feasibility assumptions are, is it possible? Can we do this with technology? Can we deliver on it? I also like to include in feasibility things like, will our security team sign off on it? Will our legal team sign off on it? Will the company accept it as a feasible solution? Some people put those assumptions in the viability category. It does not matter what category your assumptions fall in. Think about it as the categories help you generate assumptions. Now I like to add two categories to this Venn diagram. One, we're pretty darn good at, and that's usability assumptions. So we're getting pretty darn good at testing our usability assumptions, right? Do people understand it? Can they find it? Are they able to do what we need them to do? In the accessibility range, we still have some work to do. I regularly see websites where I cannot read the small print. Thankfully, my browser helps a little bit with that. For those of you that are not yet in your 40s, you are going to encounter this problem and you're gonna wish you could go back to your 20 year old self and design things for people that are losing their near vision. But generally, we're pretty good at usability assumptions. The category that we are really bad at are ethical assumptions, right? Is there any potential harm? Now, for a long time, I've added this category, this fifth category for years, and I always framed it as What are the assumptions you're making about the data that you're collecting, right? So do you need all that data from your customers? Do your customers understand all the data that you're collecting? Are they okay with it? Do they know how you're using it? Do they know who you're selling it to? Do they know how those third parties are using it, right? There's a lot of ethical concerns here. But the past year or so, I've really expanded this category. There's a lot more that we need to take into account. We need to ask questions like, Who is this solution serving? Who's it leaving out? Am I only testing with our majority privileged populations? Are we assuming that our users have a broadband internet connection? And if they don't, are we building software that people in rural areas need and won't be able to access because they don't have a broadband connection? So ethical, I go into this in a lot of detail in the book. I really would love to see teams put a lot more effort into their ethical assumptions. Okay, so we're gonna take our idea, each of our ideas. So we're working with sets of ideas. We're gonna take each of our ideas. We're gonna break them down into these categories of assumptions. What's nice about this is a lot of our assumptions, our ideas are gonna share assumptions. And so we can test multiple ideas with a single assumption test. So I'm gonna give you an example of this. I'm gonna stick with Netflix because I know everybody's familiar with Netflix. So imagine that I work at Netflix and I'm trying to evaluate. should we add sporting events to the Netflix platform? So the opportunity is I wanna watch sports. I'm a Netflix subscriber and I wanna watch sports. And so if I'm a product manager, I'm on a product trio and I'm trying to evaluate solutions, maybe I've generated three solutions. One is I could integrate local TV channels like ABC, CBS, NBC for my Canadian friends, CBC. We can integrate that into the next Netflix platform. And the reason why we think that's a good solution is because a lot of sports are on those network channels. Second solution, maybe we should just partner with Major League Baseball and the National Hockey League and the NFL and license their individual games so they actually appear in the platform. And you could search for them just like a movie. So I could say, I wanna watch the Edmonton Winnipeg series. I'm clearly a hockey fan. And I wanna go find that game and I can just find it on Netflix. Maybe my third idea is actually, I don't wanna be in the sports business at all because FuboTV has that covered. And I just wanna partner with FuboTV and we're gonna bundle a Netflix solution with a FuboTV solution. Okay, we've got three different solutions. Some of the assumptions, like one assumption is common across all three ideas. The assumption is this simple. It's a desirability assumption. Netflix subscribers want to watch sports, right? All three solutions depend on that assumption. Here's the value of this. I don't have to build anything. I don't have to prototype anything to test that assumption. I can test that assumption before the end of the day. Here's a few different ways I can do it. I can look at my search queries. I work at Netflix. I can look at our search queries and see if anybody is searching for sports today. That's a really good way to evaluate do Netflix subscribers want to watch sports? Second way I could test that assumption, I could do a one question survey. Maybe I put it somewhere in the product where in Netflix, it just says, do you like sports? And there's a thumbs up and a thumbs down. I actually want to ask a better question because I want to ask about actual behavior. So I probably want to ask a question like in the past week, have you watched a sporting event? Yes or no, right? Simple one question survey. We have a lot of tools that allows us to collect this type of feedback by the end of the day. I just tested an assumption that if the assumption is not true, I can throw out all three of my ideas and move on to a new opportunity. Okay, now I need to test assumptions to distinguish between my solutions. So for example, if I'm gonna integrate local channels, the first assumption I need to test is the sporting events that my subscribers wanna watch are on those local channels, right? Again, I don't have to build anything to test this assumption. I can collect feedback really quickly. For the FuboTV solution, maybe the assumption I need to test is people are willing to have multiple subscriptions, right? So there's lots of ways to do this. And what's great about assumption testing is the best teams are testing half a dozen to a dozen assumptions every week. And that's the cadence that allows us to look at a set of solutions and quickly iterate and find the ones that are gonna work. Okay. We just covered a lot of ground. We started with an outcome at the top. We talked about interviewing to discover opportunities. Remember, with interviewing, you need to automate the recruiting process. You need to collect specific stories so that you can identify pain points, needs, pain points and desires. You're working on one opportunity at a time so that you can explore multiple solutions, comparing and contrasting them against each other. The way that you're doing that is you're taking each solution, you're breaking it down into its underlying assumptions, desirability, viability, feasibility, usability, ethical assumptions. And then you're finding fast ways to test those assumptions. And here's what I'm going to tell you. We are living in the heyday of discovery tools, right? With usertesting.com, tools like Ethneo, Usabila, Intercom. um user zoom we have so many ways of getting fast feedback from our customers that we now can have an idea on monday and by wednesday throw it away because we know it was built on half a dozen faulty assumptions or we could find that only two or three of the assumptions were faulty we redesign it and by the end of the week we found something that can work if this sounds crazy to you i want to tell you there are teams working this way right now it is possible this is not a theoretical ideal way of working It can feel overwhelming because I know for many of you, this is very different from how your organizations work today. So what I want to leave you with this is take a continuous improvement mindset for how to get here. You don't have to do all of this next week. And I'm going to tell you, your starting point is literally talk to a customer. Increase the frequency at which you talk to your customers. As Dan mentioned at the beginning of the talk, I did just release a book yesterday. I am super excited to get this out in the world. Yesterday was one of the best days of my life. I was completely blown away by the enthusiasm and excitement for this book. I literally spent all day reloading Twitter and LinkedIn and interacting with people and just really amazed about how excited people are about this book. So I wanna tell you a little bit about it before I wrap up and we do questions. I designed this book. to be a pragmatic, actionable how-to guide for product trios. So if you wanna know how you can do continuous discovery week over week, always know what to do when this book was designed for you. There's two, it's actually, this is the outdated slide. So Marty Kagan has a fantastic quote that's on the back cover of the book. He basically said, if you didn't have the luxury of working with a product coach or had a leader who could coach you, this book can replace that and help you be on the right path. That means a lot to me because we all know Marty Kagan is like the missionary that is teaching us all to work this way, right? That was my goal with this book is to put something in your hands that you can reference that can help you find your way towards a continuous cadence. Jocelyn here in her quote talks about, it's like a workout routine. And I love that analogy, right? It's a little bit like a workout coach. And it's because it does take repetitive week over week activity to help you get there. So please check this out if you're interested. If you go to continuousdiscoveryhabits.com, you'll find some testimonials and some links and options for buying the book. If you do buy the book, forward your receipt to receipts at producttalk.org. If you get one of the free books from Dan, Just forward whatever email you get from Amazon saying you got a copy of the book. We're working on a goodie package for everybody who bought the book, just supplementable content that you can, that'll help you put the book into context. All right, Dan, I think I'm ready for Q&A or a little chat if you are. So I want to thank you very much for your time. We have a lot of positive feedback in the chats and tweets and things like that. So thank you very much, Teresa, for... sharing your book launch with us and sharing your advice. Really appreciate it. Thank you, everybody. This has been a lot of fun. And thank you, Dan, for having me. I really appreciate it. My pleasure, Teresa. Great to be with you. Take care.

Teresa Torres gave this talk "The What & Why of Continuous Discovery" at the Lean Product Meetup on May 20, 2021.

Teresa Torres is an internationally-acclaimed author, speaker, and consultant. She teaches a structured and sustainable approach to continuous discovery that helps product teams infuse their daily product decisions with customer input. She’s coached hundreds of teams at companies of all sizes, from early-stage start-ups to global enterprises, in a variety of industries. She has taught over 7,000 product people discovery skills through the Product Talk Academy. She’s the author of the upcoming book Continuous Discovery Habits and blogs at ProductTalk.org.

If you would like to see more videos like this check out the following talks from past months: 

▶ Dan Olsen's Product-Market Fit talk at Mind the Product SF 2018:
https://www.youtube.com/watch?v=bDUrQlmwox4  

▶ Jake Knapp about his new book “Make Time” :
 https://www.youtube.com/watch?v=q4Sx9o85y-s

▶ Box CEO Aaron Levie Interview: 
 https://www.youtube.com/watch?v=cJC_JuxRgIg

▶ “Product is Hard”  by Marty Cagan
https://www.youtube.com/watch?v=gCYFmrvPI8Q 

▶ “Jobs to Be Done” by Tony Ulwick: 
https://www.youtube.com/watch?v=qQFUHapOJsQ

▶ Using Data to Set Your Product Strategy by Justin Bauer, who is the VP of Product at Amplitude:  
https://www.youtube.com/watch?v=H8XQVQy8Xiw  

▶ For upcoming videos like this, subscribe to our channel: 
http://www.youtube.com/c/DanOlsen 


▶ This video was produced by Coreography https://coreography.com. 

▶ Lean Product was founded by Dan Olsen https://dan-olsen.com. 

▶ Learn more about Lean Product Meetups at https://meetup.com/lean-product 

#DanOlsen #LeanProductMeetup #ProductManagement