#37: How AI is impacting Product Discovery & User Research | Teresa Torres (Product Discovery Coach)

If you're a product team and you're building for customers, if you're looking for a way to not have to talk to your customers, I have a concern right then and there. That's my philosophical opposition. Technology needs more humanity in the process, not less. Already I'm questioning, like, why are you trying to replace this really critical empathetic work with some automated technology like that? Just on a philosophical level really concerns me. And I get that not everybody is super extroverted or is comfortable talking to customers and like we might have to learn this skill. So I actually have empathy for talking to a customer is scary. I want to put that aside. If you don't have an innate curiosity and desire to learn about who you're building for, I almost want to question why you're building for anybody other than yourself. Hey there and welcome to another episode of Supra Insider. This time, Mark and I sat down with Teresa Torres. Teresa literally wrote the book about product discovery, specifically on how teams can do continuous customer discovery to integrate insights into their decision-making process. This was a fantastic conversation with Teresa, where we talked about how AI can enhance but really never replace humans in the customer discovery process, and how some teams are getting this completely wrong. We talked about the real challenges of doing customer research, and... a fascinating experiment that could help some PMs use AI to get better at their interviewing skills. And then we talked about Teresa's insights on building an audience, including LinkedIn, her sub stack, and just knowledge for people who are looking to just stay on top of the shifting social media landscape. It was very relevant and timely for Mark and I on all of these fronts. I hope you get a lot of value out of listening to this. And thanks for being here. Cool. Teresa, thank you so much for joining us. Thanks for having me. Yeah, we've been really excited about this. We had a previous conversation with Alex Hanthor on a previous episode, and he quoted you a couple different times. And Mark and I have been itching actually to have you on so that we could go deeper. I think one place I'd love to start the conversation is you are the expert when it comes to continuous discovery, something that I think Mark and I are big believers in, just like constantly talking to customers and incorporating insights into the product process. I'm curious to get your take, given everything happening in the AI space at the moment. Are you already starting to see AI, generative AI, just fundamentally alter the way that teams are doing continuous discovery? And do you have any kind of strong conviction of where things might be heading in the coming years? It's funny. I started to chuckle because I always have strong convictions, but I don't like making predictions. So I'm not sure. I have strong convictions about AI, but maybe not about where it's heading. But we can unpack that. I like to think about discovery as there's three fundamental buckets that I think are universal. They're not going to change, which is we have to understand the value we're creating for our business. We have to understand the value we're creating for our customers. And then we have to figure out what to build to address both of those things. And if you're familiar with my opportunity solution tree, those are the three buckets. Outcomes are business value, opportunities are customer value, and then solutions are of course how we deliver that. So if I think about... Generative AI, is it going to affect any of those three buckets? I actually think the one where it might even today could impact our work is that first one, which actually surprises a lot of people. People ask what in the world is the outcome? Where do my outcomes come from? I'm going to make them up from thin air. We teach a really simple structured way. It's a KPI tree. Like start with your revenue model formula that tells you your business outcomes, derive your product outcomes from that. I think it's such a simple and structured activity that I suspect if generative AI can't do it out of the box, it's like I should probably just create a custom GPT to share with the world that will do this for you. This is not the hard part of discovery. The other two, I'm not going to say generative AI is never going to help us. And I think there's even some narrow slices today where it can help us, but I'm seeing a lot of teams push this way too far. I don't love this idea of a synthetic user. I don't love this idea of chat GPT just make up personas for me. I don't love the idea of one click opportunity solution trees. Like we're missing the point when we're using generative AI to replace our discovery. Can you share a little bit more? Yeah. Like why, why do you think like we like generative user or like a synthetic user or maybe having chat GPT creating personas is a bad idea. Is it mainly because people are skipping this important step of talking to the users actually, or? Is it something else? I think there's a lot of pieces to this and here's, what's hard. We're on day zero of this technology. We have no idea where it's going. We have no idea what it's going to evolve to. So I'm just going to give it a caveat that like my comments today reflect today's technology. I know better than to try to predict what this is going to look like next year. Right. Yeah. For people listening is December of 2024. Yeah. Thank you. So I'm going to start with like my philosophical opposition, and then I'll get very tactical about the technology itself and why it concerns me. If you're a product team and you're building for customers, if you're looking for a way to not have to talk to your customers, I have a concern right then and there. That's my philosophical opposition. Technology needs more humanity in the process, not less. Already I'm questioning, like, why are you trying to replace this really critical empathetic work with some automated technology like that just on a philosophical level really concerns me and i get that not everybody is super extroverted or is comfortable talking to customers and like we might have to learn this skill so i actually have empathy for talking to a customer is scary i want to put that aside if you don't have an innate curiosity and desire to learn about who you're building for I almost want to question why you're building for anybody other than yourself. There's nothing wrong with building for yourself. That's a hobby, right? But if you're building a business and you're trying to find customers, like we have to maintain that curiosity and build that empathy. And I think there should be this innate desire to do that. And some of us don't have that innate desire. We think we don't because there's scary elements to it. So like. We might have to do some skill building. We might have to get out of our introverted shell a little bit. That's not what I'm talking about. That's normal. We can coach people through that. You can take courses on how to do that. It's this like intrinsic curiosity about I'm building a thing. I really want to know, is it working for you? Like we should not be using any technology, let alone generative AI to remove that from the process. I think that friction can scare some people. I'm not talking to as many people as I should be to feeling like I'm actually doing enough of that. It's I'm going to have to do things that feel uncomfortable. I'm going to have to start messaging people. I'm going to have to get on calls. I'm going to have to be use sloppy terms or I'm going to have to describe. I might do it wrong. Yeah, do it the wrong way. And what I tell people is it's a muscle. And the more you do it, the better at it, the more comfortable at it that you get. Like any other form of. It's not exactly sales, but there's an element of customer validation that's in common to both research and sales that I think is important. And I told Mark that I've reflected on some of my friends who tell me that they, they don't really trust their PM. And when I ask like, why is it that you don't trust your PM? I want to say 80% of the time, it's something to do with, they don't talk to customers enough. Like I don't trust that they understand their customers. I would get worried if someone can make people think they're talking to customers by using some shortcuts. It's just do the thing. It's the better thing to be doing. Okay, but here's the challenge. And this is where I have a lot of genuine empathy for this. The vast majority of businesses are not asking their product teams to talk to customers. This is a new requirement in the job, right? I also want to counterbalance this with like, if you're not doing this yet, if you don't know how to do this, I'm not going to go as far as what some other people have said. saying you're not really a product manager, you're not doing your job. That's taking it a little bit too far. We're evolving as an industry. We're getting better at this as an industry. And a lot of companies aren't expecting this other product teams. And I'm not going to say that if you work on one of those teams, you shouldn't think about doing this. I think everybody who builds a product needs to be getting closer to their customers, but there are hurdles and obstacles and skill building and practice required to get there. So I don't like these very black and white statements of like, you're not doing this. You're not doing your job. I do think that if you want to build four people, you need to have a really rich and deep understanding. And one of those people and interviewing is a great way to get there. And I don't think generative AI is going to change that. And that's where like, let's get into the tactical technology bits of this. Like what does generative AI give us? And why are some people thinking we can interview synthetic users? Let's suppose you want to start a startup and you want to help podcasters grow their audience. And you both have a podcast. We're on it right now. But let's say you didn't, you knew nothing about podcasts. You just listened to them and you liked them. And you really wanted to like, you see a lot of people start a podcast and give up and you think it's because they can't grow their audience. So you want to help them. Okay. You probably should talk to podcasters, but maybe before you do that, you could chat with ChatGPT about what do you know about podcasters? I'm actually not opposed to that. Use these tools to learn a baseline. But if you only do that, and this includes synthetic users, if you use the product that allows you to interview synthetic users, here's what you're getting. Generative AI makes predictions. based on what it knows today across the web and other sources. So that means if you're a new startup founder and you want to start a podcast tool, you're going to build your product based on what's already freely available on the internet, which means any other person that wants to start a similar startup is going to start from the exact same dataset. Like this is missing the point. Like the more we understand our customers, the deeper and richer understanding we have. It's like we're building a moat between us and our competitors. If I understand my customers better than my competitors, I'm going to better serve my customers. If I just rely on a synthetic user, anybody else can do that. And like some people push back and go, yeah, but I'm going to write a better prompt. Okay. That is not a moat. It's just not, it's not a competitive insight that's going to get you further along. And so I think like just the nature of generative AI and how it works. It's great for like, okay, help me get up to speed on this area because it does know about everything in the world basically, but it's not creating new knowledge. And that's, I think the real challenge. Like we interview customers to get that deeper understanding that nobody else has. And you're not going to get that from a large knowledge model. Yeah. No, I think I fully agree with you that we probably, unless technology changes radically, we probably would never want to. Or at least right now, delegate the talking to users, interviewing them. And yeah, I think my current mentor model for AI is there's like, they're useful for two things. One is like kind of delegating work of maybe helping you do that by quick research, or maybe taking a long podcast transcript and creating show notes. So that's one kind of just like delegating, doing work for you and doing it faster. The other use kit that's interesting is like the co-pilot of hubs you think with you. And actually, I wonder if there's something there. And I completely agree, like... You need to have some unique data to be helpful. But let's say you record all your interviews and then you put them into a data set. And then also that copilot maybe has also read your book, right? And maybe you can give people feedback on, hey, I thought your interview could have been better based on what I know about the book you maybe asked. too many leading questions, or you actually went from one question to the other too quickly, where actually there was a great opportunity to go deep and stay online. So I wonder if there's maybe something there to have taking all the data that you have from interviews and maybe helping you maybe identify opportunities or even just coach you on interviews that could be interesting too. Yeah. Okay. So I do think there are lots of uses of generative AI in discovery. So I started with don't let it replace humans, but I do think it can be really helpful. So let's talk about some of these areas. I see some teams take their interview transcript and say, what are the opportunities? I'm okay with that. If you're treating the LLM as another member of your team and you're still doing the work yourself. So one of the things I like to have product trios do is everybody synthesize the interview individually before you discuss as a team, what you heard from the interview. Because it forces everybody to do the work. You're all going to hear different things. You get more out of the interview. If you treat an LLM as a team member, so instead of three people doing it, now you have a fourth person to doing it and the fourth is an LLM, that's great because maybe the LLM is going to catch something you didn't. Here's the challenge with that. Maybe it's a hallucination. So we can't just outsource the work in this instance. This is your goal. This is like what's building your moat. You don't want to just rely on this machine. to tell you this is your takeaway. The other challenge with this, and this is over comable with prompts, but it's hard. ChatGPT in particular, Claude's a little better at this. But ChatGPT in particular does not get specific enough. And they'll write a prompt to get it to give you really specific opportunities from a customer story. Really hard. I don't know what it is about ChatGPT. It's really hard. I'm an expert on this, right? I'm trying to teach. ChatGPT, did you do this from an expert point of view? And it's still really hard for me to get good output. So do I think the average product team is going to write a prompt that's going to get ChatGPT to get better output? I don't think so. One thing we talked about in the last episode with Tal Raviv is around how to set up projects in Claude and to feed these projects all of the critical context about the product, the org chart. like the customer base, like pretty much anything that you could think of as being material to coming up with what the output should be if the output was really good, like what the inputs would be. And then, and I've already started playing with for my course, for example, but it's like the outputs are so good when you just use it that way. And I could not do the same in ChatGPT. So for me, I feel like I've just taken almost like a leap. into what it, Jenny, I can do for me in this sense by just switching products and feeding different content. I actually love this because I've done the same. Like in the last maybe two months, I have moved a lot of my LLM work to Claude because I do a lot of text-based like writing tone voice kind of stuff. And Claude seems way better than ChatGPT at that stuff. So, and there is another, there are some use cases that I really like. So The first one I will say is this treat it like a team member where every individual team member is doing the work and then you're collating together to create a team view. Another use case I really like is a little bit like what you were just saying, Ben, where I, okay, I don't like the use case of using ChatGPT to say, create an ideal customer profile for me. But if you have an ideal customer profile from your research. So you've started to get really specific about who your ideal customer profile is. You can give that to Claude or ChatGPT. You can give like, here's our products and services. I do this a ton. When I write a blog post, I give it to either ChatGPT or Claude. I give it my ideal customer profile. I give it the article and I say, what's missing? How can I make this better? Or I say, I want to rank for this SEO term. Go look at the top 10 results. How do I rank higher than them? given that I've written this article, here's my ideal customer profile. That kind of stuff is amazing. Or like I do this for my course landing pages. Like here's a new sales page. Here's my ideal customer profile. What have I not covered? What questions do they have? And that the questions that they have are coming from my ideal customer profile. So I'm not asking the LLM to tell me those questions. I'm saying based on this ideal customer profile, what did I miss in my landing page? So it's a sounding board, right? I'm giving it the data it needs. to then give me feedback on my work. That's a great use case. Yeah. It's super interesting. The more we talk about it, I'm almost feeling like even with AI is more important than ever that do research and interviews, right? Because then it's like the stronger and the better your data is and the better you're at interviewing, then the better everything gets, right? Because then these prompts have better, yeah, they're just working with, and that has ripple effects. So there's a... There's another use case I'm working on. I'm not ready to launch this yet, but it's very promising and I'm super excited about this. Right now I'm working on it as a custom GPT. If I can figure out how to make, get this to work with Cloud projects, I probably will switch to Cloud. But I teach in my continuous interviewing course, a story-based interviewing format. So tell me about the last time you used a product. Tell me about the last time you experienced a need. The challenge with story-based interviewing is that we're not very good at it. It takes practice. It's a skill that needs to be developed. And that's what our interviewing course is all about. It's about getting hands-on practice time with feedback. One of my goals is to shift a lot of our course content from live courses. We're still going to offer our live courses. So we want to also offer them as self-paced courses for people in time zones where our offerings don't work. With interviewing, you need practice. So how are you going to get practice if it's self-paced? So one thing I've been playing with is I created these really rich, and this is the important part. If you don't do this, it doesn't work well. I've created these really rich case studies. So let's say I have five users that I've created rich content around. They're actually based on real people. And then I tell ChatGPT to be the interview participant. And there's two phases to this. The first is my student is conducting the interview with the ChatGPT. And the ChatGPT is answering the questions they were asked based on the case study content. So you're getting practice reps and then the chat GPT is giving you feedback on your interview questions and whether you stayed grounded in a specific instance. What's hard is I'm trying to simulate humans. If you ask them to tell you about a specific instance, we'll often flip-flop between that instance and generalizations. And the art of interviewing is keeping them in the specifics. And so I'm trying to train this GPT to simulate that. Like you don't want it to always be perfect, but it needs to know the rules so it can give feedback. And it's looking like I probably have to separate this into two different tools because if I tell the GPT the rules, then it tries to be really helpful as an interview participant. And I don't want it to be really helpful as an interview participant. But like I got this idea from Ethan Mollick. So Ethan Mollick, I think he's at Penn State. He's studying, he studies technology. as his like academic profession and he's totally nerded out on LLMs and how to use LLMs in teaching. And I think he released this like custom GPT to help people practice negotiating. And it blew my mind. And I was like, whoa, if an LLM can do this, then it can teach help with teaching you to interview. But just like I said, I'm creating these like people and like creating this rich case study context behind it. I got that idea from Ethan. Like you can't just. If I just tell ChatGPT, like, you're an interview participant. I did this for a blog post to show that this does not work. Like, synthetic users don't work. Is, I asked for three stories. The structure of the stories were exactly the same. And really superficial details were different. Right? So I'm not saying, you can just go chat with ChatGPT right now and practice interviewing. But I do think there is the ability to provide it with enough rich data around a detailed case study. that like I would create that would then allow my students to practice interviewing. And I'm looking at that across all of our courses. So not just for interviewing, but like feedback on how you're framing opportunities and are you getting specific enough? And so starting with a rich interview transcript and that stuff is all really exciting. Like I think that thinking about an LLM as a coach is a real powerful model if it's backed by all this like case study content. Yeah, I get excited about treating it like a coach that you could feed all the context to because I love the example you shared where theoretically, if you beefed up the profiles of these fake users that are based on real people to also include things like where are they on the introverted versus extroverted scale or how comfortable are they expressing themselves or how many? There's a bunch of things that I found when I interview people. It's like same exact script with one person. I get done. 10 minutes early on a 30 minute call and another person I can cover only half the questions in 30 minutes. Cause it's like the amount of things that they say and like the way that they respond changes things. So I wonder if in the future we'll be seeing more like metadata on people in the synthetic data that kind of allows people to get the right reps. And this is the thing I really want to clarify. Cause I have, I'm really concerned people are going to hear this and be like, Teresa, you're defining synthetic, you're describing synthetic users. purpose of this is for you to practice a skill. Nobody is making product decisions and doing discovery based on this case study content. Totally agree. It's just about story-based interviewing reps. Like, how do I build my skill? So I just want to highlight that because I realized as I was describing it, I was like, oh, she's describing synthetic users. Yeah, no, I think the big delta between the people that maybe say that you should use synthetic users and not that I'm agreeing with you on is you can go out and talk to real people. and get good at talking to them and making, getting, improving your skills and extracting the right kind of insights from these conversations, obviously record these things. So you have transcripts for them. And then if you could feed those transcripts and do whatever system you're using to try to get that fourth team member to your point, then that fourth team member will be equipped with the right data points to be able to help you find the right insights. But, and it's gonna cover you in ways that the other humans might miss, but the, but it might catch. Yeah. Yeah. Yeah, this use case is really cool. And I think it's really gonna probably transform the way like professionals get trained in the job. Like you could use the same thing for like sales trailing and like doing like role playing or even if I'm a product leader and I have a big team of PMs, they're like, oh, like why wouldn't I just train all my PMs and make before they do user interviews, maybe have them practice like eight calls or nine calls with this GPT. Like it just seems like, I don't know, it's like a low friction, low risk way to get better at something, which it seems like a really exciting future. Yeah, there's, okay, so there's a few other use cases I'll highlight too. So I really like LLMs for summarizing large data sets that like when humans look at them, we're just not that good at it. So let me give an example. We run an annual, it's turning out to be biannual benchmark survey where we ask people about their discovery habits. And some of the questions, most of the questions are structured, but some of them are verbatims. So then we get like the first time we ran the survey, we had 2000 responses. And so one of the questions, I think I had 800 verbatims and I tried to manually group them and it's so tedious and horrible. Whereas if I just tell chat GPT, give me group these for me. It's actually not that good at that. It'll like group them and it's messy and there's overlap between the groups. But the first draft is good enough that I can start to see what the groups might be. So then I give it a list. I go, here's the groups. Now group these under those groups and let me know which ones you couldn't group under a group. Okay, now we have Rev2. It's a little bit better. And for all the ones that can't group under a group, I do it again. I say group just these and it gives me some more groups. So I'm iteratively building my buckets. And then once I have a good list of buckets, I can start over. I can say, here's all the raw data. Here's my buckets. Sort it for me. And it's great at that. Like it's not, it takes some human feedback to identify the buckets, but once you have a good list of buckets, it's really great at sorting things. I see maybe 5% error rate. So I love it for that use case. And I do this for, I do a ton of webinars. We have a Q and A section in the webinar. We never get to all the questions. I have a database of over a thousand questions that I've gotten from people. So what do I do with those questions? This exact process, group them. What's my next blog post going to be about? what's my next short video going to be about? It's all coming from this like unstructured data that ChatGPT now has helped me structure and make a lot more actionable. That's not replacing my interviews. I still interview, right? But it's like allowing me to turn this big messy data set into something actionable. I'm hoping we'll get to the point where our research repositories do the same thing. You start to spec out an assumption test you want to run and it tells you Hey, seven teams at your company have already run assumption tests like this one. And you can decide are those tests similar enough and you can use their data or do you need to run your own test? Because there is this legitimate challenge of what do we already know? And maybe it can help with that. But it's, I'm not convinced it's going to just work out of the box. I feel like there's going to be this like human feedback iteration to get it to work really well. For the use case that you were describing on the benchmark. A question that I have there, we actually are just run like a similar benchmarks for the community. And one of the questions that we had was like, Hey, what are some interesting like workflows that you're using AI for and are they helpful? And I feel like every now and then like I read the answers be like, Ooh, like that one's a really interesting one that I feel like not enough people do. And that it, and I wonder have you found like any of the LLMs to be good at spotting those like very interesting outliers that might be worth highlighting and maybe, yeah. Yeah. That's a really good question. I don't know that I have an example of that yet. I, this use case of grouping and sorting things, I use a lot with LLMs. Like I export my LinkedIn data every month and I'm looking for what's resonating with people. And I do it based on type of content and also the topic of the content. And always the outliers are like unique one-off things that it's not really driving insights yet. Everything does like this level good, like medium good. And then I realized I have to do audio only like medium good. And then the thing that's like high good is like, I got angry one day and ranted on LinkedIn. It's like not something I could duplicate. Right. And I was hoping it would be good at that, like outlier, like, Hey, here's a key insight. Maybe it's cause I don't run enough experiments. There's not enough diversity to find those yet. And it is inspiring me to run more experiments and to try a variety of different things. But it's not, I'm not at that point yet. And then same with like with our survey, one of the questions we asked was if they said they worked in a product trio, we asked how often does everybody in your product trio have an equal say on the decision that you're making? And the questions were like all the time, most of the time, rarely and never. And so we took the. never, the rarely and the never responses. And we asked why, like, why didn't everybody have an equal say? And we got a whole bunch of like open-ended verbatims. And I think just a couple of months ago, we published a blog post about the 12 biggest challenges to product trio collaboration. And that content literally came from those verbatims. And we ran the survey two years ago. I wrote that blog post two months ago because two years ago, I tried to make sense of the data and it was too hard. And then two months ago, I had this insight of, oh, I've been using LLMs to categorize things. Let's throw it at this problem. And it did a really good job, but I can't say there was like one crazy one that was this big insight. It's just like things slotted into what you thought they would slot into. So it sounds like you run a bunch of experiments with your content and things like that. And it sounds, I'm curious, do you, how do you track like? How, how structured are you about tracking the results of the experiments? How, how do you store them? Like, just from selfish reasons too, like, I feel like I run a lot of experiments, but I'm not good at capturing the learnings or in a way that then I can do what you just described to be like, Hey, have I run similar tests on what have I learned? Yeah. For me, it depends on how important the test is to like, probably my customer experience versus like just trying to improve my business. So let me talk about the difference there. So first of all. I'm a one person company and my company does reasonably well. Like I don't, I'm not like hyper-focused on growth. Like for me, growth is impact, is growing impact more than it's, I'm less concerned about can I optimize every little tiny step of my business? So when I run a lot of product experiments for my like business experiments, sometimes they're half-assed because like, I don't need them to be rigorous, right? If I'm experimenting in a course and it's experience, it's affecting my student experience. Now I'm going to be really rigorous because I don't want to negate. I don't want to do any harm. So I think about it as like I have product experiments and then I have business experiments and I can give an example of both. Like in our courses, I listened to a podcast. It probably was a Lenny podcast episode where he interviewed one of the founders at Duolingo. And the guy talked about how they really think about learning in these like small bite-sized chunks. And I had already been playing with this idea of like, I was trying to think about how to turn our courses, like offer self-paced versions of our courses. And to me, practice is really important. Like you can't just take an information only product. You don't learn anything from that. So I had this idea in my head already about like problem sets. What's the equivalent of a problem set in a self-paced discovery course? And then I heard this episode about Duolingo and like the way he talked about it, it was clear that we both thought about learning in really similar ways. And it just unlocked this idea of, oh, I can create assessments that are like problem sets that are like these little mini tiny case studies. And I'm like, okay, I'm going to run this experiment. And I added a couple of assessments to our opportunity mapping course so that people could test their experience on like how to frame an opportunity or what's a parent versus a child when you're opportunity mapping. And so I wanted to run that experiment. And so I actually held. added the assessments to some cohorts and left it out of other cohorts and did a true like AB style controlled test. And I followed everything you read in my book. I predicted my success criteria up front. I tracked this in a mirror board. I had a target opportunity. I had different potential solutions. Like I probably followed the discovery habits to the teeth because this is something that affects my student experience. I don't want to break it. Okay. Now let's talk about a LinkedIn experiment. Now exporting on my LinkedIn data, I'm trying to figure out social media has changed. Now LinkedIn is a walled garden. They don't really want to send traffic to my blog posts, which is how I used to use it. So now I'm like, okay, what's the value of LinkedIn for me if I'm not getting blog post traffic? Like, oh, people like free things like free webinars and free mini email courses. I'm going to experiment with those things. For those experiments, I'm not like documenting what I think my success criteria is up front because one, I don't care that much. If I get nothing out of LinkedIn, like it's okay. It's more of like scratching a curiosity itch. And I tell teams, you don't have to do all the habits at the same level of rigor for everything that you build. It should match your level of risk. So like for me, I don't want to risk my student experience. I'm okay with risking. If I upset a few people on LinkedIn, it's not really going to negatively harm my business. So. It depends on what the purpose of the test is and who it's impacting. Yeah, I think that makes a ton of sense. And I'm actually just touching on the point with your course and how people want practice. I've recently made some really big adjustments to my course and Tal who runs another course on Maven, I think is running the same experiment where our thesis is giving away as much free content as possible is going to. increase people's desire to get reps and to get practice. And then we could focus our efforts instead of repeating ourselves over and over again in our respective courses on like giving lectures on actually developing better and more scalable ways of allowing people to get practice. And so I fully agree with your premise there. What is your current view on LinkedIn as a platform? Do you like publishing there? Are you moving away from it? Is it just a curiosity or is it something more than that for you at the moment? Yeah, I'm going to start with, I literally, I think two weeks ago posted my most successful LinkedIn post of all time. Here's what it said. Today, as you scroll through your LinkedIn post, you're going to read a million things about all the ways you're doing your job wrong. I just want you to know that you're not doing your wrong. If your boss is happy with you, you're doing your job and that's enough. That's it. That was the post. And it really resonated because LinkedIn has become this, I almost wanted to call it a cesspool. That might be too extreme. Okay. I'm probably technically an influencer. I wrote a book, I blog, I share stuff on social media. I'm going to be the last person to tell people they shouldn't do that stuff. It's actually had a huge impact on my life, but I hope I've never been the person that like writes about things I don't have experience with. I don't ever steal anybody else's content. There's this, it's a hard topic to talk about because I don't want to disparage anybody, but like I scroll through my LinkedIn feed and It's all black and white. You're doing this wrong. This is the only way to do things. And then I look at the person posting it and they're like six years into their career and they've worked at one place. And I'm like, where is this coming from? And it's really toxic. Like I don't, we've got to stop telling people they're doing things wrong. I think we can share what works for us. I think there's some nuance to it. It's gotten to the point where like, I don't really want to share on social media anymore because I don't want to contribute to the noise. I hope I share things that are a little bit of a different flavor and like a different tone. But it has really forced me to question like, what is the purpose of social media and what do I want my voice to be on social media? And it's partly why I'm running more experiments. The other challenge I have with LinkedIn right now is like probably 75% of the comments I get are clearly AI generated and add no value. And I really like, I'm not just here to broadcast. Like I want to engage with people and talk with them. And it's just such a waste of time to reply to AI generated. talk comments because nobody replies. Like they don't, we're not actually commenting to engage in conversation. So I have mixed feelings like Twitter jumped the shark is the best way I'm going to put it. And so when that happened, I started to invest more in LinkedIn because it seemed that's where LinkedIn was going, where people were going. I've been dabbling on the blue sky feels a little bit more like Twitter early days, but how long will that last? Like Mastodon felt that way for a little tiny while. So I don't know. I have a love hate relationship with it. social media. Like I love connecting with people who are genuinely curious and have genuine questions and are trying to work this way. And I hate the like self-promotion crap. And I think Jonah Revea is making it worse. Sorry. That was my LinkedIn rant. No, I, I, I think Mark, Mark and I talk about this stuff a lot. We both made pretty concerted efforts this year. Mark has been a lot more disciplined and consistent than myself, but we both made an effort at the beginning of the year to grow our LinkedIn followings. for our respective businesses that we're trying to grow. And it is a good top of funnel, I think, for Mark to find new members for the community and for me to find more people that take my courses and work with me. But at the same time, I feel like something changed in the last couple of months where we're both trying to figure out what's going on with like behind the scenes of it all. And I think as a result, I feel like the quality of content in my feed seems like there's like a higher noise to signal ratio. And I think that And I know people like, I think Jason Knight's been pretty open about this too. I think, I don't know if you know him, but he's got 40,000 followers and he can post and gets like less than a thousand impressions on something that he thinks should be pretty good. And it's like, why are you not showing my stuff to like my audience? Yeah. So. It's, turns out every algorithm base feed goes through this, right? Like we saw the same thing happen on Facebook. We saw the same thing happen on Twitter. One thing that I've been experimenting with. So let's, first of all, I'll just talk about my social media strategy, like for a long, long time. I shared four or five things a day. And the reason for that volume is because my primary platform was Twitter. And on Twitter, it's perfectly, it's a good strategy to post four or five times a day. At least it used to be. So when I moved to LinkedIn, it was really in response to what was happening on Twitter. I didn't put any thought into it. I just started posting four or five times a day on LinkedIn. LinkedIn does not actually want you to post four or five times a day. Usually it like picks one of my posts that is the most successful. And that's what people see and they don't see all the other stuff. But I've ran experiments. Somebody did a like, here's how much people post on LinkedIn. And they like looked at all the top product people and they like called me out. They're like, Teresa posts 160 times a week. And the next person posts like five times a week. And so I was like, oh, maybe I post too much. Like I never really thought about this. And so I started running experiments. It turns out no matter what I do, if I post once a day, if I post twice a day, if I post five times a day. More always gets more engagement. Even though LinkedIn only picks one that really gets a lot of, like universally, more always gets more engagement. So what do I post? Like originally I posted a worthy read. So an article that I did not write. Somebody else, I found it on the internet. I have a database now of 600 worthy reads. I share one a day. Then I share. one article from the product talk, either a new article from product talk, if it's a day we publish or it's an archive blog post. It's one a day. And then our third post was like a concept from the book. Our fourth post was something about upcoming class. And then sometimes we would post a news, like join our newsletter. I have recently dropped all of them except for the worthy read and the article a day, because I'm noticing that like LinkedIn is favoring one or the other. And I've also noticed that like when we promote our courses on LinkedIn, it just doesn't work. LinkedIn doesn't want you clicking out. And even like our articles that we share, most people don't click on them. We're instead of, I used to just post the title and a link. Now I post like a very detailed AI summary of the articles. It's another one of my AI use cases. And Claude is way better than ChatGPT because Claude will write the summary using the language from the article. Whereas ChatGPT will write it as if it was ChatGPT. And so what I'm trying to do is offer value, whether you click on the link or not. And then hopefully the summary is like enough that like it offered value. Lenny does this really well with this podcast episodes. You can just read his summaries and you know what the podcast is about. And maybe it motivates you to listen to that episode. So I'm experimenting with that and that actually works way better. And then instead of promoting our courses, I promote free stuff. Here's free webinars. Guess what happens in the webinar? I promote a course or like here's free email mini courses. Guess what happens in the email mini courses? I promote our paid courses. Like I'm just constantly running these experiments because like Google, the percentage of searches on Google that don't lead to a click outside of Google is now like 35%. It's impossible to get traffic from anywhere. Right? It's just, it doesn't exist. Like the internet's kind of broken right now. So I feel like the only way around this is just to do a ton of experiments. So that's what I try to do. Yeah. I know you don't like predictions. I mean, I shouldn't ask this question, but like, how do you think, how do you think people are going to get like distribution in the future? Right? Because I completely agree. Like I'm still like people using LLMs to ask questions. It's like perplexity. All these platforms are like, don't take, don't take my users outside of the platform. Like we're going to penalize you if you put links. So like, how do you think are people going to get people to go to the places they want to go? I think it's the same boring answer it's always been. You have to own your mailing list and have a direct relationship with your customer. I think it's that simple. And this is something like when Twitter exploded. I know a lot of people think Twitter is still fine, but I think it sucks. When Twitter exploded, I started a daily sub stack. So I have a sub stack at producttalkdaily.substack.com. It's real simple. I just took all my social media content, those four or five things that I just described. and I put them in a daily email. I don't have a huge following, but for anybody that like was on a social media network and liked my content and they don't see it anymore because the algorithm doesn't show it to them, it gives them a way to get that content. And the more all these networks are trying to be walled gardens and not send me traffic, the more I think about how do I make this daily email so valuable that people want to sign up for it. And I'm in like, I'm on day zero of that experiment. I'll tell you, I have way more people on my main mailing list as I do on my sub stack. But I think the sub stack, not because of sub stack, but because it's this daily email, if I can turn that into a really high value daily email and get in people's inbox every day, I don't need it to be as big as my other mailing list. That's a really high value audience that I have a really strong connection with. And I'm just scratching the surface on this stuff. Like I don't have the answers, but I do think. It's always been true that you're better off having a direct relationship with your customers. This is why I don't write on Medium. It's why I'm not like, at least Substack lets you export your list like you do own your list on Substack. It's why I don't write articles on LinkedIn. I want to own my content. It's why Mastodon was so appealing to me. My Mastodon account is on my domain. It sounds like that's how blue sky is going to verify. So I'll eventually set that up. I am a firm believer in... I should own my content. It's why I published my book myself. It's why I eventually will do my translations myself, even though it's a ton of work. But I think we've seen over and over again that we can't trust tech companies to own our content, which is unfortunate. It is pretty wild how I'm looking at someone like Lenny and what he's been able to grow as like an owned platform where he's, I'm sure, initially relied on some of the platforms to drive him traffic. I feel like the platforms almost care about Lenny's content coming to them more than Lenny cares about those platforms featuring his content now. It feels like something's shifted when you can get that big. And I don't know if you're trying to get that big, but the idea that I can sign up to your sub stack and I can reliably get what you are writing without hoping that I happen to be on my feet at the right moment where the algorithm chooses to show it to me feels kind of like something's broken. Like the reason I follow you on LinkedIn is because I want to see your stuff. Why else would I follow you? So anyway, I know we're coming up on time. Mark, did you have any other questions for Teresa before we start wrapping? I think we, yeah, I think we can wrap. I think I have, I mean, a million. We'll have to do part two. Yeah. I wish you had another hour, but also want to be respectful of Teresa's time. Yeah. So I guess our two final closing questions here, and I think you mentioned one of them already, Teresa, but how can people learn more about what you're up to? We've mentioned a few different links that we can include in the show notes. And then the second is how can people be helpful to you if at all? Yeah. Okay. So a couple of things. My primary domain is producttalk.org. That's where I blog. It's where you're going to learn about our courses. So we do a wide variety of courses, all on helping you build your discovery habits. People haven't heard of the book or picked up the book. It's continuous discovery habits. There's a link to it right at producttalk.org. I do have a product talk daily sub stack. So if you want that daily email, it's at producttalkdaily.substack.com. That is different from my main mailing list. So if you're on my main mailing list, you're not necessarily getting those emails. That's because one was an experiment. We'll see where it lands. Yeah, that's where people can learn more. And then you asked a second question, but I already forgot it. How can people be helpful other than maybe checking out those links? Is there anything else they can do to be helpful? Yeah, we talked a lot about generative AI. And I would say as you're experimenting, especially related to discovery. I would love to hear about your experiments. I am everywhere on social media. I'm still on Twitter, despite it being a hellscape. I am now on Blue Sky. I'm on LinkedIn. I'm still on Mastodon. I'm literally everywhere, either as T-Taurus or as Product Talk or as Product Talk videos, depending on the platform. And then we talked a lot about social media. So if you have feedback, like if you follow me on any of those channels and there's something I'm doing that's resonating or not, like this is... This was a huge theme for me in 2024 and it will continue to be a huge theme for me in 2025. It's just how do we get more people to even know about this way of working? And so that's like what social media is for me. Like how do I reach more people? How do we get more people excited about discovery? And the last thing I'll share is I started a year ago recording daily short videos. They started out as less than a minute because I was trying to be on YouTube shorts and like reels and whatnot. And then I gave up on that. And now they're around three minutes. And my purpose with these videos was here's the problem I'm trying to solve. People say, Teresa, I drank the Kool-Aid, but I have to work with my product trio and they don't know what a story map is. In the moment that I'm trying to introduce story mapping, they're not going to stop and read an article. I now have like a one minute video that explains the concepts of story maps. Or like you're mapping, you're creating your first opportunity solution tree and someone's like, I want to add this opportunity that I just made up. I have a video about why you shouldn't make up opportunities. The purpose of the short videos is let's spread the love for continuous discovery around the planet. And they're free, like this doesn't have to be for Teresa's benefit. It's for all of our collective benefit. And those are at videos.ponenttalk.org. So one way you can help me is to spread the love for continuous discovery. Amazing. We'll add all those links. And thank you so much for joining us. This has been so much fun. Awesome. Yeah, thank you. If you enjoy this conversation, please share it with someone who you think would benefit from it as well. We really appreciate it. We'd also love a follow or a rating on Substack, Spotify, or YouTube. That's going to let other people find us. And if you have any topic recommendations for a future episode, please send myself or Mark a DM on LinkedIn. We'd love to hear from you. Thanks.

Welcome to another episode of Supra Insider. This time, Marc and Ben sat down with Teresa Torres, author of Continuous Discovery Habits, to explore the role of AI in modern product discovery. Teresa shared why customer empathy remains irreplaceable, the common pitfalls teams face when misusing AI in research, and practical strategies for integrating AI effectively—without losing sight of the human touch.

She also offered her insights on how AI can empower product teams to build skills, enhance customer interviews, and streamline discovery workflows, all while staying firmly rooted in authentic customer understanding. If you're passionate about truly knowing your customers and using AI as a tool—not a crutch—this episode is brimming with valuable takeaways!

In this episode, we covered the following topics:

Why AI can enhance but never replace human-driven customer research

The pitfalls of relying on synthetic users and autogenerated personas

How to use AI as a “team member” to synthesize interviews and uncover insights

Importance to building curiosity and empathy into product teams

Practical use cases for AI in discovery, including feedback analysis and research repositories

Exciting experiments Teresa is running with AI to help PMs hone their interviewing skills

The importance of owning your audience and the challenges of modern social media platforms

And more!

---
Links:

Teresa Torres: https://www.linkedin.com/in/teresatorres/

Teresa Torres X: https://x.com/ttorres

Product Talk: https://www.producttalk.org/

Product Talk Daily Substack: https://producttalkdaily.substack.com/

Product Talk Videos: https://videos.producttalk.org/

Ben: https://www.linkedin.com/in/benerez/

Marc: https://www.linkedin.com/in/marcbaselga/


Interviewing at Stripe, Figma, Uber, or Doordash? This episode is brought to you by Insider Loops — company-specific interview playbooks built from real interviewer insights. Learn more at https://www.insiderloops.com/