AI Prototyping - All Things Product with Teresa & Petra

Hi, folks. This is All Things Product with Petra Wille. And Teresa Torts. And we're so happy you're here. Hi, Teresa. Petra. There's so much mushrooming in AI prototyping tools. So that's why I brought this topic. How do you think AI prototyping fits with the discovery habits? Should people use it? How should they use it? What are good use cases? I love this section. I've been playing with these tools a lot lately. I'm really blown away by them. I built an app in a very short period of time and a pretty complex app. Vibe coding. Yeah, but I'm also really frustrated by them. And I can see the potential for how they can really help with assumption testing. So let me get into this a little bit. So the first thing I want to say. like with everything with AI. One of the questions I got was, now that it costs nothing to build something, do we still have to do discovery? So I want to answer that first before we get into how to use these tools. Yes, please. First of all, it does not cost nothing to build something. I've been playing with these tools. It is still engineering. When you program, it used to be we wrote archaic syntax to tell the computer what to do. Now we write plain English to tell an LLM what to do. But if you don't use the exact same skills you learned to write the archaic syntax to program when you're telling the LLM what to do, you get garbage back. Right? So like, there's still engineering involved. So the user interface changed? Yeah, the way we interact with the tool has changed. underlying skills of like what to build, how to structure what you're building, how you build it, your database model. Like if you leave all that to the LLM without giving some guidance, it's going to be a tough road. So I think like the cost of build has not gone to zero. And I actually don't think when we're talking about production quality software, the cost of build is ever going to go to zero. Plus, may I add, I find it massively unfair that all these companies say like the cost to build something is zero when all the employees spend hours and hours of their weekends and evenings to try all these tools and to get good at prompting. Yeah, exactly. Just adding that. Okay, so I'm going to start with, I don't think we're ever going to get a cost zero to build. But even if we did, you still have to do discovery because if you push a button over and over and over and over again, and every time you push the button, you build the wrong thing, you're still wasting your life. So we're not pushing the button over and over again to build the wrong thing. We're going to still do some discovery. Here's where I love these tools. So I'll share my personal experience briefly, and then I'm going to talk about this in the context of assumption testing. I wanted to build an app. that allowed me to generate my own chat interfaces, so like a chat GPT, based on my own custom data and my own custom prompting, and make that chat interface available to my course students. Ask Teresa? You mean like the end user being as Teresa? No, not a Teresa bot, although I'm about to announce that too. Not a Teresa bot. Like, I have an idea for a tool in one of my courses where you submit your interview transcript and the tool grades you on how well you collected a story. Oh, I like it. That's a tool that I want to build. Well, I can do that as a Claude project, but I can't share my Claude project with all my students. I can do that as a custom GPT, but it turns out ChatGPT sucks at this use case. And even if I did it as a custom GPT, like people need an OpenAI account and I don't want my students to need an OpenAI account. So I decided like I need to use the Anthropic API because Claude's pretty good at this. And I don't want to just build this one tool because I can think of 40 tools like this I want to integrate in my courses. So I wanted to build a tool that allowed me to configure a chat interface just in a GUI. upload the relevant files and define the prompt for that chat interface and then deploy it in my course platform. Yeah. Okay. So this is a pretty robust tool. Like it involves a database with storing chat interfaces. I'm keeping a log of all the conversations my students are having with it so I can run it against evals. Like it's not just like, here's a landing page. I went from never using an AI prototyping tool to a fully working product that I have tested robustly and I'm ready to deploy to real students in about seven hours. That's impressive. With your given coding skills, engineering skills. Yeah, okay. So the things that I will qualify with this. You know how to build stuff. Here's how I will qualify with this. If I didn't know how to code, I would not have gotten to the end point. There were many, many times where the tool got stuck debugging. and I had to use my own debugging brain to be like, what's going on here? Let me help guide the agent to do the right thing. If I didn't know how to code, I probably would not have been successful in this complex of an app. I had to walk the AI through how to make sure my database is secure. You know what I mean? Yeah, I know. It was not as simple as like, an app that does a B and C or take this PRD and just build it there was a lot more engineering back and forth involved yeah but yeah maybe maybe I can add to that because I had a recent trial and error kind of experience with AI prototyping as well because you know and I don't know if my readers if the listeners know I'm working currently working on my product leadership wheel which is a self-assessment tool for product leaders to assess their leadership skills And I created it. It's currently a PDF. It comes with 12 leadership responsibilities and more than 400 detailed questions on their capabilities and skills and org maturity. And I thought about maybe it's a good idea to create a tool where they can assess themselves online. So the tool is asking the question, then they can give themselves the rating as they would in the PDF. So it's basically the tool version of the PDF. Yep. And. I created it. I can actually even share a link because the questions in there are all bollocks but it's just like to to get into the mode where I can really imagine the tool how it would actually work and imagine the user experience and touch it and all these kind of things. And with the product management wheel, which is a product management self-assessment, I had a wheel structure where at the end you had results drawn on a wheel-like structure where you can see like, okay, this is maybe my weak spot. And we did the same with the product leadership wheel and then figured it does not make sense because the wheel slightly works different from the product management wheel. the performance scale is a different scale than the one I used for the product management wheel and because of my prototype I could see oh this doesn't make sense at all and for the time being I now ditch developing this assessment tool I may be reviving it at some point in time and work on it again but really the prototype helped me to understand oh this new performance scale is not working with this type of visualization that I was actually planning for it to to be the end result of the wheel and so currently the wheel is not much a wheel it is when you do your time assessment so how much time do you spend on each of the 12 leadership responsibilities but the performance scale that i encourage people to use to rate themselves on each um responsibility not helpful as a visualization so that's quite cool and i only learned it because of my vibe coding ai prototype yeah okay so This is great because it's actually really similar to my use case. I knew I could foresee across all of my courses, I had this use case. Student submits work, gets real-time feedback from an LLM based on my proprietary rubrics and instructional content. I wanted to build something one time that allowed me to create an infinite number of customized versions of those things. And like when I started to approach this problem, I was like, okay, well, I don't want to like hand code a GUI. I haven't done front end development in 20 years. I've never used React. Like I don't want to learn any of that stuff. I don't know where I would host it. I don't know where I would do any of that stuff. I don't know how to make it secure. Like, I mean, I kind of know how to make it secure, but not 100%. And so I was just like, I'm going to see what I can get out of this tool. And I'll tell you, it took me seven or eight hours, but most of that was me learning the tool. Also, it cost me $25. Yeah, and that's wild. That's wild. I have a fully functioning app that I have already integrated my first chat interface into one of my courses, and it's being used, and it's working. And I can now replicate that as many times as I need to with as many custom sort of backend documents and prompting and whatever as I need to, and it works. And I didn't have to figure out where to host it. I didn't have to figure out. Because it's customer facing, I had to like get help on the security pieces and like make sure it's really production quality. But that's pretty darn good. $25, eight hours of my time. And seven, yeah. And most of that time was me learning the tool. Okay, so here's what went through my head the entire time I was using it. I'm going to use this for assumption testing. Because what do we do in assumption testing? I like it. We don't need production quality apps. No, we don't. Yeah, no, we don't. We don't have to worry about most security issues because we're testing with a very small number of people in a moderated controlled environment. We need some basic security things, but not like we don't need crazy denial of service attack type stuff, right? We don't need a production quality design for most of our assumption tests. True. Maybe for usability testing we might, but like for testing desirability, for testing feasibility, for testing, like there's a lot of things that we test. We don't need production quality designs. These tools just removed so much of the work in getting an assumption test life. And it used to be we would show a mock-up like, here's this mock-up, talk, walk me through what you would do. No. My bounce sandwich screens. Now, yeah. Now I can create an interactive prototype that they can click through and do things. And like you can decide, do you want to store what they're doing in a database and be able to look at all of their actions? Or do you not care? You just want to see it on the video and you're going to go back and review it and you're just looking at how they're interacting with the interface. Both are equally simple. Equally simple. And like most of the problems that I ran into when I was building my little app was the Agent struggled to really understand all the complexities that Anthropic API. So like if you're not using any APIs, you're not going to have any of those problems. And if there's problems, yeah. Right? And like, and then the second thing it ran into trouble with is like, I have this chat component GUI that it really struggled with like how to format messages and like, again, you aren't building production quality software. You can have weird idiosyncrasies. You don't, it doesn't have to be perfect. So like I could have built like an assumption test prototype in 20 minutes to two hours. Yeah, and let's face it, in half a year from now, all these tools will be so much better anyways. So maybe they will not even have all the issues that you currently run into because for how long these tools are available, this wipe coding stuff is not around for the last two years. It is for some people in some pockets, but to the broader audience and all product teams, that's barely new. So I think we will see a lot of improvement happening there. Here's the world I can imagine. A product team has an idea. They story map it. They start to identify their assumptions. They look at one moment in the story map where there's risky assumptions. They build it. They use one of these tools to build that one component of the app. They put it in front of customers. They look at their next riskiest assumption. They build that component. By the time they're done assumption testing, they have a full interactive prototype that they hand to their engineers. No more product requirements documents. No more user stories. here's your full blown interactive prototype of what I need you to build on our production quality environment. And by the way, you were involved in all those assumption tests. So you already know you're building this. Yeah, okay. So now we're talking because I was like, I think still famous Jeff Patton drawing again. with the three people having different pictures in their head about one thing. So I still think we need to human to human interaction for alignment purposes. And as you were saying, the whole team was involved in all the vibe coding stuff. So that's why they know why they're building things and for what outcome they're building stuff and for what outcome they're hoping. So I think that is still going to get more value from these five coding tools. Yeah. If you're, if you use it as a team. if your engineer in your product trio is actually building the prototypes in the Vibe Coding Trio. Because you do need that engineering mindset. And of course, I want the engineers involved in the discovery. That's a given. Yes. But you need to repeat this for our audience, Teresa, every once in a while. Your engineers need to be included in the discovery. That's a given. But my point is, when you're kicking off the sprint to do the production work, You have an interactive prototype of exactly what they're building that's been assumption tested. You know it's going to work for your customers. Like, that's an amazing move. And how often it is. And how often did we run back in the old days when we still needed Azure? No, how was it called? Azure? I don't know the tool that we all used to build prototypes ages ago and usually we had this team meeting for an hour or two discussing stuff with the engineers back and forth how do we build the prototype yada yada that and then an interaction designer went back to their desk and we're working on building this prototype for the next week or so and then we could look at it with the entire team again and only then is when we realized it was not even technically feasible what we actually had in mind building then we had to refine the whole thing and another week went by with the interaction the poor interaction designer really did all the prototyping And all that time is so much shortened and that will be so much quicker. And yeah, we all. So the question is, will the engineers be happy with doing all the vibe coding for the product people that are not yet able to wrap their head around the engineering part of this? I think the engineers are already doing vibe coding. Yeah, but are they doing it with the entire team in discovery mode? Is it because At least the engineers that I talked to, they're all trying it, they're all using it, but for them it's not serious engineering yet and it's a bit meh. So I hope they pick it up. Yeah, but think about how many engineers are like, you want me to build that prototype that I'm just going to throw away? Yeah. Right? Now you can code code it. I realize that we did not even talk about what tools we're talking about. I'm going to share. We are recording. What vibe coding is maybe. Yeah, like we're recording this in April of 2025, but it's not going to come out until June of 2025. In that few month period, there's going to be 18 new vibe coding tools. But the ones that I know about right now that we're talking about, Replet, V0, Bolt, Lovable, Windsurf. There's probably more. Yeah, no, that's the one that I have on top of mind. Yeah, and like if you've literally never played with any of these tools, my goal in this whole episode is just to encourage you to like go build something, go play with it, go see how it works. And if you're not an engineer and you get stuck and you don't know how to get it to be, here's what's gonna happen. If you're not an engineer, you're gonna like build something, it's gonna look great. And then you're gonna find a problem and you're gonna be like, hey, this doesn't work. It's gonna be like, oh, let me fix it. And it's gonna be like, I fixed it. And then you're gonna try it and it still doesn't work. And you're gonna be like, it still doesn't work. And it's gonna be like, let me fix it. And it doesn't, it's never gonna fix it. You're gonna go round and round and round and round in circles. And that's why you need an engineer who can like help diagnose the problem and tell it like, hey, you're trying to fix the wrong thing. Because what's happening in that instance is the agent is guessing. Just like when you chat with ChatGPT, it's like guessing what the best response should be. And sometimes those get, like it doesn't have enough information to make a good guess. And that's where an engineer can like help with the debugging and help steer it to a better guess. Yeah, I could not have built even I have engineering skills myself, but it would have taken me ages to actually do it myself without somebody with slightly more engineering knowledge. So we actually paired on that one to get it to a point where it actually was time wise, the investment that I was thinking about for my prototype. Well, here's what I'll say, maybe by June, I know not to promise anything, but out of my experimenting, I am going to write a blog post about how to use these tools for prototyping. Yeah, go for it. Like I recently posted on LinkedIn where I just said like, hey, I'm starting to use this. What are your best tips for how to get the agent to be unstuck when it's debugging? And I got great tips that I started using right away and they helped a ton. And so I'll include all of that in the blog post. By the time this comes out, we'll be able to put it in the show notes, hopefully. Do it. See, now I'm making a public commitment to writing this blog post. I love it. That's how we hold ourselves accountable, Teresa. So, like, people don't be afraid to try these tools if they're not engineers. They just know that they probably will hit a point where they get stuck. But you're going to get to lots of usable prototypes before you get to that point. So, like, the reason why I want to do an episode, like, why I wanted us to explore this topic is that... I do think every product team should be playing with these tools right now. Like I think every product team can get value out of these tools to support prototyping and assumption testing and documenting requirements and getting feedback from stakeholders. And even if as a product person you don't get to a working prototype, I think you still learn a ton by prompting. towards a working prototype about your subject matter what you're currently trying to build and why you're building it because you have to reflect on all of that to be good and prompting the thing so even if the end result may not be a prototype that is working you still have learned a ton i'd say yep and with that i think we're done i think that's the episode teresa thank you thanks petra thanks teresa

AI prototyping tools are mushrooming—Replit, V0, Bolt, WindSurf, and more—but do they actually help with discovery work? In this episode of All Things Product, Petra Wille and Teresa Torres discuss how AI-powered tools can supercharge assumption testing, where they still fall short, and what that means for product teams today.

From building chat interfaces with custom data to testing leadership self-assessment tools, both Petra and Teresa share their hands-on experiences—what worked, what was frustrating, and how these tools can fit into continuous discovery habits.

Whether you're a product manager, designer, or engineer wondering if these tools are a game-changer or just hype, this conversation breaks it down with real talk and real use cases.

What You’ll Learn:
✅ Why “free to build” doesn’t mean “no cost”
✅ How AI prototyping tools help with assumption testing
✅ Common pitfalls (and how to avoid them)
✅ Why engineering skills still matter in a no-code world
✅ The types of prototypes that are perfect for AI tools
✅ How these tools change team collaboration and speed

Key Takeaways:
✅ AI tools can help build usable prototypes in hours—not weeks.
✅ These tools are great for fast, interactive assumption tests.
✅ You still need engineering know-how to troubleshoot or guide LLMs effectively.
✅ Product teams benefit most when they build and test together.
✅ Even failed prototypes offer valuable learning.

Episode Quote:
“Even if the cost to build goes to zero, if you keep building the wrong thing, you're still wasting your life.” – Teresa Torres

Resources & Links:
Follow Teresa Torres: https://ProductTalk.org 
Follow Petra Wille: https://Petra-Wille.com

Mentioned in this episode: 
Large language model (LLM): https://en.wikipedia.org/wiki/Large_language_model
ChatGPT: https://chatgpt.com/
Claude (Anthropic API): https://www.anthropic.com/api
Teresa’s Continuous Interviewing Course (where she introduced a custom chat interface app, allowing students to interact with a ChatGPT-style tool powered by her own prompts and proprietary data): https://learn.producttalk.org/continuous-interviewing
Petra’s The PMwheel – a Compass for the Product Manager Development Journey: https://www.petra-wille.com/blog/the-pmwheel-a-compass-for-the-product-manager-development-journey
Vibe coding: https://en.wikipedia.org/wiki/Vibe_coding
Teresa’s Assumption Testing Course: https://learn.producttalk.org/assumption-testing
Jeff Patton’s drawing of the three people having different pictures in their head about one thing: https://www.producttalk.org/wp-content/uploads/2025/06/Jeff-Patton-drawing-with-the-three-people-having-different-pictures-in-their-head-about-one-thing.webp
Replit: https://replit.com/
V0.dev: https://v0.dev/
Bolt: https://bolt.new/
Lovable: https://lovable.dev/
WindSurf: https://windsurf.com/editor

Have thoughts on this episode? Leave a comment below.
#AllThingsProduct