The document that can replace PRDs — Rags Vadali

What is a product if you're building an agent? My kind of conclusion is that product you're building is actually now the experience layer you build on top of the agent. So that is the real product to define. More recently, I spent almost like five years at Meta building products there. And in between them, I've worked in six different startups. product leadership roles as a PM now I don't know what I don't know so what I mean is foundation models the capabilities of these coding tools are changing so much that if I try to think about hey I want to build this particular user experience for a user it's already disjointed from what is actually possible we build agents that ask people questions with no expectation of giving people any answers What's happening is that more and more, one way to think about what's happening in terms of like everybody can write code, et cetera, is you're putting everybody in the direct path of how a user is going to use your product. Rags, thank you so much for joining us this week. How are you doing? I'm doing well. Super happy to be here. So I had an interaction with you last week and you shared this fascinating document that you're doing. We're going to talk. all about your process and the product experience document and all that today but before we jump into that for anyone who doesn't already know who you are can you just give us a quick introduction what are you doing these days and how did you get into products in the first place yeah uh so these days i'm a founder first time being a founder of this startup called floto and what we're basically doing is building feedback agents for product teams right we can get into that a bit later but I kind of got into product back in 2007. That was my first official PM role and that was with Google, which also brought me to London. So yeah, I was an engineer by training and I've been an engineer for about five years and I think I kind of felt that itch of wanting to do more with shaping the product itself rather than just writing the code. I mean, good think back to what's happening today. But, you know, back in the day, the only kind of way to do that was essentially to be a product manager. Right. And this was early days of PM when Google was basically, in my view, kind of defining modern product management. So I was very lucky enough to kind of like, you know, get to go to Google. In fact, I actually was the first YouTube product manager for YouTube in London because I joined them about six months after YouTube was acquired. So. so yeah that's where that's where i got my start uh more recently i spent almost like five years at meta building products there and in between them i've worked in six different startups in product leadership bro so i've been doing product a long long time so you've got you know you've been schooled in this very serious way of doing product you write prds you do it the right way but The way that you talked about how you're approaching things at Floto is fascinating. And I've been having this discussion with people a lot lately about how, you know, the entire philosophy of what we do is try and learn fast, try and make mistakes or figure things out with as little risk and as little expense as possible so we can get onto the right track. But the math has changed dramatically. Tell us before we get into the product experience document and all that, just tell us a little bit about your team at Floto and how you guys work together. Yeah, you're absolutely right. Things are changing far too fast. So my team at Floto actually looks very similar like every product team I've been. So it's me, CEO, but I play the PM role. We have a designer, we have three engineers, and we have a growth person, right? So that's the team and it is the contours of... every other product team that I've worked in. But how we work is completely upside down, right? So whereas up until now, I have been schooled in, as you've said, but also like, you know, practiced a kind of like a flow where as a PM, you do a lot of the early work, you spec things out, requirements, PRDs. work with designers, and then something comes out that looks like either a design or a prototype, and then it goes into engineering to kind of build. We basically now start from engineering. So essentially what we do is, I don't write PRDs anymore, right? And we start with problems, right? So we take a problem and then we actually go directly to the engineers to say, hey, this is the problem we're trying to solve for these people. This is the contours of what we think we can build. can you go and kind of like and they just do the first prototype if you will right which they're building directly on our code base right the part of the reason why we do this is as a pm now i don't know what i don't know right so what i mean is foundation models the capabilities of these coding tools are changing so much that if i try to think about hey i want to build this particular user experience for a user it's already disjointed from what is actually possible so we so we've kind of like gone the other way where we just what the engineers are showing us is hey this is what is possible for this problem and what you want to solve right so here take it and they give us something to play with right so i'm kind of coming in after that and going okay now i'm going to play with this thing that you've given me or is it is it for a different reason like what what kind of started you working in this different way yeah it's a great question i think it's a combination of both right i think one well let me start from the from the more practical side of things right engineers are building things so fast that about a few months ago i found myself that In a traditional way, me and my designer couldn't just keep up with even creating the amount of work that they needed to do for a sprint. And we run weekly sprints, not even two weekly sprints, right? So that was one moment where, okay, actually this process is very broken, right? Where as a PM, I could spend time specking out like a, you know, an entire Jira board of what people do. So that is just broken. Just like the speed of execution means that I'm the bottleneck, right? And anything that is kind of like in the thinking and crafting has become the bottleneck. So we have inverted the process to say we don't want to be a bottleneck at the beginning. We'll come back and take our time later. Right. But the other thing which is probably even more critical is we build agents. And I think everybody is building agents are going to be building agents of some sort. So I had this sort of like I've had I've been having this kind of like deep thoughts about like in this world, what is a product? right if you're building an agent and my kind of conclusion is that the product you're building is actually now the experience layer you build on top of the agent so that is the real product to define right it is not anymore uis and stuff like that it is actually all about like what experience do you want your users to have of a product that is primarily agentic right so in this kind of world What I really felt I needed was to really know what the agent is capable of so that then I am able to kind of draw the guardrails, draw the instructions, et cetera. So that combination of things have actually kind of led us through a process where we were struggling, to be honest. My CTO and I have, we've had these things like, hey, Rags, he's told me, you need to kind of start specking stuff and more because my engineer is sitting doing nothing. And so then they would go off and do other things. So yeah, so I think the combination of these things is what brought us here. How did the engineers work together when you give them your product experience document, when you give them the brief? Are they going off on their own and playing with Cloud Code or whatever tool? Or are they working together at the same time as well? Ah, you mean between the engineers? Yeah. Yeah, no, they very rarely work between each other anymore. Because in a sense, right, they're all using plot code, which means these guys are, they all have parallel agents working for them writing the code. So like our engineers don't write code themselves anymore, right? So they're all plots writing, we're a plot code shop on that end. So yeah, so the fidelity of this is such that we kind of like break it down where an engineer basically owns. the whole product. And it's actually insane how much a single engineer can build, right? Like the complexity of the products that they can build and hold together today. So there's very little kind of like, yeah. So is one of them taking the brief and two others are working on other things or they're all kind of competing to see what they can do? No, no, no. So we are basically, we're building like two to three products in parallel now, right? With a team of three people. So essentially I, and this is the other thing, right? So as a single PM, I need to hold all of that context in my head, right? So trying to spec all of this at once is difficult, right? So yeah. And one of the things that we do is we do a lot of experimental stuff now, right? So we'll say, hey, can we do this? Sometimes it becomes a problem. I just ask an engineer, hey, could we do this? And then they just go off and like, you know, a day or two later, they come back and say, hey, Rags, this is what we've done, right? And we actually end up, I think, throwing away 50 to 60 percent of things that we built because i am then kind of doing that connection okay can this solve a user's problem and if the answer is no you know we're very low ego in that sense right saying hey we built it we tried it it doesn't work or it's not going to fit we just move on to the next thing this will never have happened by the way now expect you know in any engineering team that i would have been in it sounds great but also quite chaotic You mentioned that you are almost like the gatekeeper, or not the gatekeeper, but the quality control check, I guess, for is this actually solving a problem and do we want to put it into the product? Are there challenges with everything moving forward so fast that it is hard for you to keep on top of that as well? Yes, absolutely. I think there are, I am I mean, we have a small team, right? It's just three engineers. If there were six, I don't think I would be able to kind of manage, right? So I think we're just right at that time. But what we are also doing, and this is a good thing is, in many instances, for example, our designer will just make the call, right? Hey, I've just tested this. This is the right thing. And we just go, okay, let's just go put it out there, right? And we're also early stage. So we're actually not too precious about internally spinning about spinning, is this the absolute best thing to put out there? Because one of the things that hasn't changed is actually, yeah, we iterate with our users, right? So I think actually makes it much more easier for us to kind of like, yeah, so my guardrails are not like, yeah, these are not like very strict. I am going to like dot and it's much more of like, okay, do we think it's going to work? Let's put it out there. Let's see how it works. And then like, you know, we'll iterate or we'll kind of like, you know. kill the products yeah it's kind of hard to scale i think um to your question and beyond like a few few people with the pace at which we're working i was that was going to be my next question was you've obviously worked in some huge organizations like google and meta and on some you know massive product like products that a huge number of people use like can you imagine how this way of working would apply to an organization like that um No, it's hard. I do know, like in talking to, I recently had a couple of chats with folks from Google and Meta PMs, and I do know that they're all changing the way we work, the way they're working, right? For example, I was talking to a designer at Roblox, and he was telling me now that all designers there are expected to ship code, right? And they're all using cursor. Now, when you talk to the designers what is very clear is everybody's trying to adapt but it's very confusing as to like you know there's a fear that we're just putting out these things at such rapid pace that it's kind of like you know there's no yeah the end user experience can actually start to get worsen because you have four designers pushing, you know, things into the same product and you have 10 engineers doing the same thing, right? So I think there is a, I think there's going to be a bit of a reckoning to move back to, okay, consideration at some level in the way we actually build products versus what seems to be now a focus on delivery, right? And speed. And let's just give Claude to everybody and see what happens. Well, let's actually shift back earlier in the process then and cover that because I think that's going to be critical for this. You're trying to do discovery for three different engineering teams, which are one person plus agents at the same time. Is you as CEO and product person and a designer working together? What does that look like now? What is... How much do you do? Is this kind of dual track that you're working a week or two ahead? What do you actually do for discovery these days? I mean, so the principles of discovery we have found have not changed actually. So the most, the products that we are building or the features of products that we're building that are actually resonating have in every case come out of actually having spoken to people. Right. That's a pattern I can match. Now, the problem is it still takes the same amount of time that you need to do. Right. Like, you know, so one of the things that we kind of like overlay or like, you know, optimize for, if you will, we, we have the same user for which we're building all of these products, which is a designer currently we build for designers. So, so one of the nice things is actually that it's every single conversation that we have. And we have lots of me and my co-founder both are trying, you know, we try to talk to the sort of like, you know, users every week. It does two things. One, it kind of validates or invalidates what we are building, right? Because we're able to like, you can either do a quick demo or dig deeper into, is that a problem? And then we kind of almost end all of these conversations with kind of like an open-ended, the one that really works for me is I just ask them, what is the most frustrating thing for you day to day, right? And that kind of is kind of like a leapfrog for us to say, OK, you know, what is an adjacency that we work on? Obviously, this doesn't work when you just talk to one person, but like, you know, standard kind of UX principles, right? You talk to six people and you can start to see the pattern. So that actually has been something that we have done. But it's very hard to kind of find the time to do this on a much more regular basis. So what we are finding, though, is AI is helpful here. So for instance, right, like we kind of use the, I'm sure everybody's using this, but maybe they're not, but problem discovery, for instance, from places like Reddit and like, you know, in common threads on G2, et cetera. So you can send off perplexity queries or what have you, right, from time to time when we go and say, hey, you know. we're thinking about this, is this a problem, right? Like, can you get me back citations of like, you know, what people are saying and things of that nature. So we're kind of able to do that a little much more at scale now, instead of having to be on Reddit and like, you know, troll threads that you used to do before. And then we're also kind of using our own product a bit, right? So the PXT that I had shared with you was basically for a feedback agent that we're building to talk to our own users. So essentially we want to kind of use our own users as a kind of like in a source. The thing that it has done, this is basically kind of like, you know, elevated in my head as a PM. There were, there were a lot of like, I don't know, PMM type functions that I didn't really pay too much attention to. For instance, right? Like community. Yeah. It was there. I had people who just to build it like, you know, but now I, we realized that actually every user we get. is a very valuable resource if we can actually be in a mode where we can engage with them right it is the lowest cost how fast to speed so we have a discord for instance right where we kind of like you know try to get folks on there and kind of like you know those are kind of low cost high speed ways to get you know feedback some of these guys are on our dms like you know if you want to we can actually talk to them so yeah it's it's very hard but like we're scrambling to kind of like you know find the thing but the principles are still the same i think people will give you the problems that you need to solve with your products. And you mentioned there about the kind of design principles are the same. So, you know, the way that you're doing discovery is the same as how you would normally, but then actually you talked about using AI to help with discovery as well. Yes. And your product itself, I think, you know, you have them. we have what we call synthetic personas, right? So essentially it's mimicking how a normal product team would typically have built personas, but in real life, they never get used, right? They've always seen them being like, you know, in a document or whatever. We're actually able to simulate these with AI and then they can give you feedback on whatever your screens are. What we have found is this, synthetic personas do not work in all cases, in the current state of the art. What they are reflecting, quite frankly is what the underlying models we are using are trained on right which essentially are reflecting average user behaviors they do not give you anything on the edges that and anything on the edges you cannot trust because it's hallucinated right so what we use synthetic personas for and where we are actually though where we do find it effective is very early stage like let's say you want to build something right very early stage feedback that you would get from a person is going to be very much kind of like, you know, there's a section of feedback that is kind of reverts to the me. And AI users are not bad actually to give you that level of feedback, right? In fact, the really big, the really cool insight we've had is that asking synthetic personas negative questions is the best way to use them. What I mean is, you ask them a positive question, there is a tendency to please because that's how the model weights are trained on. The minute you flip the question, they're still trying to please you, but the insights get very interesting. So what I mean is, right, if you take us, let's say, if you have an e-commerce site and you say, oh, why would you click the buy button? That's a very bad question to ask. They'll find you every reason to say why. But when you ask the question, what would make you pause before clicking the button? You actually get two or three insights that you go, oh, wait, yes, this does sound insightful, right? So we do, we are also kind of testing our own product in terms of like how we use it. So yeah, to your answer, I think there's one nuance though. What doesn't change, I think, is the fact that you still need to talk to users to understand problems. And by that, what I mean is, You cannot, and you've never, this has never been the case. You can't just dream up a solution and say, I'm going to build, you can build it. They're rarely going to be successful, right? I think what AI is doing is actually giving you a few more avenues to be able to get that feedback. The gold standard still remains people. And in our product, for instance, like we are building what we are in, we're sort of like innovating on is how can we build interviewers that can talk to real people and bring you. valuable insights, right? I would say they're not 100% there, but probably somewhere in the 50, 60%, again, depending on what you want them to get. And this is something we want to build for ourselves. Ideally, I would love to send a link to you. And if you're async, if you've got six minutes and you can just have a conversation, that could give me a thing versus taking days to set up a meeting, get a call, et cetera. Rex, this is great. I want to talk to you about the actual product experience document itself. You were generous enough to share with me. We're actually going to publish this, which is brilliant. I'm curious before we go into the specifics of it, where did this come from? Because to me, it looks kind of like a PRD. It also looks kind of like an eval. It kind of looked like behavior driven or test driven design. What was the genesis? Just give us a bit of the philosophy of it and then we'll go into the details. Yeah, I think the philosophy of this was I was trying to figure out a way in which I can actually. document or layout the experience that somebody needs to have with this agent to be handed over. Primarily, this wasn't where it's different from a PRD is it isn't meant to kind of for people to sit in a room and get aligned, etc. This is much more like, you know, hey, this is something that I want to hand over to my engineers to say, OK, if you've built something now, this is how you tune this. Right. And as I was writing it, where the evals part comes in is I realized something very interesting that I am not just writing this for a person. I'm actually writing this for a coding agent because I did a loop back with my engineers and said, how did you use this? And essentially what they did is they took some of these, especially the experience principles in the sections, they then fed this into Claude to generate kind of like the prompts that they need to then use to do it, right? So that was a little bit of an unlock for me to say, okay, actually this is go probably going to go pass through in a way straight to the agents right that's where then the eval language starts to come in because you know when you when you think about that you you actually if you work with agents you can't just tell them to do something you tell them not what you should not do you tell them you draw the guard resin that's where this stuff kind of like you know i'm continually iterating on this but yeah that's that's the genesis of this let's um let's dig into some of the sections and um much more direct today because that is what really is the differentiator. Yeah. Okay. So the product experience document, which is not actually anything to do with the podcast. Tell us about the different sections that are in the document. Yeah. To Randy's earlier point, I think I'm trained in writing Google style documents, the PRDs my whole life. So there's, you'll see elements of this, but I think, you know, for any product. the key is the why right and so it was a natural thing where like that's what that's where i start with just saying you know why are we building this right and then it's sort of like we have the section where we where i talk about success criteria right so when have we won i mean i was i was in a frame of mind where i was writing this and like you know when are we successful when have we won and and in here actually for example um the kinds of things are they're not uh necessary some of them are numbers hey we want to get 80 percent of people who use this to be i think all of these are important because as in any standard product spec requirement that you do you want to have ways in which you can on the other end of it measure if the things that you built were successful or not right but i also have qualitative things and in this case i mean very specific to this i was like i want to build something where we can get the same level of insights as if i spoke to a real user for 15 minutes right so that was kind of like a a bar that we've kind of put in place because uh that drives a lot of things right and then the real meat of this is this section that i call experience principles and this is just born out of like you know having used the for instance right like i i was using this first version that the engineers hacked up and i was seeing that um i got asked three questions at once right hey do you think this is i was like no like you know people don't talk like that right so just ask one question at a time Then there would be things where I would say something. I was testing it. I was like, oh, I didn't progress further. And the agent just carried on. I was like, put some breakpoints. I was like, no, there are certain things that are serious. If you find it, stop everything, dig deep because you need to find out. Then we go a certain deep where like, try to understand what they were feeling when they felt this, not just what they did, right? So if you think about it, it's essentially. What I'm trying to mimic here is like, how should it feel to talk to this agent? Right. And those are the ones that I'm, that I've tried to codify actually in this sort of like, you know, experience principle section. Right. And, and then I actually, where we get into evals a little bit is like, I actually put some example, good, bad, ugly kind of like, you know, interactions. Right. And these are actually me. using the product and then pulling some out and going, wait, this was bad, right? And then this is good. And this is so. So if you can, if I can connect back to Randy, where we started, right? When we give people, our engineers, a product thing, you guys give me a prototype. It allows me to do all this, like use it deeply and then go, OK, this is how this prototype needs to be shaped. Right. And that is what goes into like this PXT kind of document. I love the fact that you've got the good, the bad, the anti-patterns in the example interactions. One of the fundamental problems of all this stuff is as it scales, it's not you doing it. It's not a person doing it. It's a non-deterministic, non-predictable model. And you have to teach it what is good look like and what does bad look like. And most of the time, I think we're just focused on this is the ideal, but we don't think about the guardrails. No, it's 100%. And I think I should mention it, but that is actually one of the most critical things, right? because it is non-deterministic, you cannot just specify a requirement and expect that it will be built. You really have to think about ranges here, right? So these are a range of good answers, bad answers. We don't want these answers, right? And hopefully we can build it so that, you know, it kind of like, you know, lands in that place, right? And then there are certain, and then I kind of end this with like success metrics that has never changed, right? In terms of like, if you're a product person, you're specifying certain success metrics that you want for the product. There's two other pits that I wanted to touch on briefly. Besides the example and directions, you've got critical moments and the close of the conversation. I love the way that you're focusing on the entire experience. So tell us a little bit about those two things. What do you mean by those? Yeah. So let me start from the last one. I have found that actually the way you you close a conversation is really important, right? Because in our, as we've built these agents and as we've done this, it doesn't have much of an effect on this conversation because it is ending, right? But what you're doing is you're habituating the user to say, hey, you know what, will you talk to me again the next time, right? Like, was this a good experience? And a big part of that is how you close, right? There are very, very abrupt closing arguments and like, you know, things. So that's why that closing kind of like, you know, conversation closing section came in where I was like, okay, these are ways in which we should close, right? Because... there is a certain behind this you know as you're thinking and writing through you're getting a certain you're building a certain personality of this agent in your head and things of that nature right and and then the critical comments are actually the heart of this yeah if you think about any conversation you've had where you're where you're doing discovery the real insight actually comes when from a problem that a user has right or they say something where you go wait can you tell me more that i mean all of us have had those experiences and when you're and i realize that when you build these agents they're just so naturally monotonic that they kept missing these things right and i i've done a lot of testing but i would say something so so this is so what we've tried to do is kind of say okay there are certain break points here like if you hear this stop what you're doing and go deep right because really that is where kind of like you know we need the the insight right And one thing I should say is the way we build these agents is every time we have like one of these, it goes into a conversation. The agent has a set of goals that it needs to achieve. So it needs to go deep in and out to get those goals. And so this is also breaking a bit of that pattern saying, I know you're on a way to understand how somebody did something, but if they actually said, hey, just break out of that loop and actually dig in here because this is kind of like, you know, important. So that was where those sort of like, you know. critical moments things came from. Rags, designing a conversation, the arc of a conversation and all of that, that's an art. That's not something that I've heard you talk about in your previous history. Where did this come from? How did you acquire the skill? How did you realize you needed this? I guess this is sort of thanks. I mean, this is credit to all of the amazing like. content designers that I have worked with right over the years, because I think I've just like observed, like, you know, and by osmosis, like, you know, learned certain principles just by like, you know, having spent time with folks. And when we kind of started building these things, I kind of, you know, it's something that I then I kind of taught myself, I suppose, right, that, okay, we really need to kind of build this in. In a sense, we build experiences that are the opposite. of what you and i have with llms like our experiences is we ask llms questions right to get answers we build agents that do the reverse like we build agents that ask people questions with no expectation of giving people any answers that's a very difficult kind of like you know uh thing to think and and and we've also seen i mean i have also kind of just observed and seen how chat gpt right like they changed i think it was four chat gpt 4 or 4.5 where there was the model change and the way chat gpt spoke with you change and people were like i'm gonna quit right so it actually matters quite a lot and so this is something where like yeah i think i just fell into it right as we were building these products and we realized actually the conversation is a very big and important part of the experience when you're dealing with an agent so works how do you kind of envisage other people getting inspired by your ways of working and you know where this where this can apply and where this can be used versus where it's probably a bit too bold a change for an organization yeah it's a really good question I think where I suspect this can actually be used quite well is if you're building agents and like you know agentic experiences because i feel like this is quite tuned to to that right where i would say this isn't still this sort of working still isn't um a great fit is actually if you are doing something that is still quite interface and ui heavy because there like the experience still isn't just about show me the possibilities and then I will fix it. The experience actually has to do with, okay, there are things we want a user to kind of like, you know, to feel, we want them to kind of like, you know, go through this in a certain way. And the ways you achieve that tends to be either design, it can be sort of like, you know, the look and feel of it, but more importantly, it's like the UX of it, right? Like what happens when I click this? Should we do animations, et cetera, et cetera. So there I think. it's still i would i would not do this you know i would i would find it very hard to sort of like you know i would i would still work in a more traditional if i may where where i'm actually maybe i'm still not i'm not writing a prd yet but i'm actually doing a very close session with my designer and maybe one of our engineers and we're sitting together and going hey this is what we should build and like you know what and then it probably results in a design or a figma first as a reference that the engineer then kind of like you know goes and builds it. Right. So I think, I think, yeah, the world's like changing, but not everything is changing at the same time. Rags, I want to know about the hiring process that you have now, because, you know, you've got three devs, but what you're doing here is, you know, as you said, they're not coding. They are orchestrating code, but what you're trying to build is an agent that delivers an experience. And I can see that it might be someone with technical skills but also the humanities skills but it might also be the other way around it might be a conversation designer who understands how to build where do you bring people in what are you looking for what is where do you think the future is going with this yeah it's a really interesting question uh i've been thinking about this because uh we we were gonna hire an intern and i was gonna kind of like you know thinking about the interview loop i think so For me, going forward, you know, I think I would make product sense a required part of the interview process for everybody in the team. And regardless of whether I'm hiring an engineer or I'm hiring a growth person, right? Because what's happening is that more and more, one way to think about what's happening in terms of like everybody can write code, et cetera, is you're putting everybody in the direct path of how a user is going to use your product, right? And there were internally, there were, you know, there was specializations where, okay, your engineers wrote the code, but there was a certain limit of thought, for instance, that a designer put into how this experience should be. Today, that is getting short circuited, right? And so what we really need to have is that everybody who is actually shipping code, I would like everybody to have a certain amount of what we would call product sense i would basically transplant my traditional uh google like meta product sense interview directly into my loop actually and in fact i was i was talking to an uh to an intern candidate and like you know i did have a section where i was talking about like simple stuff right tell me what a product that you've used that you really like and like you know why and because i think that sensibility becomes a really really key element of everybody in the team So I think that is a that is kind of like a check to your question around like, would we get somebody who's like more specialized in content versus X? I think what trumps that is basically the ability to use AI in a very native way, because I think you can get the answers to some of these things. if you know the questions you want to ask and you can you know that you know how to ask the right questions right so we are kind of like you know in a so our there's six of us right so me and my co-founder we're we're old guys right like like i'm you know we've been around the block for a long time everybody else we've hired has actually been pretty much straight out of college right or they are interns that we kind of like you know they're growing into full-time roles because i think that's the other part of it like teams are going to be there's a whole There's a big change happening, et cetera. But I think people who are very curious about what is possible actually are extremely valuable because like, you know, we kind of, and that works because I can do the editing, right? And I can do the guiding. So for our situation, I think that kind of works very well. So yeah, I would say product sense trumps a lot of things. And if you can get a product, somebody with good product sense and who's like AI native, yeah, I think we could slot them into multiple roles. Megs, this is so interesting and I love that thinking as well. And you mentioned that you've worked in product and in tech companies for 20 odd years and you've obviously seen a lot of change and a lot of adaptation in that time. And you've adapted your ways of working with the product that you're building and the team that you have. What's your kind of like final tip for anyone else listening to this who? you know, is working in this rapidly changing world at the moment to, I guess, sort of keep their ways of working and their sort of product operations relevant or, you know, experiment beyond the, let's call it traditional product management. Yeah, I think if I could, that's a very difficult question to answer just because so many things are changing. And I think everybody is kind of like, you know, there are some things that are out of people's controls they're being asked to do different things in their same roles um i think what is going to happen right is we are going to have this explosion of products because everybody can build right in companies that's just going to have a localized you're going to have a localized version of you're going to have an explosion of features right you've got 15 people on a team now building All of this will essentially mean that there will be feature bloat and there will be the, all, you know, this has been true of product from 30 years, right? Nobody will use a bad product or an irrelevant product or a product that is not built for them. I think we are not yet at the stage where we are able to draw those lines back to say, hey, you know, we're focused on the velocity of building. And I think in this, people are forgetting about the... quality of what we are building. So I think the one thing that I would say is it's less about what they should do, but I think what they should not stop doing, which is essentially like if you're a product person, right? PM, designer, even a product engineer, just have a track where you're trying to continually understand what the user wants, right? And you can use whatever technology and AI you want to do that. But if you've got your pulse on that, then what will happen is that that feature you shipped is going to be the one that is going to be bringing the users, the revenue, etc. And I think that sort of shakeup will absolutely happen. It'll happen with startups who are shipping all these things and it will definitely happen within products. So I think the core, like first principle has not changed, right? Focus on the user and build something that they want. So I think if you can kind of like still keep your compass pointed there. I think people will still continue to do well. Amazing. Thank you so much, Rags. And thanks for sharing your product experience document, which we will link to in the show notes. But this conversation has been fantastic. Well, thank you both for having me. This has been very enjoyable. Thanks, Rags. Thank you. The product experience hosts are me, Lily Smith, host by night and chief product officer by day. And me, Randy Silver, also host by night. And I spend my days working with product and leadership teams, helping their teams to do amazing work. Lou Ron Pratt is our producer and Luke Smith is our editor.

What does product management look like when your engineers aren't writing code? Rags Vadali, founder of Floto and former PM at Google and Meta, joins Lily and Randy to talk about how building AI-native products has completely inverted his process. No PRDs, prototypes before specs, and a new artefact at the centre of it all: the Product Experience Document (PXD).

They get into why the real product when you're building an agent is the experience layer on top of it, how synthetic personas work (and where they don't), and what discovery still requires that AI can't replace. Plus: what product sense means when everyone on your team is shipping code.

Chapters 
0:00 What is a product when you're building an agent?
1:00 Guest intro: Rags on getting into product at Google, YouTube, Meta, and now founding Floto
3:33 How the team at Floto actually works — and why it's "completely upside down"
6:01 Why building AI products forced a process inversion (and why speed made it necessary)
7:11 Agents and the experience layer: redefining what the product actually is
9:39 Running two to three products in parallel, and throwing away 50–60% of what gets built
14:31 Discovery principles that haven't changed — and the ones AI is helping with
18:15 Synthetic personas: where they work, where they don't, and the insight from flipping the question
22:03 The Product Experience Document (PXD): genesis, philosophy, and why it's not a PRD
25:57 Experience principles: encoding how it should feel to talk to an agent
27:06 Good, bad, ugly: why example interactions and anti-patterns are critical
28:55 Critical moments and closing conversations: designing the arc
33:33 Where this way of working applies — and where it doesn't
35:10 Hiring for product sense: why it now applies to every role
39:43 Final advice: what product people should not stop doing

Key Takeaways
— The product is now the experience layer. When you're building an agent, the UI is no longer the product. The real work is defining what experience users have on top of the agent — and that requires a different kind of spec.
— Invert the process when the engineers are faster than the spec. At Floto, engineering goes first. — — PMs come in after the prototype to shape what the experience should be. The old flow — spec, design, build — breaks down when foundation model capabilities are changing faster than you can document them.
— Synthetic personas are useful early, but limited. They reflect average user behaviour, not edge cases. The most useful technique: ask negative questions. "What would make you pause before clicking?" yields sharper insights than "Why would you click?"
— Discovery principles haven't changed. The products that resonate still come from talking to real people. AI helps you scale signal-gathering — Reddit, G2, Perplexity queries — but the gold standard remains direct user conversations.
— The PXD is written for agents, not just people. The Product Experience Document covers: why, success criteria, experience principles, example interactions (good/bad/ugly), critical moments, conversation closing, and success metrics. Engineers fed sections directly into Claude to generate prompts — which reshaped how Rags thought about the format.
— Non-deterministic systems need ranges, not requirements. You can't specify a single correct behaviour for an agent. The PXD encodes a range of acceptable answers, hard stop moments, and anti-patterns — because guardrails matter as much as ideals.
— Product sense is now a universal hiring criterion. When everyone is shipping code, everyone needs to understand how users will experience what they build. Rags applies a product sense interview to every role, regardless of function.

Product Experience Document: https://docs.google.com/document/d/15kCm8ZcPqY12174WjyfuVLhrWOXGGqnB1vow7o_2ZqI/edit?tab=t.0#heading=h.l62rzxz2fw6v