Ralph Loops: Build Dumb AI Loops That Ship — Chris Parsons, Cherrypick

welcome so this workshop is on ralph loops uh hands up here who knows what a ralph loop is That's almost everyone. I'm guessing that the other folks who came in were just here because they thought that sounded weird or maybe looking for a quiet place to work. I don't know. But you're very welcome. So what we're going to do today, this is a two-hour workshop. We're going to, if you could just kind of make a little bit of space if you need to as people are coming in, that would be really helpful. Thank you. This is a two-hour workshop. We're actually going to build Ralph Loops together. We're going to do this together. on our own laptops in order to uh to make some stuff happen and get some things done so so it's not just about theory this is a very practical thing so if you've got a laptop you're welcome to get it out in a second we're actually going to try and do this ourselves so i have a few slides but not many most of this is going to be live demos and kind of interaction points as well and the idea is that the end of this you should be able to leave with something that works that will will apply it to a kind of toy code base just for fun to create a pomodoro timer but uh but hopefully the idea is that you'll be able to use this on your real work when we get uh when we get done so another show of hands it's okay who is using clawed code or codex specifically to write code hands up Quite a lot of people, specifically to write code. Who is using it to write all their code? Who is no longer writing any code? That's quite a lot of people. Look around for a minute. That is a huge change. If I'd asked a bunch of programmers six months ago who was not writing any more code, you'd get a very different answer. So next question, who is using either Claude code or Codex, and I'll include Cursor here as well, in their non-coding work? Okay, quite a lot of you. What about for all your normal non-coding work? Okay, interesting, interesting. So can you just see the future in the room, right? There's a few people who are starting, but we're still on that journey for sure. And who has built Ralph Loops before? Last show of hands. One or two people. Okay, great. I'm going to be looking to you for all the answers. That's great. So just a little bit about me. My name is Chris Parsons. These days, I spend most of my time trying to help teams like the team I used to run figure out what on earth to do with AI mostly. So I'm a CTO by background. I've done a couple of BC-backed startups and scale-ups. And this has taken me and my friends by storm, rather, and we are trying to still all figure it out professionally together in terms of how to help our teams. adopt and use AI, so that's what I do for a living these days. I have about 30 years or so of building software professionally. I've been the CEO of an agency. I've done a lot of agile consulting, remember that? And that kind of training as well back in the day. And funnily enough, all of those, it's a whole nother talk, all of those principles and practices that we taught for years and no one really listened are still very much applicable to AI. So there we go. um so these days i'm actually running ralph loops all the time 24 hours a day to get my work done so i'm using them to write my emails i'm using them to check my calendar i'm using them to write content and newsletters i'm using them to help me do my client work so i'm using them in absolutely everything i also use them for code which is what we're focusing on today but they are very much applicable to every part of our lives so by the end of the day the idea is that you will be in the position where you can do that too So this is how I used to work with AI until quite recently. You probably can't see that very well. This is an N8N workflow that I used in order to create my weekly newsletter. It took me probably a week to write, let alone actually test and debug. It's got a huge number of different things in here. This is like a featured article flow which would read various different articles from my blog. figure out whether I posted it before, summarize it using AI and put it there. And then there's another one for grabbing links that I'd posted into a particular list. And it kind of did a bit of commentary on that. It was really quite complicated and difficult to run and maintain. And it kind of worked OK, except that 2 p.m. on a Monday, pretty much every Monday, I would get the dreaded notification from NITN that my workflow had failed. And I was just like, oh, no. And then I'd have to go in and figure out. in here what whatever had broken and try and run it and fix it now this is nothing against na10 na10 is a really cool tool and it can do some really cool things and i'd never have been able to orchestrate ai in the way that i was doing before without a tool like n8n just despite being a coder it's just so much easier to manage in here and you can manage all the api keys really easily and it's a nice tool to to stick things together but it was so brittle to use at this kind of level of complexity and i didn't get a huge amount of value out of it and then every time i fixed it i would do something else and so honestly it was probably easier for me to just write the newsletter than it would have been to maintain the thing that wrote the newsletter and i probably had a slightly better newsletter so this wasn't great um but but this was to my mind a few months ago they're really the only way to use ai you had to kind of orchestrate it and manage it give it the right data and kind of handle all the context and i thought that this was the future of automation but it isn't really the future of automation the future of automation is a lot more like something like this uh running in clawed code so this is obviously not the actual skill But I have now, in Claude Code, a skill that writes new letters for me. And it has all of those instructions. In fact, I copied and pasted the NA10 JSON code from there into Claude Code and said, write a skill based on this flow. And it did a great job. And then what it does is it goes through and does all of the things. But what's interesting about that is how does Claude Code work? Well, it reads the first thing. It decides on the next step. And then it reads the next bit. And then it decides on the next step. And it kind of works through over some minutes to actually write. and produce the newsletter that i was writing and it's the same for code it's the same for anything that we want to build using claw code claw kind of just takes care of it you describe the kinds of things you want and it does it now what's interesting is that clawed code fundamentally is running on a loop isn't it it just reads the skill calls a tool goes back to the beginning reads the skill again calls it all calls it all and then at some point it figures out that it's done and it stops and it gives you your newsletter in whatever form you want it in so what's interesting is that this ships much better more coherent newsletters than the previous uh the previous workflow i still have to change them write them screw around with them but but they are uh they are much better first draft than they ever were and i haven't really touched this skill all i really do with this skill is i say at the end of a newsletter writing process please just update the skill with anything you can figure out from the session you should have done differently and it makes the odd tweak here and there um so that's a loop that is this kind of form of working in loops with ai so so Agents that originally start in workflows, where you have quite complicated orchestration that looks a bit like something hellish like this, end up in quite a simple loop, perhaps with a bit better context. Now, this didn't work for the longest time, but it's now beginning to work with the latest models. And by latest model, I really mean... gpt 5.x really gpt 5.12 onwards uh and claude opus 4.6 or sonnet 4.6 upwards so those models uh started emerging around about uh the end of november uh i've no idea about mythos by the way i i've spoken to people who've used it and they say it's it's it's good but it's mostly marketing but we'll see but um but yes uh we'll see what that where that takes us maybe we won't even need skills maybe we'll just say write a newsletter and i'll do it who knows But what I'm trying to, my point is, is that rather than using complicated workflows, we're actually using skills and loops, much more in context and loops. And any kind of agent that we run is in some way a loop already, okay? And this kind of powerful looping construct is something that you could more generally apply. So what happens when we take loops a little bit further? So the first stage, um is this idea or the first idea came from jeffrey huntley uh a little while ago ancient times in ai which means probably about last june and he said basically what we should do is whenever we finish using an ai to do anything we should just try the thing again in some way we should just uh just give it exactly the same prompt and see what happens see what it does again uh and it sounds it sounds a bit stupid and it's based on does anyone know where the story comes from uh who knows why it's called a ralph loop like two people it's called a ralph loop because of ralph wiggum which is a simpsons character who basically says um the uh he just tries the same thing over and over and over again and eventually it works um and it's all it is really all that a ralph loop is is uh build this thing or do this thing inside a prompt then the ai goes away and does the thing and then it finishes and says okay i've done the thing and then it says okay great go and build this thing and you know and do all the things i said to build this thing and it goes oh okay i'll do it again and and the the groundbreaking nature of what that meant was that the uh ai would often review its code and realize it had missed something in some way right so it figured out that uh there wasn't it wasn't quite finished and this is quite a common problem with ai coding tools last year it wasn't quite done it didn't quite get to the end and therefore go oh yeah i should have fixed that bit then does it again and then and then when it stops it says right i'm definitely finished now 100 it's done it's finished And then what you do, you give it another prompt again, say, go away and build the feature. It's like, I built the feature and it looks like, oh, yeah, there was actually this tiny thing that I should have done. I really am now finished and so on and so on. So you can kind of see the utility of kind of going through. that loop where you just build the feature and just then ask it to build the feature and then you ask it to build the feature so so that's kind of the first stage of raf loops and what i'd like us to do is i'd like us to try that so firstly i'm gonna do a bit of live coding hold on to your hats we'll see how that goes and we're going to try and do that process using clawed code to see where that takes us so let me start change what i share sorry just takes a moment Okay. Great. This one. Can everybody see that? Okay. Can people see that at the back? Okay. Do you want me to increase the size? Got thumbs up? Great. Okay. So this is a piece of code that I vibe coded in about three minutes last night. So it's not good. But that's the whole point. We're going to fix it. So it is literally a Pomodoro timer. And you can see how it works. If I go to Python and type Pomodoro, start. Woohoo, we've got a Pomodoro timer. That's all it does. It literally just does start. There is no way of finding out whether it's finished or complete or anything like that, but that's what we're going to change. The other cool thing, which is very important for any self-respecting Vibe-coded AI project, is that it has tests. So look, it's got a test. There's one test and the check to see whether it starts. So that's great. So if we just have a quick look, and you have to forgive me if you're not a Vim fan, because I am. Although I hardly use it now, it's quite sad. you know 20 years of muscle memory just gone but um but yeah so all it does is it literally just runs a start command and then it saves in your pomodoro dot pomodoro in your home directory it saves the time in which you started really really simple so this is a very simple quite straightforward project the difference is that it has a new folder with different things in it and these are tickets so there are a whole bunch of ways in which we could improve this pomodoro timer And the first ticket is it would be really nice to know how long is left on our Pomodoro rather than just starting it. So what I've done is I've created a very simple ticket system to allow us to just kind of capture some changes. These are not, this is one-shotted, so I have no idea if these tickets are actually good. In fact, I haven't looked at some of them, so we'll see how that goes. But the idea is that we can use these in order to start building a loop of work in order to get something done. What I'm going to do to start with is I'm going to start Claude, and I'm literally going to say, write the first ticket. So bear with me while Claude fires up. In some ways, I'm quite glad they didn't actually release Mythos yesterday because I don't think Claude would be working today if they did. That is really not working, is it? That's frustrating. Wow. Let's try again. I think there's a problem. Okay. I mean, that could cause some problems to my talk, but we'll have to see how that goes. i don't have one of those fancy new um macs that allow you to run no this is actually locked up my computer can you believe that it was working literally 10 minutes ago uh let me just have a look uh bear with me while i debug my machine there is um i've got i think i'm on a different wi-fi so it should uh no yeah the wi-fi has gone down fun Might have to be, okay, hang on a second while I tether to my phone, which I think has decent 5G, so we should be good. Okay, let's see if that's any better. Hooray! Okay, let's try again. Not that one. Cool, so code Pomodoro Workshop. Okay, is that big enough? Okay, good. Let's try this. Claude via the power of 5G. Look at that, it works. uh fantastic okay so um what i'm going to do is we have as i said a very very simple stupid pomodoro timer and we're going to implement a ticket so what i'm going to do is i'm going to say implement this ticket so it's in doc tickets zero zero one great and i'm just gonna say that see what happens so what it's going to do is going to read the ticket which I showed you briefly earlier. It's very straightforward, and all it does is it implements a status to see how far we've got. And then what I would like it to do, what I'm going to do after this is I'm then going to say when it's done, because it will literally be two files. It's not going to be difficult for it to do. And then I'm going to say, you know, implement it again and see what happens. So there's a few different ways that you can do this. And there is no kind of one set way of doing a Ralph loop. It's really about the concept, not about anything else. Great. So it's done the ticket. If I just quickly do a quick get diff, you can see that what it did is it added a status command. I think when I had a show of hands earlier, most of you are coders. So hopefully this is not tricky to follow. And then we've got a new test. Look at that. It added a test. It didn't even ask it to. It added a test. Oh, my gosh. What is the world coming to? So now what I'm going to do is I'm literally going to say the same thing. Now, a year ago, this would have been really important. step because it would have definitely missed something whereas now it's like i've already done it it's fine right so opus is now much better at noticing when things are done now a traditional route fleet would just keep doing this right and it would keep going with this kind of implement this ticket implement this ticket implement this ticket and um And this is kind of boring, and it's not really going to do very much else. And at some point, actually, when I tried this earlier, it's interesting. It's done something different. It actually noticed that what it should have done is it actually should have updated the status to done. So the process worked. That didn't work earlier. So that's great. So it's actually noticed something that it didn't do. So there you can see the fundamental early principle of early RALF loops, right? The idea that you can just zoom through and do something and it will find things eventually that it missed. Because it missed that, I'm just going to try once more, but I don't think it will come up with anything else. Like I said, latest models really don't need this step in quite the same way. They tend to just kind of get it done. And in fact, this time it's just like... um oh if you're running a ralph loop that picks up the next ticket oh that's hilarious it's literally giving away my presentation um that's fantastic okay um what i'd like us to do is i think as a starting point the other thing you can do is you can just kill the context and then you can do the same thing again and you can say implement doc tickets zero zero one and i can't bother to spell it it'll find it um and then um so now uh what i'm doing is basically doing the same thing but without it knowing about the previous context so it'd be quite interesting to see what it does with this i'm assuming it'll find assuming it found the ticket yeah it did find the ticket that was easy enough for it to do it's just running the test to make sure that they work and it all passes okay so it's happy so some people when we first started using ralph loops is that they weren't doing it within the same status and and there was an early claw plugin that just on the stop hook which is what runs right now when it stops running it would just do the same command again so rather like me just typing the same thing in each time but that didn't really work very well because it didn't get very far whereas now what's more useful or what people started doing was just running claude code in a kind of loop so they would do something like while true and then do claude implement ticket zero zero one Right. And then done. And then that will just go through. Not quite, actually, because I didn't do Claude P. But effectively, that's what they were doing. Oh, no. Now I've really screwed it up. I really shouldn't have hit enter on that. Should I? OK, there we go. So that's effective what people were doing. The dumbest Ralph loop is literally that, just a while loop. And it just goes through and implements stuff. Super, super easy. um what is the next step in ralph well in fact what we're going to do now is i'm going to get you to get to that point and then i'm going to take questions from other folks so um let me just switch to back to here i'm hoping there we go yeah great So what I'd like you to do, if you could crack open your laptops and grab the code from here. So it's just on my GitHub as Pomodoro workshop. You should be able to find that quite easily. And you saw how I ran it. It's very simple. You might need to set up Python in your machine. So hopefully that won't be too hard. Or you just run by Python. Pomodoro.py will give you the command you can type. and then it's um just a unit test thing to run the test pomodoro super easy and then in step four i want you to fire up claude code or codex and i want you to try and build that ticket and make sure that it's working and that will be a great starting point but don't build any more tickets yet don't don't let it take you too far and then if you are really used to that and that is just like a literal no-brainer for you try it in codex try in something else try and maybe try setting up something similar in one of your own projects um so something different so i'm going to take questions now while people are typing away on that i'm going to give you maybe a few minutes just to get that set up and then we'll kind of move on to the next step does anyone have any questions or comments or thoughts i have a microphone here uh if people would like to ask anything yeah there we go it should come on in a second hopefully the guys at the back It may not be on. Is it on? Yeah, you could shout and I repeat. Yeah, that could work. Can I just check it's on first? Yeah, it looks on. That's weird. Shout and I repeat anyway. Oh, there we go. There we go. I've played a bit around with the BMAD method, which I don't know whether you've seen that. He's got a guy who's basically written a whole load of skills and commands for following a full agile process from... you know and he's got he's got an agent for build it test it you know everything and and i i guess so um have you done anything where sort of using this kind of rail flute process you go through that cycle you go through the full software development life cycle of each stage and then yes um i i might get as far as that at the end um but yes i have tried some of that stuff It's really interesting and it asks some really very good questions both around context and actually the value of the work. It's really interesting. So yeah, we'll talk about maybe that a bit more at the end. So ask again. If I haven't got to it, just ask the same question again and we'll get there in a raffle. Okay, thank you. Anybody else got any questions? How are people getting on with setting that up? Has anyone managed to set it up? You know, kind of wave at me if you've managed to get it running. great great start has anyone managed to implement the first ticket hooray a few people you got a question okay you've done it great fab yeah i probably should have given different directions for asking a question versus having finished that's great so a few people have got started fantastic great so you can probably tell where this is going and if you were paying attention to the live demo you'll already know the answer i'll grab the mic from you so you don't have to keep holding that thank you um but yes you don't have to just stop at one ticket now is matt pocock happened to be in the room i know that he's doing the workshop after this one he is the person i got this from so if he's watching the video thank you matt uh this was a this was a revelation to me back in sort of september last year he posted a brilliant youtube video just about exactly how to kind of take ralph loops to the next level because when they first came out i spotted it on the internet i played with it i was like Yeah, this is fine. It's kind of cool. It kind of spots things that AI can do where it's missed things and it can maybe do a slightly better job of things. But it's not going to change everything that I do. And then the answer is actually it does change the entire way that I work and approach code now. i guess the the really interesting thing is not how do i make sure claude has finished this one thing it's what happens if i point this kind of loop at a whole pile of things to do right what happens when we point at a whole list of things now i tried this i wrote a blog post about this which was a bit depressing because it just showed abject failure in the entire post to be honest but it was more about i tried what i tried to do is i tried to get um claude i think it was claude at the time to break up a big project into a lot of different tickets and then i got it to break down all of those tickets into smaller tickets and then i got it to to figure out what all the dependencies were between those tickets and write them all down really carefully and then i got it to figure out how it could use like a ton of different agents sorry did you have a question that's fine you've got someone there with a mic if you wanted to say it One, two, one, two. OK. I had a problem with Wi-Fi, and I didn't do the clone. Oh, I'm sorry. You wanted to go to this one. A couple of minutes. Yeah, thank you. That's fine. I'll leave it on there. That's fine. Has everyone appreciated the slide? OK, good. OK, there we go. So these slides, by the way, are created using a slide skill using Nanobanana Pro. It's absolutely incredible at making slides. I haven't, apart from the tiniest thing, like adding a QR code, I haven't. added any text to these slides they're just flat images in google slides um they're actually it's incredible at making slides um what was i saying yeah so uh so the idea of just creating uh a ralph loop to just do one thing seemed a bit pointless and and it was just working around a few limitations if you point this loop on a whole pile of work then it becomes incredibly uh powerful and as i was saying before i created this huge complex dependency graph with a whole ton of different tickets about how i was going to build this really complex system for me and then i got i fired up like six or seven parallel agents and i was like right you do one stream and you do that stream and you pick this ticket and it just failed horribly because the the system just couldn't figure out what had been done and what hadn't been done picked up there was lots of contention between tickets like well i can't do anything until you until i get that share ticket done so i'm going to do it and another claude was like well i can't do anything until that share ticket so i'm going to do that too and then they both implemented the same thing and it was a huge mess so that was really very depressing and i wrote a whole thing about it and i was basically like it's impossible to orchestrate large numbers of agents you know you just can't do it which was obviously nonsense but um that's how i felt at the time and what was interesting was that what i'd done effectively was recreate the waterfall processes that were seen in some of the worst companies back when i was starting to code for myself in the 90s where people would write requirements documents that you had to stagger to carry into the requirements meetings where the the entire project was specified up front with all the intricate dependencies handed to the development team and then given two years to build i can see some perhaps um slightly more seasoned people in the room sort of nodding and smiling at me when they hear me talk about this but yeah i thank managed to avoid working on any of those teams but i some of my friends did and it was absolutely awful and what i had done was basically i had given that to claude to do i'd given that waterfall process to claude to to um to organize and figure out as as it went which was really bad so no wonder it didn't work if humans can't do that how was how was ai supposed to do any better um however If instead of saying with all of your tickets, right, the first one is the most important, then this one, and then you should do this one, but don't think about this one until you've done that one. Instead of doing that, I'm going to go back for just a minute. Are we all good with this slide, by the way? Does anyone still need the slide? Okay. Okay, we're good. Right. Instead of doing that, you can just run a route flute where you say something like, hey, just pick the next most important ticket. It's as simple as that. Here are all the tickets. Just figure out what is the most important next one to run. You don't have to worry about the dependencies. You don't have to figure it out yourself. The AI is quite capable of looking at all of them, figuring out the dependencies on the fly based on what's just been done and figuring out what the next most important thing to do is. That's actually quite easy for an AI to do. The one thing it cannot do so easily is manage that process in parallel. But to be honest, when we're running these kind of loops, if you're running them continually, the bottleneck is usually not the number of agents. It's usually you just keeping up with the AI, just doing things over and over again. So let's forget parallelism just for a minute and just start with a loop. See, if you can keep up with an AI, just one AI that's running continuously, you're fine. Don't worry about parallelism just yet. Don't worry about Gastown or any of that stuff just yet. As impressive as those projects are, you can just start with a simple loop. It is okay. So what I'd like you to do again at this point is i'm going to quickly show how this works for those of you who don't have your laptops but then i'd like you to just try it on your computer so again let me find my mouse and then move to sharing my screen again okay great so if i go back to claude in fact i'll go back to vim first and look at the tickets folder so i've got a whole bunch of tickets here i've got a status command i've got a stop command i've got custom durations never use that anyway i use um other things like you know labels and all of that um i could try and figure out the dependencies myself but i really just don't need to i can just simply go into claude and say implement the next most important ticket using tdd principles from doc tickets commit when done something like that Okay. So let's see what it does. So it's now reading a whole bunch of tickets, as you can see. It's read number one, two, and three, and it's decided that the next one is number two. It's just going to do it. That's great. So it's going to work on it. Now, the interesting thing now is that once this finishes, now it's using TDD, so it read the test first. When it finishes, hopefully, yep, it's marked it as done. Very good. Remember that time. And then it should commit. Let's see if it does. No, this is a brand new session, although I think it probably had the working directory from the previous one still. So in fact, I think it has. So what it hasn't done is committed those atomically, which is definitely something I could improve in my prompt, but we can cover that in a minute. So then hopefully it's just going to do that. And then it's going to finish. Yeah. Great. It's done it. Fantastic. Now what I can do is I can either do that again as a route loop, or I can just restart a new session. Just do the same thing. And this time it'll pick something else and then it will keep working. Now you can imagine that if I put this inside a while loop, then it should, in theory, work through all of the tickets in some way. now whether what i get at the end is actually what i want is a whole different question but it will definitely get a lot of work done in a row so i'd love you to try it so if you're if you've got the app working on your computers see if you can get it to work through just as many tickets as you want to within that amount of time so it should be able to just carry on see if you can get it to to actually maybe write a little bash script just like i've done where you do a while true and then claude i'll show you how to do that briefly in fact if i just quickly git reset so it can start just from the beginning and in fact i'm actually going to go up one more hard head there we go great yeah that's the right place to start so if you can get it to do that then that's great the other thing that you can do is instead of using claude like this you can do claude dash p uh can you all see that okay by the way i'm not sure how i can make that um yeah yeah clear is a good one yeah claude there we go so what we can do is actually use claude.p like this and you can get it to output by just doing stream json or something like that what's it called something like stream json um hang on a second let's see i think they've removed it that's annoying never mind we just won't see any output so there's nothing to stop you setting it up like this and then you can just do that yes so the only way that this works is if you want to run this properly and for it to not stop you have to be quite selective about the permissions that you give it so the question was presumably you have to run claw with full permissions for this to work yes yes you do it depends on what you're doing yeah if you're working in a little sandbox project like this the chances of it going elsewhere to find stuff out is very small i have a project called lockbox and the sole purpose of that is to try and stop it doing stupid stuff by why when it reads untrusted tokens which could potentially send it off track it um it basically just prevents any kind of file system access or anything after that so there are ways of kind of managing it um so what this is doing in the background is you can't actually see it doing anything um because i don't have that that output mode but you can figure out basically that to to run this in in some kind of script and if i in fact if i quit that um hopefully it will stop there we go no let's just keep going Sorry, clearly a more production-ready Ralph loop would not look like this. But you can see it's done a bunch of work. So if I go to here, you can see it's already started on the status command. And it's just working through that at the moment. So it started at the beginning again. It was just working. So there's a few things to be aware of here. One is that feedback is really, really important with Ralph loops. You need to be able to have it run. in a uh in a way that you can tell what it's doing and how it's doing it so this kind of super basic one that i've given you there isn't very good that's not one that i would recommend running in production um equally you need to figure out exactly what the prompt is for ralph and that's a really really important point and i think what i'd like you to do when you're trying this is yes it's going to be building a bunch of tickets in a row but equally it's going to be it's going to be doing them in a way that you don't like. So for example, if I'm running this test, which is literally just implement the next most important thing, let's start with this one. I would probably do something like run simplify, which is a really useful skill from Claude, from the Anthropic team, when finished and ensure you refactor to reduce. duplication you can imagine that you can create quite a complicated skill for this and I'll show you I'll show you my kind of actual skill for this at the end but as you're kind of working through this do try and figure out if there's ways that you can improve what it's doing so give it a go let it make a decision and then let it write some code and then read the code and think okay what could I have actually improved about the process and then reset everything and then improve the prompt after that Have a go at that and see how that works. Whilst people are kind of working through that on their machines, I'm happy to take questions. Yeah. Have you used the skills like superpowers? Which one, sorry? Superpowers one. The superpowers one, what I did is I pointed Claude at the entire repository and said, figure out anything that isn't currently in my skill set and implement them for me with my own context. And that worked quite well. So I haven't used those ones particularly, but I've basically ripped them off. That's great. Then, because I use Superpowers a lot, and then I just give tasks like this, and then ask it to run multiple agents in the background. Yeah. Have you done that? Yeah, yeah. So what you can do is there is an agent teams version, which I think I've got turned off in this particular instance of claw code, but what can happen is you can get clawed to use sub-agents within Tmux. So in fact, I think I might be able to turn that on if I can find the... agent teams there it is claw code experimental agent teams so if i grab that and then run clawed with that on then you should be able to say use an agent team to implement the dot tickets in this repo or something like this and i don't this isn't actually running within tmux so um so actually maybe this won't work so i might just try this again Bear with me a second. So if I grab that and then paste that there and then grab that and then run tmux and then run that in theory, in theory. um this should uh start pulling up other agents and it and because like i said trying to orchestrate uh ralph loops might orchestrate agents myself to try and organize all of the dependencies and complexities is actually really really difficult to do But what you can do is just give the job to Claude2do, and it does a much better job of managing that for it. So as you can see, it's already decided to print out the entire thing, the file name and the ticket for each, and it's got a whole bunch there. So it's actually decided that they're all sequential, so therefore it should run none of them in parallel, which is kind of interesting. And then it should, in theory, start an implementation agent. Let's see if it's going to. I think it's just running as a sub-agent. Never mind. I'm not sure that's going to work. If you can get it to do it, then let me know. But basically, it's an experimental feature that only came out a few weeks ago that allow it to start sub-agents in Claude as well in order to do things. Any other questions while people are working through that? Yeah. That's great. what good looks like if that's dynamic if that's static do you put that in the cloud md file yeah great question how does that how does that so um the question is just to to repeat the first half of that is when i use the n8m workflow in order to build a route flute for a newsletter creator uh how did i know how did how did the agent know what good was when you define good okay great question so so in terms of newsletters i had already been writing my newsletter manually so i knew roughly what i wanted it to read like and sound like i also did a bunch of research using a research skill which was something like which i built which is something like great newsletters and i've also did things like that i also said things like this is a fantastic written new in fact i'm just gonna this is not what i do i actually do this is a fantastic newsletter that i've written or that i've read somewhere could you please figure out why this is so good and what are the kind of editorial principles that go went into this newsletter for it to to work really really well and then i would just paste that into paste the newsletter in get it to figure out what what was good about the newsletter and then and then i would check it and then i would say yes there still is an element of human taste here you can't entirely get away with that having said that i do also have a simulate audience skill which basically uses a whole bunch of different personas for different clients or prospective clients that i work with and then i would run run the finished newsletter through that and say run all of these in parallel and then once you have finished that figure out ways that i could improve this newsletter a skill in order to do that so there's a number of different ways you can do that they're kind of audience simulations super experimental but it's actually really effective and and often will will surface insights i just hadn't thought of um i'm uh my My personality, as I'm a bit slightly all over the place, slightly kind of the way that I talk and communicate, often my clients are not like that. So I tend to barrage people with information. And sometimes my skill will say, OK, there's a lot of ideas in this, Chris. You just need to focus on one main point that makes sense. And I'm like, that's so helpful. So yes, what's quite helpful and interesting is that you can use AI to give feedback on AI like that. The great thing about this particular project that we're writing this little pomodoro thing that people are writing is that it's a command line tool and it's really simple to know whether it works so it's perfect for a ralph loop and in fact these little tools that we build for ourselves like for example the newsletter skill perfect for this kind of loop i will often say i want to improve this skill could you please back and forth and and write the content then use another agent to read the content decide if it's any good come up with things things to improve then send that back in and just run that as a loop there's a really cool um i wasn't going to tell you about this until the end but i'll tell you now um there's a really cool uh feature in cyclore code called loop where it is instead of um creating in fact i'll start this in a new session instead of just doing this thing where you have to create your own while loop you can say loop every minute build the next ticket from dark tickets basically. And then what will happen is the loop will set up a kind of almost like a repeat timer. And as you see, it's got a cron create tool, which for the uninitiated just means do something every minute. That's what those five stars mean. And what it's going to do is it will literally just build the next ticket. When it finishes, it will then check the cron again, build the next ticket. When it finishes, it will check the cron again and keep going. So that's great for working through a bunch of tickets. but it isn't just applied to a set of things that you've got from before if you think about it i'll just leave that running up there you could have a loop that does something like this loop every one hour check linear for new bug reports from test um and then i just leave that running um oh yeah um just leave that running and you're going to get you're going to annoy your testing team but but anyway the point is is that you can run these kinds of loops in order to get work done you know in a in a quite an interesting i guess dynamic way even though it's quite a simple loop just find the next thing do the next thing if you think about it heck of a lot of our work is just loops if we're software developers what do we do we look at the backlog we pick the top thing from the backlog we pull it over to in progress, we assign it to ourselves, we check on the architecture, we figure out whether there's other context we need, we look at the change, we make the change, we submit a PR, we wait for reviews, we comment on the reviews, we reject the reviews, we implement the changes occasionally, we submit the PR, we merge the PR, we then go through the release process, then we start again, pick up the next ticket, and so on. That is a loop. It's quite a complicated one. um like we talked about just a minute ago but but it is still a loop it is possible to get an ai to run that entire loop there's no reason not to and that's effectively what's happening here when when um you can set up in fact you would never actually write this you were much more likely to write something like this where you'd say every one hour linear bug finding and you'd have a skill that encoded all of those those chunks of information that I just gave it in a way that would work for you and your particular team does everyone does anyone not know what skills are before I go any further no I think pretty much almost everyone knows what skills are if you haven't figured out what a skill is yet then then this is your homework go and understand how skills work they are the best uh way that we have at the moment of packaging up useful little parcels of context and scripts and moving them to different places or creating different things so for example i'm gonna have about 50 of them that i've written and then they just do lots of different things the great thing about skills is that you can pull them into your context whenever you need them so for example and i'll just do this i can say do you know how to create images using uh nano banana and i can ask um the ai the question and the answer is well i could kind of look this up but it actually knows that i have an image of skill for this funnily enough but if you hadn't got one it wouldn't know but if i then do images and say how do you create images give me the step by step then and what it's going to do is it's going to pull in that images skill and then it tells me exactly how it does it and i've actually written In fact, I will make that bigger so you can see I've actually written a script within that skill that actually does the generation for me. So it's codified the process of doing that and it just picks whichever model it wants to and it gives it content. And I have these specific templates that I use in order to create specific Nanobanana skills. Nanobanana is brilliant. This is how I created the presentation that you're looking at. I have a slide skill and an images skill that work in tandem in order to create. these presentations cool so let's see what the other thing has done as you can see it's already on ticket six the great thing about ralph loops is you just keep working keep talking about something else and it's done a whole ton of stuff here and it's just stopped at this point but in a second hopefully if we just wait it will start the whole process again there we go it's got the scheduled task to run and it's going again So you can just leave Claw code sessions running with these kind of loops in them. They last about three days, so you do have to keep refreshing them. But you can just do that and keep it running even before you get to a more complicated writer script that wraps Claw to do a thing and all of those kinds of things. Any other questions? Anyone got anything interesting or surprising out of their Ralph loop? Has anyone tried this on their real work yet? This would be the interesting thing. Yeah. What was your experience? Have you still got the mic? Yeah. I just made a screenshot Ralph loop for a website context engineering framework. So Claude just takes the screenshots and then looks at the layout because it has problems with geometric like spacing. It works well. Nice. Cool. So you're actually using Claude screenshotting to get feedback. Yeah. Yeah, that's pretty advanced. Not many people are doing that. The people are trying to use like. um playwrights and things like that as well to take screenshots and the clawed in chrome plugin that comes with clawed as well you can use that in order to get it to drive chrome and then take screenshots of what's going on i've had mixed success for that because it's quite a complex thing for it to manage but but for just basic screenshots it works really well for my images and content that I write, when it runs those images skills, it will always look at the images first to see whether there's any kind of weird AI garbled text or whatever, and it will reject them without even showing me if there's a problem with an image. Was there another question or comment? Yeah, there's a question just back here. Can you just pass the mic? Is that okay? Thank you so much. I think it's close to a question that has been already asked, because I'm not quite familiar with rough loops. If I ask... the agent to implement task one that has already been implemented, would it actually check the quality of what was implemented or only check if it was done or not? Great question. Yeah, it very much depends on what you set it up for. So there's no kind of magic to a Ralph loop, it's just a loop. So this loop that I'm running at the moment, in fact, I probably should just say loop stop, otherwise it's going to keep going and use my quota. I think you can just stop like that. I've got quite a fully featured Pomodoro setup now. Come on, time to stop. So it entirely depends on what you write. So if you go through to, what was the loop that I set up? i think it was this one i just said build the next ticket that's very ambiguous and not very helpful so it might not actually finish it it may just decide to build it and not ship it it may not actually be very helpful so what's more interesting is if you go to um it's probably the easiest thing to do if i load my my ralph skill This is my actual skill that I use for Ralph Loops. And what I'm doing, actually, this is slightly out of date, but the one that I've got here is actually using a doc changes folder. You can see that there, but I'm using a doc tickets in this example, but I've changed it on the latest one. um but um but ultimately you don't have to use a ticketing system like a flat file in a in a um in the github repository you could use beads which is steviaghi's version of of this kind of approach which is quite cool i've used it um you could use linear you could use jira you could you know as long as you can get access to it from the ai you can use whatever ticketing system you want for this for the purposes of this exercise that you're working through i tend just to use uh flat files because they just work you know it's not you don't really need anything sophisticated um in the same way um the uh the ralph loop is entirely what you make it in terms of its effectiveness so so for example um in this particular one um i've given it a proper kind of role in the sense that you are one engineer in a relay team do exactly one change then drop the context and stop and start again that's the idea so but this one is designed to be run in a shell script where it has an entirely fresh context each time because i didn't want the context to pollute each time These days, I care much less about that because context is so much larger than it used to be. But when I wrote this, that was very important. As you can see, it's specifically for code that doesn't need human review before shipping. And then basically, it tells you about when work should go in there, what the right time for the tool is, read the claw.md, change the format. This is the format of a ticket. This is all of the rationale. These are the different status values. I ask it to check git state to make sure that it hasn't got a working directory. It's also got, you know, recovery states. So if it crashed, it knows that if there's a dirty working tree, but the tests are passing, but you're probably done, but you might not be. So just double check if the tests are failing. then it's probably just mid-flight but broken. So you should probably just throw it away or just treat it differently. So you can imagine that this was built up over time of trying to get this working, trying to understand what the user wants, make sure test passing is not enough, verify the actual behavior works, run things in parallel, mark it done, blah, blah, blah, blah. There's an awful lot going on. So with a real Ralph loop, you want to be building up over time for your specific project exactly how to check something is working, exactly how to run the testing. your particular framework and dialect, how you submit things to the test team, how you want to comment on particular changes, and what style you want to use, whether you want to pull off the thing that feels most obvious to you or the thing that's highest priority, or a mixture of both, depending on how you're feeling that day. Whatever you want needs to be coded in it. So when you're writing a Ralph loop, I'll show you a link to grab this one at the end, but there's no need to use just mine or something else. Just start with mine and then say, fix this for my project and allow it to change and morph and evolve. A couple of questions, so just come forward. Thank you. Thanks for the talk. Could you expand a bit more on the topic of sandboxing? Because that would be the thing stopping me from running this work. Yeah, it makes a lot of sense. Yeah, absolutely. So there's a number of different ways to sandbox this. For this particular small project, I'm not doing that. Most of my work happens on a VPS, which is away from my main machine. It has a few keys on it that are specific to what I want it to do. and it can access developer tools a lot of them it can only access them read-only it can also access my email but again it has quite strict fine-grained clawed permissions for not sending emails because that's quite important i don't let it ever send an email only ever let it draft them so so i use a combination of positioning the code physically not physically but like away from the machine on a different machine on a vps i use clawed permissions for that as well the permission system is a bit broken but it mostly works i'm trying to you to build lockbox to make it even better i use What else do I do? So the keys that I use are separate keys. So the AI has access to its own keys, which I don't use for my other stuff. So I can see the kind of audit trail of what it's done. So there's a number of different ways of doing it. If you want to just run things simply on your own machine, there's Docker Sandbox, which is quite cool. It's a new feature in Docker, which just allows you to do Docker Sandbox Claude and a run Claude within that sandbox. So you can kind of isolate it within a specific container. That's quite powerful because it allows you to only change things within that specific place in the file system. The challenge with that is that it can still leak data from one of your systems to another of your systems. There's a thing called the lethal trifecta. I don't know if you've heard of that. Simon Wilson coined it. It's an idea that if you have untrusted tokens, internet access, and access to secret important data you don't want to lose you're going to lose that data basically that's that's the the um the bottom line of it so you have to kind of minimize the amount of times that those things collide in the same context um so yeah lots to say about security and sandboxing specifically I tend to run, I don't run with dangerously skip emissions, but I do run with a number of things turned on by default, but not everything, basically. And you kind of have to go through and figure out what your risk profile is and how much you care about those things. And certainly as you're giving, the main things to read up if you're interested is to read up about the lethal trifecta if you weren't already aware of it. and kind of be thoughtful about how much power and permission you're giving to your agents, especially if you're using something like OpenClaw, which is unfortunately insecure by default. I know that they've been doing a huge amount of work on OpenClaw to make it more secure, but it is still a challenge for those kinds of agents. They do have access to a lot of things. Any other questions? Yeah. Yeah, you had a validation step in the loop. this might be anecdotal evidence but as soon as i changed mine to use subagents here now for the validation step it started finding things whereas as long as you're doing the validation in the same step with the same context it just pats itself on the back and like yeah that's a really good point um there's definitely uh confirmation bias going on with agents where they're like oh yeah of course i wrote it fine it was fine i checked it a minute ago uh yeah using sub-agents is really powerful because a sub-agent starts with only a small chunk of context it doesn't start with the full context right so so you can get much more power from it so as a good example from this particular project a really useful skill which i mentioned earlier is simplify simplify is a clawed coding bundled skill and what it does is it will look at the most recent changes and it will run three sub agents to try and figure out what whether your code should improve so you you can see here what it's doing so hopefully this will run these will load and it will probably find a bunch of problems yeah great point great presentation thank you uh did you try open spec or combined with open spec or any other spectrum no i'm i if i'm honest i'm not a huge fan of spectrum development i know that's that's um controversial and i'll qualify that i'm i worry that spec driven development is taking us at the extreme is taking us back to the bad old days of waterfall where we would specify the entire or try and over specify a project even these little set of tickets i'm not that comfortable with i feel like specs should be much more iterative um and uh things that we can see it's already fixing a bunch of things that's quite cool so it found just to finish off that point it found a bunch of issues there um and it's got some fixes uh yeah so specs um i like just in time specs i like the idea of building or thinking through what you're trying to build creating some kind of plan and claw code and then executing it that's fine i'm happy about that and i think that's a useful step what i worry about is a i worry about things like kira where they've codified that into the to the tool i worry that that will almost fossilize that that one approach with with ai that works today but may not work again when mythos eventually comes out right so i worry that the tools are being too quick to jump to a specific structure of work that may not be the right thing in the future. So I'm cautious about that. I think it is obvious, it's a truism that AI needs more context in order to do well, so we should try and give it more context. But I think the idea of overdoing that and over-speccing a project is one to be careful of, as well as over-structuring our process based on what we know about agents today. Because then we'll end up with working with a new kind of AI-driven process that worked best with agents that came out in 2025 or 2026, when we'll still be using that in 2030, and that'll be a pointless waste of time. So those are the kind of concerns I have with it. Any other questions? Yeah, great talk. So you mentioned that you don't like spec-driven and you use RALF, so basically there is no human in the loop. So the question arises, does Claude actually need you there? Where is your input there? Great question. so i've been thinking about this quite a lot recently and having a bit of an existential crisis i don't know about anyone else um but um but yes what what value am i adding here to this thing um certainly not with writing a pomodoro timer i'm not sure i'm adding much value at all i mean i literally said i one-shotted those specs and and there was no point there at all i'm not saying that i don't i don't like planning out a system what i'm what i'm interested in at this point and i don't have the answers is the fact that ai often will pick better specs and write better specs than i can write and will often have a better idea of the kinds of direction my software should go in than i necessarily will have so i like the idea of actually having ralph loops that create other ralph loops potentially or having ralph loops that track whole customer engagements or even whole startups so i have a skill that i'm working on should i show this i'm going to show it it'll be fine What could go wrong? Which is called startup. It's pretty ambitious. But the idea is that it should basically guide a product through an entire startup framework. So it is meant to be run as a loop. The idea is that it, oh, I see you all taking pictures. Now I'm owning this thing. damn it um but with great thanks to ash mario he wrote some brilliant stuff on this i should say that for the for the for the tape um so uh no really really helpful um to me so i built this out of um basically all of the cool books i i've read about stuff so i i'm a startup founder co-founder cto so so this is a near and dear to my heart and what i'm trying to do here is i'm trying to give the ai enough context such that it could run my startup for me and potentially figure out what the next most important thing to work on and then do that in a loop you know and then it then there's a big outer loop that runs that says okay well what's the next most important thing to do let's do that um so it doesn't work but it's it's interesting and it's getting somewhere and um it will often um the first thing it does i don't think i've got it to show um but um oh yeah i will show it because it is hilarious hang on just a second um there's a it i asked it how it was doing uh on one of its loops and it produced a a startup update deck as an investor memo which was i didn't even ask it to do this i'll show you the the demo hang on a second um because it's absolutely brilliant uh let's see if i can just show this window There we go. Air skills, startup update. And so, yeah, it said, yeah, I need to give him an update. So what I did is it said basically, this is how far we've got. These are the problems nobody has solved. This is what we know that's real. This is a skills management tool that I'm working on in the background. These are all the kind of issues. And it came up with all of this cool stuff that could go into an investor deck. To be honest, that's not bad. I think it's actually the GitHub for AI skills, but there we go. And who's going to pay for this thing? How much will they pay? Those numbers are definitely not right. But what's interesting is that it decided that it wanted to do this and figure out all of these numbers based on this, which I think was hilarious. And it was quite proud of this deck, to be honest. And I have to be like, hang on a minute. There's some serious thinking you need to do before you go to that. Will orgs pay for skills government? Would your org pay for skills governance? Great question. Not sure yet. So anyway, the reason for showing that is more to kind of point out that AI can do a heck of a lot and it doesn't do startups well yet, but that's probably down to my skill file, not down to the agent itself. I have a feeling that there are an awful lot of things that potentially will be... um will be loops in the future um i only got that far on my slides oh my gosh um hang on a second um so we've done that we've done that if you are still if you're not just listening to me and i'm still working on this demo i've got a couple of challenges for you if you'd like to do this one is is um you could try upgrading your ticket format um if you like the raw markdown file the doc tickets is fine if you wanted to just type bd install or install beads it's super easy to do that and you wanted to kind of get ralph loop to work with your beads try that out see if that works you know no there's no pressure on you to achieve anything and this little folder uh beads is great because it only works within your folder so and it just installs a little tool so it's quite a useful thing to to try it on so if you wanted to try a different ticket format or you wanted to kind of move this into your main project and connect your your ralph loop to your ticketing system to see how that feels yeah maybe not submit tickets yet but you know you potentially could you could try that and see where that takes you so that's an option the other is is the skill you you're going to need to keep upgrading and working on your loop the loop basically contains all of the know-how about how you as a person will go through that you can take it all the way through from do the next ticket and you can take it all the way up to do the next step in the world dominating startup you're trying to build or whatever it is um you know it works for all of those kind of things and what's super interesting about this is that i'm i'm more and more convinced that everything in fact is a loop maybe maybe as an engineer i'm definitely on a loop on a lot of the work that i do maybe as a project manager or project manager i'm on a loop maybe as a ceo i'm on a loop who knows Maybe, certainly, a lot of the kind of cadences that I work on run in loops too. So I have a skill, and if you're running an OpenClaw bot, you're doing a similar thing, that just runs a heartbeat every 15 minutes. It just, on my VPS, it just fires up Claude, checks a few things, checks my calendar, see if I've got anything happening, and sends me Telegram messages. Maybe that's on a loop. I mean, that is definitely on a loop, it's 15 minutes. I have a worker loop, which I'll show you in a minute, and I have a morning loop where every morning at 6 a.m. it comes up with a full briefing of my day, figures out exactly what I should be doing, and just gives me all the information that I need that's happened overnight, all the emails that come in, all of that stuff. The worker loop is particularly interesting because it basically, I'm not sure I can actually show this. Let's see if I can find something that I can show. Let's see. No, the reason I can't is because it's got a bunch of client information in it, so I can't show you that. But what I can show you is, for example, this screen I can show you. So if I quickly switch to this. So this is an app. I'll make that slightly bigger. This is now how I run my worker loop. So this is an app that I wrote. to manage projects. So I don't have tickets inside my work vault. I have project files and each of the projects is a set of work that I need to do. And then every so often I basically vibe coded a Kanban system and a worker will pick up and do the next step on the project. So if the next step on the project is writing an email because it has an overview or it's checking things or it's producing the slides for my project, it will do the next step. So this, for example, is the workshop. uh prep spec uh project that it's working on and it's it's got a bunch of front matter that is just looking like that um and um it's got some questions for me i haven't updated this it needs to be updated it's got the context it's got a decision trail of things that it's done and why it's done them um and so basically for every different thing that's happening um it just is figuring out the next one um it's also got um notes on other talks that i might be giving that didn't happen in the end It's got feedback from a previous workshop I did on a similar topic, which you can click on. Actually, I can't click on that. There's a bug. But it will basically show the notes from a feedback session. So this project pulls everything together from all of the contexts you can find and then does the next step in a loop. So you can run everything in a loop. You can run all of your work in a loop. When I wake up in the morning, normally I have about 15 or 16 draft emails where people have got back to me and it's had a go at replying to them. I always have to edit them. They're always... okay but um but it definitely has a go at getting getting on with trying to schedule some of my work i have very specific rules about what it can and can't do my basic rule is is this reversible without embarrassment to me and um if the answer is no don't do it um and but just make a little note in the project and hand it back to me and so sending emails it's not allowed to do creating a slide deck like for example This one, that's reversible. It doesn't cause me any embarrassment. So it just got on and did it and gave it to me, for example. It doesn't post on LinkedIn for me. It doesn't send emails. It doesn't send messages. But it does get everything ready for me to review. To your point earlier, which is a very long answer to your question. It has caused me to genuinely question what I'm good at and what I'm here for. Quite a lot of the time, I've got to a point where I'm just the email person who just checks emails and send, check, email, send, check. That doesn't sound like a proper job. That doesn't feel good. So therefore, what does that mean for my work? And I've had to make a conscious decision. Which bits of my work do I want to do and which bits of my work don't I want to do? I don't want to be the email reviewer, but I do want to be the strategist. I do want to be helping organizations think through what on earth is going on with AI and how to kind of fix it for their organization. Now, I could get AI to do a bad first draft, but I don't want to be reviewing AI's draft. I actually want to be doing that thinking myself. So therefore, I basically said, don't do any of that work. I want to do that work. Just give me all the information I need and I'll do the work because I enjoy that work and I'm good at it. So. um ai can do all of the rubbish work but it can't and it shouldn't do the work that i'm uniquely good at but because everything is a loop and ralph this is getting so existential because ralph loops are so um really everything can be used for everything we have to start asking hard questions about which of the bits of work we actually want to do what do we want to do um out of this work it's not just about what ai can do or can't do anymore um yes there's a question there's like loads of questions but let's go at the back i think your hand was up first i think the um chap's coming with the mic well we just for the recording it's really helpful thank you so with the open-ended tasks yeah how would you think about sort of when to stop so do you set kpis up at the beginning or how do you how do you know when it's done yeah great question um i ask it to so again this is this comes down to if i just go back to sorry different window uh this one it comes down to upgrading your loop and you have to basically tell it when to stop so i don't just have one ralph loop file that works for all of those different loops that i showed you earlier um i have different ones for each and for example the worker says when you get to a point where you've either running out of context or you've got to a point where there's an irreversible thing to do then i want you to stop and and report and what report means is in this case update the project file which is just a file in a repository with what where you've got to and in a way that where you present it to me for review. So I'm working quite hard at the moment about that kind of presentation step because I definitely don't want to be reviewing a diff. And just reading text I find really difficult just because there's often a huge wall of it and it's hard to parse. So I'm getting it to start giving me things step by step in slide format. I'm trying to get it to that interface I showed you before. You can kind of see a little bit how I'm trying to get it to show things in different places in different ways for it to uh to get to uh what i need to uniquely do next i suppose so it so to the answer your question it depends on what you're doing i think the most important bit is that you figure out what the edges are for yourself and and and have that real you know moment of you know what do i actually want to be doing how do i want to be involved in this work here not just uh you know ai is helping me do my work and a companion to me it's much more now which bits do i not even need to know about Next question. There's one here. There's a mic just at the front somewhere, I think. Yeah, great. Thank you. Hi. So since you brought up this topic of our involvement, right, so at what stage we really get involved, I mean, you mentioned you don't really review the diff. I guess the most important part of our work now is creating these tickets, right? I mean, first identifying what the most useful feature to implement is and then describing in a way that you foresee all the different edge cases or just explain it the best way possible so that the outcome is what you desired in the first place yeah what's your process of creating this ticket yeah great question so like my concern is sometimes you don't really know yourself until you start implementing i mean the way we used to do it right so during the development process you encounter certain different cases where you need like a custom logic you need to it's just difficult to foresee these things from the get-go and do you like iteratively improve the tickets and you re-implement them or what's your process yeah great question so In terms of how I work, there are two modes of work that is done on my behalf. One is the fully automatic work that we're talking about here where the AI will just get something done. I don't need, when I've got a decent spec that I trust and there's a way of feeding back so that the AI knows that it's good, I don't need to be involved in that work. That can just happen. For every other piece of work I do, I have a, I work on it with and in Claude code. So with that system I showed you earlier, um in fact i'll go back to it so i can show you um let me just change back to this window where is it there it is with this one at the very bottom of any of these kind of projects that it's running for me there's a little thing here which i this is just a vicoded app that i've written for me nobody else has access to this um it has a little vcp command which if i take that and i type this into a terminal window So if I go back to this one, for example, I think this is okay. Yeah, I can't easily show that just because my internet is not going to be able to connect to my VPS. But the point is, is that I'm able to go back to... Yeah, I'll go back to that. The point is, is that I'm able to type that in and just... paste that into my VPS. What that does is it starts a new code session. That session knows where to find that project and it pulls in all of the project context. So rather like loading a skill, it loads the project. It reads the entire project file and knows where everything else is. At that point, it's loaded in everything it needs in order to supercharge that session with me. And then we work together on it. So if I'm, for example, to your point about speccing tickets, I'd have a ticket. i don't know probably a project for that particular feature and then i would say okay loading everything you know about that and it would load them all in and then we would work back and forth on specking out those tickets and then the output would be whatever i needed to get done in order to get that done so if i'm working on something that where i i want to usefully and uniquely do that myself that's when i would jump into a project with claw code and when i say do it myself i don't mean typing it myself or don't mean doing all the thinking i normally mean i get claude to interview me to ask questions so that i can uh give it the information it needs to formulate to do the writing because i don't like doing the typing but i get it to pull the information out of me in order to to get that work done so those are the two modes it's the the back and forth iterating and then it's the it just the automatic stuff and i should also point out that i don't I don't like reading diffs, but ultimately that's the only way that you can review code. When I'm reading a newsletter item, I don't want to read the diff, I want to read the newsletter. Whereas if I'm reviewing code that's important, that is for other people, then yes, I read the diffs. I don't like doing it. Nobody likes reading diffs, but I check to make sure it's working. And I can't see myself not doing that for a while, especially not with security, like conscious code. Maybe with Mythos, I'll just delegate it. That'd be nice. Any other questions? Yes, one here. How do you deal with context rot? So, for example, your example where you have a loop and it takes one task after the other, is it the same cloud code session that takes all those tasks? You'll have to experiment with that, with the slash loop command. Yes, it is. um it's the same session um i when you run it as a kind of while loop outside claw then it's a different session um you have different trade-offs uh with that with the same session you have all the context of the previous tickets and the previous changes that might be useful in practice i've not found that so useful because it can just pull the files as it goes if you're not typing anything into to the session you're not really adding anything to that so there's nothing really in there that's useful so i tend to i've tended in the past to prefer starting a fresh context for each new session um but the loop is very the slash loop is very easy to run and it just works and especially opus is very very good at long context retrieval so it's less much much less of an issue okay uh sorry yeah there's a one at the back as well is there a microphone as well Are you reviewing sessions that are done by your loops or are you just reviewing diffs on the GitHub? Great question. I don't allow any of my workers to close a project. So I would always say, if you think you're done, tell me what's finished and I will close that off. So it could be that there's a big list of completed things that I need to check off. myself but i i want to be that kind of final step of verification the reason that i've added that is because i worry that i'll miss something there's a thing uh that someone coined recently called cognitive debt uh which is the idea of just not being up to speed with everything that your code base can do or or all of the code in your code base and that that worries me so so i tend to want to to at least understand how the code fits together and and how or how the piece of work that i'm working on fits together so i don't let AI get away with just putting something out of my sight without me having a chance to look at it. Otherwise, I feel like I'd lose track of what's happening. Yeah, because I mean, for example, I'm using sessions to track tickets. So instead of reviewing the code or diffs in the code, I'm just reviewing what the particular session was doing. and I even have like a marking system which session is on which status. Is there any way how you do it similarly? Similar, yeah. I think the sessions and the status, I think that can work. I haven't tended to use sessions like that. What I've tended to do with sessions is I get Claude to every night go through all of the previous sessions that I've run that day across all the machines I run Claude. It saves them all into a JSON file for me. And then I get it to uh both figure out how my system could improve uh and also just what i did so that i haven't i don't forget um what happened so it writes a little paragraph for how much i did and um and i use that in order to uh to kind of track work but it's not quite the same as one ticket per session i quite like the idea of having like one context per session i think that's quite a nice idea and one sorry one like by per unit of work. I just haven't made that work. I found it really useful because then I can go back to the particular session when the particular thinking was happening. Yes, and I do do that for sometimes when I've got a project that's running over multiple sessions, I can go back to the previous session. Instead of the VCP command I showed you, I could type VCR and it'll do the same thing. But in practice, though, i i like the discipline of it having to pick up again because it means if it has to pick up again from a fresh context it means that all of the information that was in that session has actually been codified into other places that any claude code session or human could find which means that you end up with a a much more um i guess richer kind of repository of knowledge that you're working in so so there's a question mark around if sessions are truly not ephemeral and you've got them as a store, are they accessible as future context? If you treat them as ephemeral and make sure you capture everything within them into your repository anyway or into documentation files or whatever, I think that could be more powerful. So worth thinking through for sure. Any other questions? Feels like we've come a long way from just write this ticket, but there we go. It's all good, yeah. thank you so much for the talk i have a question it seems like uh in the loop some of the stats might not be necessary like you might go to the code and then find nothing there and would you consider to optimize it somehow or you just let the token burn no just burn the tokens they're not that expensive depends what you're doing um i think we're at the point i should this is a whole other thing um we're basically in the era of free tokens right now um you know i i have a max 20 subscription, I definitely use more than the average person probably who is paying for one of those. So I think at this point I would optimize for freeing your own time up as opposed to optimizing for burning a few more tokens. I don't think tokens will ever get that expensive. I think that the frontier models potentially will be very expensive, but we have really good cheaper or freer alternatives just around the corner, not quite as good. for the latest kind of work that we're trying to do but they're really really good so um you know i think um uh there was at least one that just kept the glm one that just came out looks really promising um that's the zai one i think it just came out this week really really interesting i'm still running claude but that won't necessarily always be the case so i think i i would just burn them i would like i said at the very beginning um you know i i spent a long time doing the whole optimization thing where i was doing this if you weren't here at the beginning this thing you know i spent a lot of time trying to to screw around with with all of this but but ultimately i just now let it run it's much simpler i do get quite close to the end of my max subscription sometimes though i'm slightly slightly nervous about what that means i have to figure out how to get another account yeah max to 200 a month one yeah yeah i get pretty close to that every week i'm quite about 80 now getting the jitters um yeah you had a question do you want to bring the mic back down is that okay thank you i'm not looking at that anymore hello is this uh i wanted to ask you about uh thank you so much for the presentation about fine tuning for for the prompts do you version it do you have data sets that you use to fine tune your entire loop So in terms of versioning the Ralph loop specifically, like the prompt for the Ralph loop. So I use skills for that. So as I pointed out before, everything like that goes into the skill and I get Claude to write the skill for me. And that saves in either your... your dotclawed skills folder within your project or it goes into your home directory under dotclawed skills i use github to version all of those for myself i don't think git is the right skills format for this long term i think we need a new thing hence trying to build s skills in fact which is this idea of trying to make skills much more portable and shareable within teams which i'm trying to figure out so yes i do um i do version control them and i treat them as quite important code and i don't actually i do share some of them but i don't share all of them routinely because they there's a lot of my own ip in there and actually a lot of my customers ip is in there too yeah the question is more with regards to the performance of the prompts so um you were saying that in the beginning you as you go along you improve the prompts as you go along and but are you versioning the that going along and are you versioning the the performance of the prompt overall so when you say prompt do you mean the um the skill itself that i'm using yes yes yes yes so yes so the skill so the prompt lives within the skill so when i type slash bug tracking or slash ralph uh that that is the prompt that that um that gets written by claude and managed by claude which means that the um that whole file is the prompt and therefore that is version controlled. So I always, I have a git running within that setup and then every time I change it, I update. But you remain subjective, how much you have improved. Let's say that your dataset will be an issue, let's say that. And the expected output would be the new feature added to the wrapper. So you could more how do I evaluate whether it's any good or how exactly? How do you know you're actually improving? I see. So how do you know if you're improving? That's a really good question. I do stress test my skills. So with other skills and I say, you know, is this skill any good? Could you improve it? Could you write it? I do. I spent quite a lot of my time tinkering with my system and my skills probably more than I should. I think. It's a bit subjective at the moment. What I haven't done, and this would be a really good exercise, is to try running blind testing where you would run a set of tickets with one skill and a set of tickets with another. Ultimately, because Claude is non-deterministic anyway, I think there's a high level of variability with any of those kinds of tests. So it's really difficult to think about how to... to construct a useful test in that way, to know whether you're actually improving or not. In general, the more context you give into your prompt, the better it will do up until a point which isn't very easy and obvious to figure out where it becomes worse. So it's about kind of balancing that ultimately. But yeah, I haven't done a kind of objective improvement process. A great question though. It's a question just behind you. How are you version controlling the skills? I'm using GitHub at the moment. I do have a product that I'm trying to build, which is, I mean, this thing here. So if you want my skill, by the way, that's how you get it. It's a project called AirSkills, which you saw a brief preview of earlier from my slide deck that my agent put together for me. But the idea is that you can package and manage those skills as a unit. So you can create skills for your organization, you can create skill bundles, you can... create a skill set for your org that works for different teams within your org, and then that all gets versioned and updated for you without everyone having to learn how to use Git and GitHub. That's the idea. It is a real pain at the moment. I found it really, really difficult to manage. Just for myself, even just putting a skill on GitHub, I can't imagine anyone from, there's quite a lot of friction for a coder like me. I can't imagine non-coders using that. So yeah, trying to build this. So yeah, run that command on your machine, you'll have my skill. Sorry, there's a question just here first and then go next. How do you sort of touch on this a little bit, but sort of around the edges, how do you do knowledge management? So I guess, you know, I use Claude for why is my VPN not working? And then I learn something and I want to record that. And then I'm like, I've got a meeting with somebody with a transcription and I have that somewhere else. And I've got a bit of code that I'm writing and I've got all of these different contexts, but they're sort of. very disorganized do you have a way of thinking about how you organize all of that yeah so i have a code directory and i have a vault directory and those are the two directors i work in so the code directory contains a few different projects that i work in a more classic way the vault directory is where i do all of my other work and and frankly i i mostly start working in there even if i'm working on code and just tell it where the code is because the vault contains several thousand files with all of the different stuff that i have picked up learn um worked on with claude over the last several years well not with claude for that long but you know what i mean i started with obsidian a long time ago and i've been working on that vault for for a long time um and and with clawed now it just works on that for me so when i do some research on how to fix my vpn or whatever it is it just saves a file in there i have some specific rules for how to kind of structure and manage that um if you're interested more i haven't written a lot about this but i know andrew kapathi has just written about it using lms as a wiki that's a great article if you haven't seen it already um i know there's a Mina Jojovic actually, funnily enough, has done a thing on Mem Palace yesterday. That's another version of this. You can use that too. There's lots of different systems for that out there. The best way to get started is it's Markdown files in a file system and use it like a wiki. So run Obsidian in one window and Claude in the other and just kind of work with it and save things as you go. and so do you have an agent that then structures and puts those fault into folders or something yeah so it depends on your method i use the zettelkasten approach which is the one where you have one note per thought so any thought of all just goes into a flat folder then i have a slash projects which has all of the projects that you saw, including one for this presentation, which is my kind of unit of work for an agent that we work on together. It does a lot, and then I do some, and then it does some. I have transcripts in there. All of the calls that I've ever recorded go in there. And I use a tool called Lian, which is a command line embeddings tool. So it basically just runs embeddings across the entire... all of the text in the repository all of the transcripts all of the links i've ever saved including all of the content it's huge and then it can find things usefully and easily in there so you you the best time to start that is today because it just takes years to put together i need to write more about that any other questions yes one here you said you had friction while versioning your skills i've been using skills only for last month so i'm I'm not aware of this friction. Can you explain what the friction is? I can. I'm sure there are. Has anyone here had any kind of friction with managing and using skills yet? Has anybody else? Yeah, quite a few different people. So yeah, it's an emerging thing. It's not surprising you haven't experienced it yet if you're not using it for very long. What I found is that if you're just using them on your own, creating a file of skills and managing them is quite straightforward. Putting them in GitHub is quite straightforward. It's symlinks and GitHub repository. It's fine. Where it becomes difficult is how you share that. So how would you share a skill? Okay, well, if you want to use MPX skills, you have to then put it in its own GitHub repository. That feels quite heavyweight just for one skill. I'd have to have 50 of them in order to share all my skills. So that doesn't really work. Then it's more like, okay. If I don't want to do that, do I just send them the skill file? Do I send them a zip file? I mean, I can't think of a better way of doing it. Do I have to have a sub-module in my skills folder for every single Git repository I share a skill with? It just doesn't make any sense. So I think... I think the idea of Claude has got some stuff in there around plugin marketplaces where you can have a plugin which has a bunch of skills. That's the best way, but then you're versioning the plugin, not the skills. So that's probably the most seamless way. It just doesn't work that well. Also, there's a challenge around if somebody contributes to your skill, do you want their changes or not? It will depend on what the contribution is. Are they local to just them or are they... uh changes that could be generally incorporated and that depends on the skill and depends on them so you have to then manage that so do you run a backlog for each skill where you have tickets to improve the skills do you see do you see the kind of i think these are all unsolved problems i'm trying to my contribution is trying to solve some of those but these these are big problems we haven't figured out yet um other questions there's one right at the back um if there's a mic that would be amazing thank you okay it's working My question is about how we can scale up this approach with Ralph Loop, but like for the actual production team, like, I don't know, three engineers, how to coordinate, how to cooperate. Do you have any idea how we can organize it? Do you have any experience? That's a big question. So just to make sure I've understood it, how do you scale this up so that you can coordinate whole teams using this kind of looping approach? Is that, yeah. With all the tickets and the skills, yeah. 100%. Yeah, it's difficult. I think the teams where I've seen this work well is where they are proactive about updating tickets. The great thing is if you connect your ticketing system to the AI, it's really good at updating it. So you should definitely do that. Make sure that you claim the ticket and move it into the doing column before it starts work. And make sure that somebody else hasn't just done that before you start work. Do you see what I'm saying? That's really important to avoid contention. Those have always been issues with bigger teams. Just in the same way, a couple of controversial things, just in the same way that Ralph loops work really well by just doing one thing in a loop and quite sequentially, you know, it could well be that the coordination overhead in our teams is caused by the fact we've got too many people in our teams and maybe we should have smaller teams and just more of them. right so maybe maybe if you're trying to get 10 people to coordinate and using ai and ralph loops and all of that that's just not going to work maybe you need three and maybe that's the way to to kind of run that project and then you split it down and then you have another the other seven people doing something else or whatever does that make sense so so making the problem go away is the first step and making sure that you're already using your coordination mechanisms as the second step um and then just try it and and figure out what what the bottlenecks are be really you know be really good at retrospectives uh with this stuff i think retrospectives and teams are often pretty anemic it's like what should we do less of what should we do more of that's just a recipe for the same or more of the same um and just changing tiny increments which can be good but ultimately this requires a radical rethink so be really conscious in making sure that your retrospectives are changing actual things about how you actually work or or have space to try let's just try using a route loop on all of our work for a week and see what happens you know and if it doesn't work after two days that's fine you know and then if you are someone who's in a leadership capacity and is able to sponsor that kind of work this is what it means to try and move to ai if you want to transform your team you are going to have to sponsor these kind of experiments and be okay with failure because so i'm speaking to the leaders in here for a minute because it's going to be messy and it's going to it's going to fail a lot but if you want real transformation that's the only way to get it you've got to give your team space to try a whole bunch of different things so give them air cover so yeah it's it that's a big and complicated question um i think If you're able to and have the agency to, just try it and see where it gets to. There's a whole separate thing called theory of constraints, which I haven't talked about at all, which is the idea that within any team, in any system, there is always a bottleneck. There's always one bottleneck that's the big bottleneck. If you don't work on that one bottleneck... all of the other work that you might do to optimize improve the system is pointless and actually probably counterproductive so this is why some teams when using ai tools and using advanced ai tools like ralph loops which is just you know ai or you know what we're doing now but just on steroids some teams when they implement it actually go slower some teams go amazingly fast some teams go slow why is that it's because they're not working on the constraint the constraint in those teams might be the review process if you or the release process if you release your code once a month and you're shipping 200 prs not 20 in that release how do you think that's going to go right you know it's not going to go well so that's why teams go slower because what they need to do is fix their release process not their coding speed um so always fix the thing that is the biggest bottleneck first then figure out where the bottleneck moves in the system. And that's not predictable. It's random. So you have to figure that out. Then move and fix the next thing in the system. For more on that, read The Goal by Elio Goldratt from 1984, no less. It's an amazing book. One more. Is there another question down here? Is there another mic? Where's the mic? Oh, you've got the mic. Great. Keep going. Since you were talking about the constraints part, this reminded me, I'm part of the AI team and we have an AI team and they write a lot of microservices. It's in different repositories. Okay. How do you deal with coding now since is it like one big monorepo or is it like small, small repos? You have to try it different ways and see. I don't think that the... The GitHub or sorry, Git architecture, whether it's many repos or one repo really matters. You can always start your AI in a folder above all of your other repos and just get it to work. It does a great job of that. So that's okay. I think the bigger question is what are the coordination patterns within your teams and your services? Who's responsible for what and how does that change with AI? I think that's a more interesting challenge. The main reason I'm asking this was because like Some of the microservices depend on others and then you have to release one of them and release a tag and then updating another and that's just yeah i think what ai will do is it will expose all of the places in which that process is inefficient because it will do everything faster which means that you if you're seeing those bottlenecks where you are getting dependencies between your microservatives guess what that's your biggest bottleneck um therefore you fix it so how do you fix that bottleneck well you might try atomic release system or you might build something that i'm using cloud that using a ralph loop that um figures out a way of um uh coordinate releases across multiple repos more successfully i don't know but that's what you do that's where you work don't work on anything else until that's fixed if that's the bottleneck yeah there's a question here do you want to pass the mic i should say I'm kind of at the end of the content, which probably was clear half an hour ago. The only other thing, I mean, I had a Q&A slide. If you are leaving, you're welcome to leave, but you're welcome to stay for more questions. I would really appreciate some feedback, though. So this QR code is the only thing that I manually added to these slides. If you could just fill that in, that would be lovely and amazing. Thank you. It's literally only three minutes, four questions. um it just helps me to improve and make sure that i do a good job of these workshops going forward um that's also my linkedin i post a lot of content on there do um do you connect with me do um engage with me some of this stuff disagree with me i love disagreement i love it when people say surely chris that's nuts you shouldn't be doing that love those kind of comments uh because it really helps me to think and improve which is what i love to do and because all because the ralph loops is doing all my other work so god no got nothing else to do Great, thank you. So I just wanted to put that up there. Very happy to continue answering questions, though. But if people wanted to drift away, then that might be a good time. Go for it. My question is regarding multi-agent orchestration tools. I'm curious if you've tried things like Steve Yeggie's Gastown or there's another guy who does like MCP agent mail. Yeah, there's some really cool and interesting stuff. I still think we're in the Wild West, literally, with Gastown, things like that. But we don't really know how that's going to go. I have tried Gastown. I couldn't really get it to work, but it was pretty early on. For me, I feel like the agent orchestration side of things is, I think that we overcomplicate things by assuming that they need to be in parallel. I quite like the idea of just starting with a loop to start with. I don't feel the need to... um have my ai um do lots of things at once before i can get just get it to to do one thing well it kind of goes back to the theory of constraints thing again um i don't think speed of you know number of tokens per second is the bottleneck i think that it's our ability to specify what we want and review what the ai has done so so if that's my bottleneck i don't want to introduce more agents so i haven't spent lots of time with those tools kind of for that reason i feel like they're solving a problem that not many people have yet so so Hello, not just for speed, but for example, I don't know if you've experimented MCP agent mail so agents can Lock files and speak to each other so they don't step on each other's toes and you can use Different like odd opus and codecs work on the same project. So you get different brains working on the same project yeah no I have not just I think I've heard of it but I haven't tried it sounds like a super interesting idea rather like someone mentioned earlier about sub agents trying to look at things from a different perspective I've had a lot of value for with doing that I mentioned earlier my simulate audience approach which takes ultimately the way the way that it works by the way is it takes like transcripts and also survey responses on my website and it creates personas and has those personas think differently in parallel sub-agents to take fresh looks at content from different perspectives um so that whole idea of having it um having two different things and two different models as well in that instance is a super interesting one i think we'll see a lot more of that i can see a lot of value in it we definitely know that they're pretty good at agreeing with themselves and you often get better a better contrarian take if you throw away the context and look at it again so great principle I think that the tooling is still super early, which we all know, but they're interesting ideas for sure. There's a question just behind you. Yeah, just keep going. They'll turn on. So great talk, by the way. Thank you. How much importance do you put on? So in terms of phases of how you develop, you're spending a lot of time building out the context, creating the tickets and you have a system to run them in sequence, gets pushed up. How much do you focus or emphasize on CICD, running automated tests, linting? Does that give you the confidence to reduce the amount of code you're reviewing? Absolutely. Well, yes and no. It depends what the code is. firstly i think ci cd good testing is absolutely essential linting and all of those things if you want an ai to do a good job for you why wouldn't you give it those tools to to help it do a good job for you right just the same way that humans do much better when they have linting and ci cd and good tests it's exactly the same it's the same with clean code bases you know um you know it's worth doing all of that work to make an ai do well so so that does give me more confidence in what i'm doing the challenge is if the ai writes the tests then and also writes the code then there's a good chance that it's got something wrong about what you're trying to build so often it doesn't make it doesn't make kind of obvious mistakes the things that it gets wrong is it just completely misunderstands a feature builds and says yep that's fine and then ships it and then i'm like oh my goodness i don't i don't quite let it ship all of the things only only with pre-release projects do i do that but um but yes um it does give me confidence in in knowing that the thing uh is is i guess uh functionally acceptable for release or releasable what i i still want to read the diffs because i still don't trust an ai with security um so um i don't know if i've lost uh lost the screen i think um i think my computer went to sleep i i i just don't quite trust the ai not to lose my customers data and i just won't compromise on that so i will read the disk because i don't want to be responsible for that um it doesn't feel it doesn't feel like it feels like there are some problems for which you can trust ai fully like for example linting testing there are some problems which you can't really trust ai just because it's not responsible to do so so maybe specific changes around security if you're running production database migration she should probably check that they worked before running them production and there are some that are a bit more fuzzy and hazy so UI testing is quite interesting early, you know, the idea of having a great feedback mechanism. If you're able to get an AI to click through your project to check that it works, that's really powerful. It works 50% of the time in my experience, but it can be quite useful, at least have a first go at it. Having good end-to-end tests is actually really helpful if you're using Playwright or something like that. which it maintains for um for the air skills project i showed you i have some very comprehensive end-to-end tests that set up full file systems of skills and get the ai to create two different um personas with running each you know a publisher and a creator and it just checks all the files are in the right places and that's really really useful for those kind of full end-to-end tests equally if you're able to to build those kind of feedback messages mechanisms and give the ai a chance to um to really know whether it's done well that that's a brilliant place to be and so i'm always looking to figure out how if an ai could tell whether something was good or not rather than me and when i'm able to take myself out of that loop it just massively improves the whole process it's not always possible or desirable but as much as i can i do you are designing the feedback uh process though yes you're deciding on the criteria and then letting AI execute on top of it. Yes. If I'm working in a team of one, yes. If I am working with a product owner or product manager or a designer, I'm really interested in utilizing their skills and testers as well to figure out ways to design those. This is the value that they bring to these processes, right? Just in the same way that coders are thinking, how can we avoid doing the typing ourselves now? What about If you're a product manager, how do you get the AI to do the easy stuff so that you don't have to do it? How do you go through that process? Same with testers. Super interesting area of research. Two things that come to mind on this. In a small team, what worked well for me is setting up adversarial reviews. You have spec. The dev agent goes, develops. There's a reviewer that does an adversarial review. You pass that context back. dev agent iterates generally catches a lot of things increases the amount of confidence i have to ship it but even with that the with the rate at which you can create specs and how much code you actually have to review i end up being the bottleneck in the review process still yeah have you Yes, I'm always the bottleneck. I have 30 different things that I need to now review that my AI has done overnight or something. And I'm just like, oh my gosh. And the challenge for me is that a lot of that is not work that I should be doing. The only reason that it's given it to me is because I can't trust the AI with it. But any human could do that kind of work. So I'm now like, do I hire humans to just do the boring work? Is that... ethical you know this is kind of an interest really interesting questions um for us to think through but you're right if we're able to design system i love your adversarial point to build on something else somebody else was saying um if we're able to do that and design these systems such that we don't have to be in the loop i think that is better for all of us because i don't just want to give a human a terrible job until then we're employed so yeah i guess thank you no worries any other questions Should we call it there? Folks, it's been a pleasure hanging out with you. Really, really fun.

Dumb loops beat clever workflows. Most teams building with AI agents reach for multi-agent orchestration, planning graphs, and elaborate tool chains. Then they spend months debugging them. A single loop that processes one ticket at a time, evaluates its own output, and improves on the next run will outperform all of it.

In this hands-on workshop you will build three things. First, a working Ralph Loop that processes real tickets end-to-end. Second, a synthetic feedback loop so you can test and iterate locally without waiting on production data. Third, a self-improving cycle where the loop's output quality gets better with every run without you touching the prompt.

Speaker info:
- https://x.com/chrismdp
- https://www.linkedin.com/in/chrisparsons/
- https://github.com/chrismdp

Timestamps:
0:00 Introduction to the workshop and Ralph Loops
6:12 How AI agents work on loops
10:06 Using loops for software development
10:43 Live coding: Building a Pomodoro timer
12:13 Explaining the ticket system
15:01 Implementing the first ticket (Status command)
17:37 The simplest Ralph Loop: A while loop
22:12 Next steps and taking the concept to the next level
28:19 Implementing further tickets using TDD principles
40:33 Advanced feature: The 'loop' command in Claude Code
49:39 Structuring and managing Ralph Loops
55:33 Using sub-agents for better validation
59:39 The 'startup' skill and ambitious automation
1:04:10 Real-world application: Worker and morning loops
1:09:19 Q&A: Determining when to stop open-ended tasks
1:13:31 VCP command and project context
1:15:52 Q&A: Handling context rot
1:17:01 Q&A: Reviewing sessions and verification
1:22:38 Q&A: Versioning prompts and skills
1:27:12 Q&A: Knowledge management and system organization
1:38:34 Closing, feedback, and final questions