Orchestrating Multi-Agent Development at Scale with Gas Town

I've got some personal life developments. Listen, I know we tried to keep this to 10 minutes and keep it tech related, but I've recently become mayor of a place called Gastown. Actually, I'm trying to become mayor of like three Gastowns and I figured I should talk to you about it a little bit. So I think last time I got very animated while we were talking about the Ralph loop and this new development feels like it's like a Ralph loop or maybe concentric Ralph loop. So there's big loops, middle loops and smaller loops. What I'm talking about is essentially a way of managing coding agents at large. Imagine you're managing one or two or three coding agents. I think some of us have been there before. Maybe you're spawning sub-agents in Claude, or perhaps you're using, I think we talked about Roundtable before, to spawn different agents in Codex. Maybe you're using Ader. We talked a little bit about AMP before, but basically ways of taking tasks and making smaller bite-size and then distributing them to other agents that will report back. I think this is one of those paradigms that we got used to toward the end of last year, I'd say using agents and sub-agents to get work done. Gastown is a re-imagination of this type of workflow, but made concrete with things called beads, which I'll get to in a moment, and some roles of this quote-unquote factory that's essentially going to spit out code for you. The question is, how does it all work? What happened was Jan 1, while I was still trying to take my break over the holidays, Steve Yegg over at Sourcegraph dropped this blog post. It's pretty long. I think he says it's 25 pages. It's not that long. I think he's being a little bit sarcastic, but it is a fairly long read about how you might want to come up with maybe a company or like a CEO role and how you would manage many different developers, many different engineers within that company to get a lot of work done. In fact, I think his update just the other day said he landed, I want to say 40,000 lines of code with 100 PRs that came in from the public that he didn't even look at, but rolled into Gastown. What I'll try to do is give you a quick overview of some of the roles. There's some nice imagery in here. Of course, it's AI generated. It's got a throws and cons. You might see some typos in these images, but I think they do a pretty good job of illustrating what's happening. Imagine you're in a fictional city. It's got a steampunk vibe and you've got some people. some roles, some some agents in this city that are helping you essentially go around and look at the tasks that you've got and work on them, but also ensure that the people that you told to work on the task actually have done the work. or if the work is done that it lands essentially. As I'm sure you're aware, working with multiple agents in Git can be a problem. You can use Git work trees, but still even then the agents can trip over themselves and you can end up in an area where you've got or into a zone, I guess, where you've got a whole bunch of merge conflicts and then you're working through them. I've certainly done that to myself by mistake while using multiple agents at the same time and run into issues. Without further ado, I'll quickly go into an image that I think explains this fairly well. Again, this is from Steve. I'll see if I can zoom in a little bit more here. My apologies. There we go. A little bit too much. Sorry. I'm going to be ends with things that are a little bit differently. Gastown is like an engine to deliver features for you. Now, this is one way of anthropomorphizing. like an engineer or a workflow that will work for you. I think in terms of the roles and the names are things that might get a little bit confusing, but imagine that you're the mayor of Gastown. So the city that we're talking about with all the employees or workers inside of it is your city to rule. In fact, the city may have many roles that are stratified, so set in stone, essentially. A couple of them would be things like the dogs. So these are ones that are going to go around and do health checks for you. So think about a supervisor. We can use these names, but I'll try to make it English language so we understand what's happening. So these are at the bottom, the dogs. So you're doing health checks, diagnostics, these guys here. And then when we move over a little bit to the left. from the the crew the dogs who are doing all the work you've got what's called the pole cats now the pole cats are transient workers and again just imagine these are contractors so you're bringing contractors when you need help you might hire 50 at a time you could use them for maybe a week maybe two weeks depends on you what you need but you get a lot of work done really quickly very similar i think to a sprint where you know you've got to deliver something but you understand that it's a lot of maybe grunt work or work that might take a lot of time or hands but it's not really that valuable. It just needs to be delivered essentially. That's what this is. Now with those quote unquote polecats, I've been having nightmares about this. I'm so sorry. With those polecats running, you may also need witnesses. People to essentially manage those contractors. As the work is being done, making sure that the work that you said should be done is done correctly, that it's done timely, and that nothing has been dropped. We're still working on things. Now, these witnesses themselves will actually have something that watches them. So we've got a deacon and we've got the... Sorry, I'm trying to go through this as I go. It's hard not to laugh. It's kind of a funny analogy. But you've also got a refinery. And this refinery is very similar to what we were talking about in terms of plan-driven development or spec-driven development, or essentially deciding what good should look like. prior to going forward. So this is on this left side here. So essentially what you do is you give it plans, this engine, and as the mayor, you watch how the plans go through this machinery to be able to watch things. Now, you've probably realized that above the mayor is your overseer. And this is, you're kind of like operating this whole thing without necessarily touching it. In fact, I think a good analogy that's been bandied about is like, pruning a garden that you can't see. So imagine, for example, that you know that the garden needs to be pruned, but you don't know how well it's grown. So you have to have someone go out and work on the garden, but you have to describe to them what they need to do. So you have to say like, hey, my rose petals need cutting back a little bit, make sure the hedge is round, that kind of thing. So you've got to put in those plans in order to see what comes out of the machinery. Now, all of this, again, this is a whole image. You don't necessarily need to buy into this metaphor or this analogy, but because of the way it's been set up it actually looks like this in tmux now i'm going to talk about a little bit about the the the bad things here just just for a quick second so steve will tell you that this is dangerous you shouldn't touch it it can cause a lot of problems in fact at the very least it's like a wallet attack so it's going to use a lot of tokens in fact i think we've talked a lot about my anthropic plan being a max plan i thought it was on the max 100 plan but i forgot about exchange rates and inflation and things like that and i was actually on the max $100 plan, not the $200 plan. I ran out of credits really quickly. This is something that burns credits. You'll probably want an unlimited plan if you're using multiple computers. Some people use multiple cloud plans. You can use codex. You can also use olama. all three of those working on two computers right now, just to do things like the snap judgment I've got Olama working on, for example, or if I want like a longer task, I'll have codecs work on those. But for everything else, it's essentially cloud code with Sonnet that's running. Again, so looking at this, you can understand. So you're the Panda in this equation and all of this work is being done here. So you can see that they've all got names and you're conversing with the mayor or maybe many mirrors as you go along. I mentioned Beads early on and Beads is something that Steve developed. prior to Gastown, but it's essentially a way of handing to-dos back and forth between people. If I want to give you a task, Jason, I'd wrap it up in a bead and essentially put it in a database that you would check prior to doing work and then you might maybe make a GitHub issue or a PR or merge request or some commits locally, and then those beads would be considered either done or handled or in work. What's nice about this whole system is it can show you what's being worked on as well as show you the work that's being done. At the end of the day though, in terms of monitoring the system, of course, I tried to set it up with data logs. I'm watching logs and trying to figure out where we're going in terms of token usage. But in terms of what you're actually doing, your biggest interface to view what's happening is essentially pull requests and looking at this. It's a lot of changes. In fact, a lot of code will change really quickly if you're not watching this. This might not be one of those things that you want to run overnight because so much can happen so quickly. If some of these quote-unquote poll cats or your contractors, pick a direction that you're not maybe in line with or want to happen, it can go very far down that path and you have to claw back some of that context in order to get to where you were before. Anyway, a word of caution. The other is that this operates pretty well in YOLO mode, but in YOLO mode, a lot of things can happen including deleting files if you run out of space. In fact, I got into a situation on the VM that I was running this where it was trying to delete. its own VM file because of the way I had shared some folders to it. So be very cautious about how it runs and what the environment is that you've got set up. In fact, I think the same applies to Ralph. You probably want to have this set up in a container or a VM. I ended up using Virtual Buddy for this just because I want to run it on Mac, but I feel like you could use anything else. VFKit if you wanted to. I think on the team we were talking about Lima and Kolyma as well would be a good example, but you do want to containerize this or contain it. All right, so that's a lot, but there's a lot that happens. Imagine you've got Gastown set up, everything's running. I think what happened to the community is a bit of a murder mystery, and I'm almost happy that I came to this a little late so I could get a different version of Gastown, for example. I think what was happening is that essentially early on, you would hand these these bees, these tasks to your polecats and they might go AWOL, missing. And you would go to look for your tasks and you couldn't find it. And it might be that you need to find out what you sent to them, but it wasn't there. On top of this, I corrupted the beads database, which is like a local gate or jujitsu, so you put tasks on it and then check it back, maybe like a Kanban board. I've corrupted that database numerous times at this point in time, and I think it's because we're running into this space. But without that database, you don't know what you've assigned or what you've picked up, so it's very important to watch those as well. All this being said, I'm going to scroll quickly through because I think there's some other graphs in here that might be interesting. There's an inner loop in here. Let's see if I can find it. My apologies. I think this should do. This is what it looks like. If we zoom in here, we can see at the bottom, we've got these beads, Gastown, and maybe the font's a little bit large, so it doesn't actually show properly. But you can see what happens. On the far right, you've got a mailbox that tells you what's happening. You can see that it's been handed off, so that's a handoff mail. You can see that currently it's paused and quote-unquote Wyvern is what was watching it. Then we've called it Gastown and then we can see that there's Beads in use. This is basically what it looks like. Now these are three different interfaces, so it can be a lot going on at the same time. Now for the last bit, I'm just going to try to show you my console as well. My apologies just to show you that I got it running. All right, so here we go. All I was doing was working on an AKS cluster, essentially like a demo that I had set up six years ago. I don't know if you've done any Kubernetes demos, but generally after a month or two versions will have changed and APIs will have changed. It's typically pretty trivial to do about six months, but after six years, a lot of things have changed, both on the AKS side for Azure, on the Kubernetes side itself, as well as in the app that we deploy at Datadog for a sample app. A lot of changed and I felt like it's not insurmountable. It's only a few files to deal with, but like, hey, can I move this from Terraform to OpenTofu? update things for ARM, that kind of stuff. Now, I'll try to scroll up here a little bit, but you can see that it just plows through. Here we can see that it deployed everything to Kubernetes. It was just flying through this. This I think was in like an hour. Again, these are thousands, thousands. I think it's hundreds of thousands for Zolkens that just flew through. It's a lot of work. Again, not really too much to see here, but you can see on the bottom that it is in fact running on the Tahoe VM. Me, the mayor is alive, we're watching at this, but there's nothing actually happening right now in the refinery. I know it's hard to see here, but on the bottom, that little factory there, zero to one, there's nothing currently happening. It'll tell you the time and tell you where you're running, etc. That was pretty long. There's a lot to go over. Again, I want to stress that this is early days and that agent orchestration can't only be done by Gastown. There's multiple different ways of doing it. But this is one opinionated way to think about how you might orchestrate your agents. Now, I'm going to give you the shortcut. All of this is really complicated to set up. It has some dependencies. Go is one of them. Tmux is another because it uses Tmux for a lot of this. It will use Beads as a dependency as well. Depending on the installation that you've got, things could be complicated because you might already have Go with certain versions installed or Python or what have you, and it becomes a bit of a mess. I would suggest starting fresh in a new folder. I'd also suggest having Claude run your mayor for a little bit. This is because unfortunately, this whole system can corrupt your Tmux config, which can lock you out of it. But Claude's actually pretty good at being your quote unquote mayor stand-in. I think that's why Steve had you in the operator box above the mayor. With Claude, I was able to essentially set all this up, deploy it to two computers, have them use Claude, Codex, and Olama on both. Make sure I'm using Olama for the the faster tasks that I need that could be less expensive, don't require a lot of context. or if I need to do something like text-to-speech or multimodal or compare images, I've got Olama doing that. Now for long-running tasks, I've got Codex, and again for the more chunky tasks of the bulk of the load, I've got Cloud Code running. Now this is across two computers and two Cloud Code accounts, so you can imagine the cost is not cheap. I just happened to realize that I had signed up the Cloud Code Max on my iPhone, which is quite a bit more expensive than just signing up directly. So as one expires and I unload to the other. This is what I'm doing right now. Now, that being said, I don't think I'm alone. And Steve himself, I think has 40 Cloud Code accounts, I believe is what he says publicly. So it's 40 about right. I think it is. I think he's paying around $4,000 per month to Cloud Code Max plan. So it's non-trivial. It's a lot. Maybe that's 20. My math isn't great. My apologies. All this to say, it's fairly complicated. There's a lot going on. But if you want to approach this on your own, you could probably point Claude at the article and ask it to come up with a system that's similar to this that it can run with sub-agents if you're a little bit more, let's say cautious. I would also say that combining Ralph Loop with this or sequential thinking and Ralph Loop with this will allow you to run. like a mirror of mirrors essentially, which is what I was doing for a couple days going through this. I got a lot to talk about early days as well. I hope to be able to show a couple of examples, some graphs and monitoring because there's a lot going on and you do want to be careful with the features that you're landing. But I think this point in time. So imagine all this is done. You've got a company, you're going in Bezos mode, so you're watching all this stuff. There's a couple things to be wary of. One is you're going to get tired. So Steve said he falls asleep for like 90 minutes at a time on a cot by his computer because he's just watching this. I will say that this type of work is more similar to management and it's quite stressful watching all of the code go by because you don't want it to trip over itself and cause a problem. A lot of the times you're merging PR. I think the CI CD monitoring is maybe where a lot of the time is better spent or watching your PRs come in. So your pull requests and making sure you can compare the code to make sure you get a feel of what's happening. But some of this, it feels like a bit more than by coding. I know we talked about vibe driven development maybe about a year, year and a half ago at this point in time, it feels like just yesterday. Steve would say that this is more like beads driven development. I think in my case, it feels like a CEO task. It's pretty stressful. I think I said enough cautionary things here, but please be careful. It's early days or other orchestrators, you can use Claude to emulate this for yourself just to see how it works as well as run it if you want to. phew i think i got that out of you sorry no that's great i've got one question uh what has this like changed for you in terms of uh you know thinking about how to monitor or create observability around llms like there you mentioned in that model in the gastown model there's the dogs that that sort of keep an eye on you know some level low level metrics or something right so I would love to know more about like what that component does, what that persona is doing. And is that supposed to be reporting back to the human on top in terms of some sort of feedback loop there? Right. Yeah, you got it. So all of those roles essentially roll up to another supervisor. And the way that you use Keepalize throughout the system, throughout that train that we're looking is nudges, essentially. So imagine I gave you a task, you're a polecat working on things, and I'd say, hey, Jason, I gave you something. Did you work on it? Are you working on it? Could you give it to me? It feels like a project manager poking for an update. But I think the thing that's different between a human and a polecat would be that you could have gone somewhere else. You're ephemeral, you're gone, you're not there anymore, so I can't ask you. I've got to go into essentially a mailbox and figure out what's been sent your way so I can reassign it. product or project management, product management on the user side, so on your side, defining what you should do and project management within this organization trying to figure out what's going on. But all of it is essentially nudges. Now, you probably saw an owl in there. I think it's an owl, which is essentially another supervisor which will nudge the mayor because the mayor can fall asleep as well, go to sleep on the job. So you got to make sure that they're also watching. But a lot of this is just careful nudging. And some of this, again, you could do in Cloud Code as well. write a to do and make sure you're auditing the to do's but the to do gets so big at some point that it's very difficult to manage that context as well as split up and divvy it up to agents so it's similar but not the same to that to do workflow it's just as distributed amongst a team of quote-unquote agents okay all right great well as always it sounds like another micro step change like there's it's kind of sounds like a little significant i don't know boost in something, but also per usual at this phase, it's very token hungry. But that just seems like part of the process. So interesting. Okay. Well, thanks, Ryan. Appreciate that. And we'll talk to you again soon. Sounds good. Talk to you next week. Bye.

Ryan dives deep into Gas Town, Steve Yegge's ambitious multi-agent orchestration system that manages dozens of AI coding agents working in parallel on large-scale development tasks. Drawing an analogy to concentric Ralph loops and a steampunk city factory, Ryan explains how Gas Town coordinates different agent roles using beads for task management.

The system uses 'beads,' a task management database similar to a local Git or Kanban board, to distribute work among agents. Steve Yegge landed 40,000 lines of code with over 100 public PRs using this system, though it requires running ~40 Claude Code Max accounts costing thousands per month.

Ryan shares hands-on experience setting up Gas Town across two computers, using Claude Code for bulk work, Codex for long-running tasks, and Ollama for fast operations. The demo includes modernizing a 6-year-old Azure Kubernetes Service cluster, migrating from Terraform to OpenTofu - work that consumed hundreds of thousands of tokens in about an hour.

0:00 Introduction - Ralph loop and concentric loops
0:43 Multi-agent management paradigm
1:25 Steve Yegge's Gas Town factory concept
3:22 Mayor role and city workers
4:21 Polecats and witnesses explained
6:04 Tmux interface and wallet attack warning
7:11 Beads task management system
9:22 Murder mystery - missing tasks
11:13 AKS cluster modernization demo
15:05 Management stress and monitoring

#AIToolsLab #GasTown #AgentOrchestration #MultiAgent #ClaudeCode #Codex #Ollama #Beads #Automation