Develop at Idea Velocity - Jeffrey Lee-Chan, Snapchat

Workshop setup. So there are some people who kind of have like never installed OpenClaw and I want to kind of get them set up. And then the other people like want to try an experimental thing, which I haven't even fully gotten working, but I think for more advanced people, that'll be cool to get like a sort of staging environment and run to OpenClaw. So I think if I'm not too busy helping new people set up, then I'll try to help people like finish debugging the last part. or maybe someone else will get it. And I think 10 minutes Q&A, 10 minutes, some more advanced topics, and then just more Q&A. And probably after every five minutes or so, we can do a bit of a few minutes of discussion. So I'll just set some timers to make sure I don't go off track. All right. Let me start. Can everyone see this presentation? Yes. All right. Cool. Yeah. So. This kind of describes my setup. I'll just keep it really high level for this part where when everything works, maybe like 70% of the time, you describe your task. And then what's nice is OpenClaw will have context and memory about what you're talking about. So for example, let's say I'm like, all right, fix this thing. It's a very brief Slack message, but OpenClaw can remember what I've asked it to do before. So for example, I'm working on this thing I'm calling a skeptic agent, and it's a custom code review thing. But if I'm like, all right, fix the skeptic agent or whatever, it kind of knows what that is, which is nice. You don't have to re-explain a lot. I use multiple agents with work trees. That's very key for parallelization, CI code integration. But you can kind of see, you can talk to it and be pretty brief, and then... you know, usually get reasonable outcomes, but sometimes you'll have to debug it. So then it's like, all right, there's some timeout thing. I'm like, give it a longer timeout, right? And then I asked it here, like, you know, what are the priorities and what we're working on? And I'm reading it here. And then I might've just said, like, I just made code for it, right? So another interesting thing about this is you can kind of see from, like, the types of responses I give. they're not that like special as in I actually think an agent could replace me. So that's kind of what I'm working on right now to like get things even more autonomous, you know, and you'll see this with LLMs like you're working with them and they'll be like, hey, I've got this thing ready. And I'm like, okay, well run the test, right? All right. So I'll show the next slide now. Okay. We'll go into some concepts right now. Like what is OpenClaw? So I think what's cool about OpenClaw is it's not just about a particular repo or the code, but more of the concept. So, you know, I've got this cool super diagram, but first concept is frictionless communication. So I know we have other things like Clawed Cowork or whatever, but I think they still have a similar concept where it's like, can you like easily talk to your AI versus like you've got a remote desktop into your computer or you have to go sit down. So that kind of enables a certain really higher level of velocity. Then I think the central axis is very important. So, you know, there's the spectrum. Like I kind of go like a little more cowboy and I give it access. OpenClaw has these agent orchestrator managers that manage your workers for you. Or sometimes when I want more personal control, I use CMUX terminals. So CMUX is like a terminal program, but it's pretty good for parallelization and AI development. Then I used the open source framework that I forked, Agent Orchestrator, to do the workers. And then once it gets to here, this part is not really controlled by me as much anymore. So I'm calling them managed agents, but basically you have a worker that runs Cloud Code, and Cloud Code itself can run agents, and those can have subagents. So once you get to this cloud code part, right, this part is not exactly my stack, just stuff I use, but this part of the stack, I'm changing a lot more. I'll pause here to see if anyone has any questions or thoughts. Can you hear me? So I have a question. So the real question is, you know, why use OpenClaw versus directly use Cloud? So yeah, that's definitely a good question. The reason I use OpenClaw is specialization. So when OpenClaw makes a decision, I want that context to be more about the spec or the goals or the history of what I want in the task rather than the code. And you can just imagine this, right? As soon as you open up Claw, it reads ClawMDs, it reads skills, it reads MCPs. A lot of those things are sort of independent of the actual task. It's more about how to do the task. So imagine like 25% of your context already taken up by implementation versus like when you think about OpenClaw, you're like, okay, I wanna think about exactly what I wanna do and how it relates to all the other slacks Jeffrey has sent me in the last two weeks and put it all together and give me like a reasonable spec. Does every single thing a player does, but it does it without a browser, right? So you could use MCP, which usually is JSON or whatever, or HTTP, doesn't really matter actually. And then the final thing is the browser test. So some things are visual. I mean, maybe you could even have CSS test or JS test, right? And that kind of goes here. But then the final thing is visual. So I would recommend you try similar approaches to this and be like, OK, which problems truly needed that human or not? And I think what's kind of cool is every quarter or maybe even every month, it improves. So before, I had to really manually test a lot because this worked fine, but these things didn't work that well with agents last year, like a year ago, even six months ago. But now agents are pretty good at nailing down a lot of browser tests for me. Versus when I started, my agent would have a lot of problems finding a pop-up and entering a password. Now, no problem. That was a little long, but does that kind of address the theme of your question? Yeah, I think this is good. Good. Thank you. Just remind me for all these slash commands, I'll put links here instead. All right, cool. Anyone have any other questions related to some of that stuff? All right. If not, I will go here. All right. So here's two websites that I've built. While that loads, I'll show this one. So this is like an AI RPG. What's cool about this is I've got a default campaign, for example. um okay pick some options you got an avatar and uh what's cool about this is it builds a custom world for you that reacts to you so you can like kind of say whatever you want do whatever you want you'll get a story back um the main difference between playing this versus like um you know you could just always go into gemini chat or or chat gpt or whatever is that um i have like a dnd system so if you play your own like games or novels, like you kind of just win too much versus like with this, you'll actually do like dice rolls and be like, you know, did the person actually, did you actually succeed in your action or not? So while this is going, I'll also show this thing. Yeah, so this is funny. This website is like a multi AI analysis website. So what I found was like, Whenever I was doing research or whatever, I would go to multiple models and I'd be like, what's the answer? Then I copy and paste them all. I put them into one model. So pretty simple concept, but this does it for you. So usually I like these answers better than like asking one model and I'm like, okay, how do I do this workshop? So I can be like, okay, status, the latest PR now. And what I like about this is it has that sort of vertical tab. type of thing, because when you go horizontal, it's really easy to lose track of your tabs. But we've got notifications too. So when this thing is finished or this is finished and I need to look, it'll give me a notification. And I can just focus on clearing the notifications. So here, the way I'm using these terminals, usually it's more like a manager rather than a coder. And it's kind of interesting, but it gives you a certain benefit where I feel like the worker is not biased anymore. So when I code with these directly, I usually feel like there's a bias where it wants to say things are really working or whatever, right? Versus like, you know, I'll show this. If this had been working in PR 294 by itself, I think it would have been like, this PR is amazing. Like we got to merge it, right? But then this one was like, no, like there's another PR that should supersede it and probably we should just close this PR. right so that's kind of the benefit you get where the manager has a different context than the workers let's see yeah so this is compacting whatever so it'll take some time to complete don't want to wait for it but when it completes you see these notifications and i think my efficiency has improved a lot with cmux um i think is austin or someone from cmux here yeah here what's up cool yeah so austin is uh One of the guys behind CMUX. You know, big fan of it. And I'm not paid to say this or anything, but big fan of CMUX. I've been using it a lot. I don't know. You can spend a minute if you want to talk about anything cool about CMUX. Yeah, so, yeah, one of the creators of CMUX. Really happy to have Jeff as one of our power users and also, like, kind of endorsing it for free. But, yeah, lots of things that we're shipping. We shipped a Cloud Code Teams integration. If anyone is using Cloud Go Teams and wants to actually see what the sub-agents are doing, it will automatically spawn terminals for that. We also shipped CMUX SSH. So if you do any SSH work, you can use our native CMUX SSH to do your tail scale, et cetera. To other computers, you can even use it to run your own open claw in your Mac minis. But yeah, just one of the creators. Please feel free to email me if you have anything that you think is a bug or any feedback feature requests um i'm always available and i just love seeing people using cmux and always want to take as much feedback as i can to like get them as get them okay so just download that yeah they actually have an app too so the app might be nice yeah okay and then use that to help me get back in and set up yeah okay okay cool thanks all right just using the sandbox it should not Where would you, have you run into any troubles, Aaron, about sandbox token usage? No, I'm just, I'm not technical at all. Oh yeah, yeah. I have OpenClaw installed, so very newbie questions. No bad questions, all good. Yeah, the sandbox wouldn't do that. I think if everything's working well for you, you know, keep using the sandbox. For me, I just, you know, don't use the sandbox. so so far nothing bad has happened but i think if i had an external bot then i would use a sandbox too changing the testing a new model like it'll go down then i have to figure out how to get it back up again so i think i want to set up the sandbox oh sorry sorry um okay wait there's two kinds of things okay the sandbox that most people talk about means like they run it in docker or they like have it running an isolated part of their system But the other one was the one where I made the staging instructions where I was like, oh, OK. So that could double your token usage if you send the same work to both of them, if you have two of them. But I think the way I would use it is not like that. I think I would do local development, and then I would run integration tests on the sandbox or staging one. So I have two of them. And then once everything's good, then I would merge the code and deploy to the production one. So it wouldn't double your usage, but it would definitely increase it. But then the benefit is you might have some more reliability. So I made some instructions and I was kind of in the middle of setting it up. But then, you know, it's like competing with the other ones. So I got to debug it. Yeah. But I think, you know, it's worth trying out, though. So I add that for more advanced people that give it a try. Cool. Thanks. If you want it, you know, you can try. Yeah, I'm definitely going to play around. But what model? do you use like do you have a something that like a default orchestrator model and then yeah so say for this mostly use codex 5.3 um i found i found uh gpt 5.4 to just use more tokens um and even 5.3 like like i just get destroyed all the time right so i might have to adjust um so basically i use this until like this is getting low and then i just switch to minimax which It's not as good, but it kind of gets the job done. And then this is more about money than preference, but some work I can still kind of just do here. So I don't always have to...

The biggest gap in production AI agent systems is not the model—it's the harness. After 1,000 hours of orchestrating autonomous fleets under human direction, the pattern is unmistakable: agents that finish complex tasks on the first run routinely fail on subsequent iterations because the surrounding loop lacks persistent memory and contextual guardrails.

In this talk, I dissects the key multi-agent primitives required to turn raw models into deterministic teammates. Moving beyond simple API wrappers, we explore how separating your stack into distinct "Agent Orchestrator Managers" and specialized workers prevents low-level implementation bias.

Using concrete examples from production systems, we will walk through real-time terminal routing via CMUX, analyze the token-burn tradeoffs between leading models, and look under the hood of high-context consumer applications like WorldAI and Consensus ML. You will walk away with a practical architectural checklist you can drop directly into your own agent infrastructure on Monday morning.

Once you're setup you can truly develop at idea velocity ie. natural language  to  code  to  automated iteration  to  evidence produced  to  human review where human interaction in the middle is pushed to the beginning or end allowing improved parallelization.

Speakers:
- Jeffrey Lee-Chan (Snapchat): Most teams use AI tools wrong — humans still on the critical path. I build parallel multi-agent harnesses so one engineer directs 10–20 coding agents instead of becoming the rate limiter.
  X/Twitter: https://x.com/jleechan2015
  LinkedIn: https://www.linkedin.com/in/jeffrey-lee-chan/
  GitHub: https://github.com/jleechanorg