The Developer's Guide to Multi-Agent Automation | Itamar Friedman and Robert Brennan

When you're developing a multi-agent system for your software development lifecycle, you can have multi-coding agents, all of them writing code in parallel or sequential, and you need to decide how you want to do that. You can have a diversity of agents with different roles. So roughly speaking, there is multiple consideration you need to have. There are certain tasks that are super repeatable, super automatable, where you can get just another level of automation, another level of productivity boost by, you know, running multiple agents in the cloud to take down these like very... Automatable tech debt oriented tasks. Is there value in saying I'm going to use different models, different agents, different vendors of agents in terms of doing different roles? So I think some people when they're thinking about different agents, the first thing pops into mind some people is like, okay, I give it a different prompt or something like that. That architecture is basically like an architecture of one single agent that rules them all. All you need to do is give it different tools and different prompt or so. That's one option. The other option is that agents are much more different than that. For example, their entire architecture, their graph of how they build might be completely different. For example, if you're dealing with a coding agent, then in order to really do its job, like it's actually, at least until now, proven that you need to enable creativity. Yeah. Because we developers, we bump into a lot of issues and we bypass and this and that, like. And actually the fact that the core is an LLM making the decision is a good architecture for a coding agent. At least what we're seeing, for example, in Sweebench, if you're familiar. And then when you're talking about other types of agents, for example, you want to have a security agent or you want to have a code review agent, that agent might want to have actually much more structure. checking a b c d for example you might have like 100 rules that's how do you get them with subtopic we can talk about you want to have 100 rules that you want to make sure that are obeyed and then you want that agent review agent to be much more struck like structure there is an llm there but the graph of that agent might be like uh one to ten there where llm is involved but actually it's quite quite structured or so and by the way you might give it different permissions etc so so overall like there is a discussion do you have one agent that rules at all or multiple multiple agents what we're seeing of course like when you're dealing with enterprise grade heavy duty brownfield when compliance, standardization, et cetera, you must have like different agents that are completely different than each other. Different vendors. So maybe like, I don't know, Claude Code or Claude or something like that. Do you find like that will be better in a specific role? So like, is there a specific vendor that is best for code reviews or security testing? Yeah, actually, it's a good point to continue from my last point where different vendors have different DNA and different like point of view on the world. Antropic believe, at least what I understand, is that model is everything. And you need to give that model the brain of everything and you need to give them agents in order to operate in the world. These tools need to be quite simple, actually on the fly. Why do you need to start doing retrieval augmented generation? Just give it a simple tool like ASD, grep, etc. That's actually pretty good, I believe, let's say structure and way of thinking for coding agents. But then when you look under review security review agent if you're familiar like three four five months ago what all the rage and you like to click on the prompt you are now a security review agent and then there is a prompt and then there is like a list of exclusion don't look at DDoS don't look on that like basically it's when you're doing a security agent you need one plus of one minus of that yeah you here is everything that you need to check rigorously and then you can be creative on all the rest yeah when you're doing coding agent you want to relaxes and say what what not but when you're doing a security agent then actually you need to do a one minus of what they're doing yeah so i would use their their uh cloud code for example as a vendor when you're talking about cloud code as coding agent But I don't think it's like you can just set up with a different prompt to be like all type of agents out there. Absolutely. And very briefly because I see people are kicking out now so it might get a bit noisy. Very, very briefly in terms of context, like security is a great example where there's a ton of context that you can provide an agent to say, look, here are your rules. This is what you have to check for. At what point does... At what point where does the task that you give an agent just become so overwhelming it starts doing a poor job? Security review is a great example. Code review maybe even with a large number of styling rules and those types of things. Where's the limit for that? Yeah, context is extremely important. I can say superlative context is the king, the kingdom, etc. But it's extremely important. Actually, I talked about it that developers reports that 33 to 80 percent depends on which tech and how you ask it. think that Pontix is actually the major problem of bad quality and hallucination and things like that. And that's our number one request from coding agent to improve. The thing is that context is not just one thing. It's not just a code base. It could be, for example, a database of vulnerabilities. It could be, for example, Coda is a leading code review tool. For example, we have a database related to best practices and rules that we project depending on the stack or what you have, et cetera. And we learn it from different interaction with developers and the context engine. So we collect the database that related to best practices and rules. And then it's really like you really really need to fetch the right context. And then it's not just like when you're thinking about context on one hand, it is like you have too much not accurate. Like the other is like exactly only what you need. In the middle, for example, it could be like a lot of non relevant, but all the relevant, et cetera. So it's kind of a spectrum. And of course, like the one of the hardest thing is actually bringing the right context. And then what we see. the hardest to bring the right context and then making sure it's only that there. These are the priorities. And then that could define a lot. For example, as I mentioned, security companies might have context relevant for security that regular coding agent do not have. Code review solution like Codo might have additional context that is relevant to best practice standard, etc. that regular coding agent don't have. That's the way I look at it. How do you bring the right context and then cut that you don't have too much because there is like the... context rots if you want to read more about it yeah yeah like on tropic and other companies talk about the context fraud yeah super super interesting and but yeah so correct so it's so right as well because context but giving too much context you think okay i've got all these rules that i need to follow you throw it all in one go and actually it's just going to forget what it wants to forget or it's not going to add it it's not going to you know i think like if i connect all the dots we talked about context is very important maybe one of the most important and then it's how you use them When you upload, for example, context, plot code or cursor, for them it's like part of the, you know, part of their, like, just literally the context. And then they just keep pushing that in the context in their graph. If you, like, give it the same prompt with the same context, seven times you get seven different solutions. But, so, like, I do want to say, like, context is important, but then how you use it. While maybe other tools... will use it very differently more like in a workflow verifying different things etc so context is extremely important uh and i just want to add on what you're saying not completely following the context is also a matter of how you build the agents yeah absolutely so that's another point absolutely it's more it's been awesome it's been awesome catching up hello joining me here is robert brennan who is the ceo of open hands robert good to see you here yeah good to see you as well and of course I saw you here, well not here, but in New York only a few weeks ago for AI Native DevCon. Yeah. Good to see you again. Yeah, no, likewise. What's your session going to be about today? So we're going to be talking about how to scale code maintenance with AI agents. I would say there's a lot of tasks where developers are for their day-to-day feature development. They're used to like pairing with quad code or the open-hand CLI locally on their laptop. That works great for your ad hoc work. You pick up a ticket, you need to work on it. But there are certain tasks that are super repeatable, super automatable, where you can get just another level of automation, another level of productivity boost by running multiple agents in the cloud to take down these very automatable tech-debt oriented tasks. Awesome. And I guess when we talk about maintenance, what kind of things are we talking about? Typical maintenance tasks that a developer would do. Yeah, so very typical things would be things like dependency management, resolving open source vulnerabilities that happen in your code base, and then some meatier tasks like maybe upgrading from Python 2 to Python 3, or upgrading from an old version of Java to a new version of Java, maybe migrating from COBOL to Java or another more modern programming language. There's a lot of these toily tasks that developers don't really like doing. They don't involve a lot of creativity or critical thinking. It's mostly just a lot of churning out lines of code, but they're still too big for a single agent to one-shot in many cases. Doing what we call agent orchestration can allow you to put some forethought into how you're going to solve this repeatable problem in a way that can scale across many code bases and then just automate it 90% of the way there. uh and just have a human in the loop for you know the very little bit where you need that last uh last bit of verification so let's talk about agent orchestration then so when we talk about agent orchestration we're obviously talking about multiple agents do they have to be the same agent or can they just be different vendors that that kind of thing yeah i think it's it's helpful to be working within like a single agent framework right uh there's nothing necessarily stopping you know one agent from talking to another agent um and there are you know there's the a-to-a protocol and things like that um But what we've tried to build with OpenHands SDK is like a single framework that allows you to create multiple different agents with like different system prompts, different sets of tools that they have access to, different MCP servers that they have access to, and, you know, maybe different behavior patterns. Right. And then getting those agents talking to each other is, you know, it gives you kind of like a single framework for thinking about how to. coordinate multiple agents together. Right. And when we think about when we start with kind of like a large task, typically that a number of agents can all work on. Yeah. We don't obviously just throw that large task at all these agents, group of agents. We need to break that task down. Exactly. What's the process of kind of breaking that down? Is there is there a matter of human interaction in that or can we somewhat automate that for AI as well? Yeah, I think that's that's kind of the most critical step is like, how are you going to break this problem down into sub pieces that individual agents can execute on? Right. So I think that's where the most human thought goes into the process. But you can work with an LLM like Claude, for instance, to kind of talk through the problem and get some help from an LLM to break down the problem into manageable bite-sized tasks. But you're also going to have to really lean on your own intuition for what is actually solvable, what's reviewable by a human, what can you give to the agent that you can then look at on the other end when it gives you some output and quickly verify, did it do the thing or did it not?

A “one agent rules them all” approach doesn’t work at scale.

In this episode of AI Native Dev, Itamar Friedman (CEO of Qodo) and Robert Brennan (CEO of OpenHands) unpack how teams can evolve from ad-hoc AI pairing to platform level automation built on multiple agents.

On the docket:

• how coding agents and security agents require fundamentally different designs
• how agent orchestration enables 90% automation with minimal human review
• what “maintenance work” means in a developer’s day-to-day

AI Native Dev, powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development.  

Connect with us here: 
Itamar Friedman: https://www.linkedin.com/in/itamarf/
Robert Brennan: https://www.linkedin.com/in/robert-a-brennan/
Simon Maple: https://www.linkedin.com/in/simonmaple/
Tessl: https://www.linkedin.com/company/tesslio/
AI Native Dev: https://www.linkedin.com/showcase/ai-native-dev/