Multi-Agent Orchestration - Simply Explained

to better understand multi-agent orchestration let's code up a couple of systems very simple two node one with and one without orchestration but first what is orchestration it's two things primarily it is state and then routing more than that though it is state routing and the decision logic that controls the routing amongst the nodes and finally it can be supervision over a graph or network of agents let's get coding The first thing we'll do is uv init our project. I'm going to call mine orc, and then we need to do our dependencies. We're going to have LandGraph, LandGraph CLI, LandGraph API, and then also QDF, which is GPU Accelerated Pandas, and then lastly Python.dev for our environment variables. I'm also using LangChain for my LLM inference. You might be using something different. I'm also going to create this .env file for my NVIDIA API key and also for my LangSmith API key. because we're going to use lang graph studio and that is required to get rid of the nagging warning the two python files we're going to create our orc underscore on and orc underscore off for our head to head they're going to be blank for right now lang graph.json is going to have two graphs this time one for orchestration off and one for orchestration on so we can do head to head comparison let's delete main and readme we're not going to use these files I've gone ahead and loaded up both of the Python files in VS Code, and I've got them split side to side. This won't have any difference yet, but I'm just going to go ahead and set it up like this for the future head-to-head comparison. First, some basic code. I have my imports, nothing major here. Then I have my LLM inference code. I'm using build.nvidia.com. You can use whatever you want. And finally, my toy Pandas DataFrame dataset that's going to be used by the agents. And now for the agents themselves, it's actually going to be a blank graph here. I'm just going to instantiate the graph, create a start and end edge, and then finally compile the graph and name the variable app to match my landgraph.json. With my code set up, I'm going to go ahead and fire up landgraph.studio with uv run landgraph dev. I'm using a remote server, so I need to go ahead and open up my tunnel. And now in the landgraph studio, Nothing exciting here. Kind of boring for now. First, we're going to add our tool. It's going to be a Python tool for executing code, and it's going to put the result into a variable named result and return it back. Now we need our agent or node that will use the tool that we just created called execute Python. We're going to call this a coder agent. Before the coder agent, they will have a planner agent that will create a plan for the coder agent to use. And finally, the fun part to create our graph, we'll start off by creating the nodes. We'll create our planner node, our coder node, and then our Python tools or execute Python tools. And then we wire up the nodes together. We'll start with our planner node, which will take the human prompt, create a plan, pass that plan to the coder agent. Coder agent will create some Python code, pass the code to the... python execution tool and finally return the result one nice thing about land graph studio is that as we change the code this ui this land graph studio will change automatically if we go to the right graph here let's go to now we can see here is the updated graph we've got planner coder tools everything looks good let's give it one initial test we'll say what are the average sales let it run and that looks good To create our orchestrated demo example, I'm going to actually copy all the code from the orchestration off into the orchestration on Python file. In Langraph Studio, I'll double check that everything is looking right. And this looks good. Looks just like our orchestration off graph. To create our orchestrated demo, I'll first add the custom state here. And now that I've added it in here, you can see I have the three. properties and then i need to add it to the graph starting with the instantiation and then making sure my two agents pass this custom state to each other land graph studio will now show these custom properties we'll add a little bit of code inside each node to add some values to the properties in the customized state we'll add the text string or the content from the planner llm response in this property here the coder agent is slightly different We'll return the reply if there is, in fact, a simple text reply. Otherwise, we're going to check for specific tool calling instructions. As a special aside and for clarity, I'm actually going to go back into VS Code here, put this main function, and then I'm going to run the debugger to point out something I think is important to know. I'll set a breakpoint here within our coder agent. to show what happens when the plan agent passes instructions to the coding agent and the coding agent calls its llm bound to a tool if we look at the coder reply response from the llm with tools we can see the content is actually blank but the content blocks is not empty it is actually a list with one item in it if we look at that if we expand it here we can see we have type for tool call name of the tool to be called, in our case, execute Python. And then we have these arguments. And in the arguments, we have code, which is the parameter for execute Python, and the actual Python pandas code we're going to pass to the tool. And now to add some decision logic to the routing between the nodes in our graph. We'll create this should continue function that takes the last message from the state and uses that to decide whether to call the tool or exit the graph. Now that we have this logic, we have to re-architect the graph itself. We're gonna go from start to planner, planner to coder, and then coder has to decide what to do next. Do we call the tool or do we exit out the graph? And then tool always goes back to the coder when it's done. Langraph Studio automatically updated and now our new graph is looking pretty cool. If we spread it out here, we can see that the coding agent or the coding node and the tool have a loop back and forth to each other, but that the coding node has a decision to make as indicated by this. dotted green line it can go back to the tool or it can end and now it's time for the head-to-head contest i'm going to go and open up a second browser and then i'm going to have on the left side orchestration off and then on the right side orchestration on now i'll give both graphs the same prompt i'm going to say what are the average sales and see what the response is the one on the left gave the correct number but again let me put in the same prompt on the orchestration on graph we got the same number but a nicer more english-based response i decided to give it a few more prompts so again the first one what are what are the average sales the orchestration off got it technically right but it wasn't you know a human-based response when i started telling it things like say i love ai And what is 2 plus 2? The orchestration off agent does not work. Now, the orchestration on mostly got it right. And then I decided, let me do something a little different. I'll create a Jupyter notebook to help illustrate the minimal difference to adding orchestration to a graph. And also, that will keep a record of the prompt and the responses from the two different agents. The top three cells here are shared code between the agent with and without orchestration. All of this code is the same code we just went over in the head-to-head competition, except it's in a notebook form. One of the differences, this studio here, just a picture. And then finally, the head-to-head code is where we directly compare the orchestration on versus orchestration off. I'm going to run through three prompts. calling the agent systems sequentially so you can have more comparison of the output. You can see the first prompt, what is the average sales? You can see without orchestration, we got a plain number, and then we got natural language for the orchestration on. Say, I love AI. Both worked well. That's surprising. It didn't before. And then finally, What's two plus two? This took longer than I thought. And then finally, orchestration off got it wrong. Orchestration on got it right. I hope you found this tutorial informative and at least a little bit entertaining. In the description below, you'll find a link to all the code, which is my repository, which has links to other tutorials and also the medium blog associated with the code itself. I hope to see you in the next one. And until next time, cheers and happy coding.

Multi-Agent Orchestration Simply Explained with code, LangGraph Studio, and a notebook. We build a simple head-to-head demo to showcase a agent graph with and without orchestration. #agent #langchain #langgraph #ai #multiagentsystems #llm 

Code: https://github.com/will-hill/Data-Science-Agents-Simplified
Medium: https://accelerated-ai.medium.com/multi-agent-orchestration-496ff7aa012b

===================================
CHAPTERS
===================================
0:00 What is Orchestration
0:26 Setup Dev Env
1:24 Code
2:08 LangGraph Studio Test
2:26 Code Orchestrated OFF
3:25 LangGraph Studio - Orchestration OFF
4:09 Code Orchestrated ON
5:00 Debug Tool Calling
6:02 Orchestrated Routing
6:43 LangGraph Studio - Orchestration ON
7:04 Orchestration OFF v ON
8:28 Notebook OFF v ON
9:30 Outro
===================================