Reliable Agentic Systems need Durable Execution 🤖 - Marc Duiker - NDC London 2026

All right, well first of all thank you very much for showing up at the last time slot of the day I mean it shows true dedication so I really really appreciate that That doesn't make me feel so so alone here so perfect I'm gonna talk about reliable agentic systems need durable execution So yeah, let's look what that means. Let me start my timer and by the way everything I'm doing is straight in Visual Studio. My slides are based on Markdown. Everything is scripted, what I'm running. It's all thanks to a very nice VS Code extension called Demo Time by Elio Strijf, another Microsoft MVP. Yeah, if you're into presenting and doing stuff, then definitely check out Demo Time. It's really a super, super cool VS Code automation tool, I would say. It's not just a presenter tool, but really like a very cool automation tool. So I'm Mark Duiker, I'm a developer advocate at Diagrid. That's a company founded by the co-creators of Dapper Open Source. And yeah, that is the main topic of today, Dapper. I'm also one of the Dapper community managers, Microsoft MVP, and I have a lot of like creative hobbies. A lot of them involve like pixel art. So there are some pixel art sprinkled throughout these sessions. I saw that the previous presenter used VS Code Pets. That's another extension in VS Code. I also contributed a lot of the VS Code Pets for that extension. So yeah, feel free to check out markduiker.dev if you want to see more pixel art or creative coding stuff or some music stuff. There are some interactive elements to the session and I'm using engage time for that. Again, that's all created by the same guy that created demo time. So please scan this QR code. You can ask questions asynchronously there. You can react to this content with emojis. You can participate in two polls that I'll show later. More importantly, you can actually access the resources that I'll be sharing at the end, and you can also give me feedback, which I would really appreciate. So please scan this. When there's time for the polls, I will show this again. So if you didn't catch it now, I'll share with you later. So just a warning. So this session is not about yet another agent framework. This is more about using existing tools and patterns and concepts to build reliable agentic systems. So you're still remaining in your seats, so that's good. Right, so let's make sure that we all understand what I mean when we talk about agentic systems. So for me an agentic system means that they use LLMs to make decisions and take actions. So and you always have like a couple of key components. So there's like reasoning and planning involved that you instruct these LLMs to do. There's some memory involved, there's some state stores, some tools that you can use and some orchestration, right? Because you need to coordinate multiple steps. So there are some workflow elements in there. So everyone and their own goal is like building agent frameworks these days, right? So of course Microsoft has a Microsoft agentic framework, there's OpenAI agents, there's LandGraph, CrewAI, and of course I have to list Dapper agents because that's part of the Dapper project and I'm usually promoting anything related to Dapper. So if you're a Python developer and you want to use... agent framework, have a look at Dapper agents. But again, that's not the topic of today, right? I mean, if you want to use like new shiny toys, please go ahead. But the question is, do you really need an agent framework to build agentic systems? And I don't think you need to. I think we have a lot of tools and things already that we can use to build these type of systems. So But you need to realize, and I think probably you know, that agentic systems are typically distributed applications. They are distributed systems, so they involve communication between different services, in this case, LLMs or agents, and they communicate either synchronously or asynchronously via message brokers, and they need to store their data in data stores. So there's not that much new things going on. Of course, they have some specifics, but the familiar challenges from the distributed systems are also problems with the agentic systems. And we've been dealing with those type of systems for decades really. So we can actually reuse a lot of the tools that we know. All right, so the first thing that I want to cover is like durable execution. Maybe you haven't heard of the term, but you probably definitely know some tools. So what is durable execution? So that enables code to run to completion, even if the process that runs the code fails. a new process will be started and it will pick up the code execution where it left off. And it's all made possible because the state has automatically persisted in a state store. There's a replay mechanism and that state will be rehydrated from the state store and then your code will continue to run. So it's super powerful and it's being used by a lot of workflow engines. So why is it important for AI agents? Well, For instance, you can resume from your last checkpoint instead of rerunning your whole workflow and executing all these LLM calls again. So you can save some costs there and also some latency. There's very predictable behavior when you use your execution, when you use these workflow engines, and they typically have like built-in resiliency and built-in observability, which is of course things you really like. Which brings us to Dapper. And before I explain what Dapper is, I first want to know what you think that Dapper is or how familiar are you with Dapper? So again, you can use the QR code or the web page that you probably already have opened. If not, you can scan this QR code again. I ask this question at every conference, every session that I give, and there's always like a very big variation. Some people are completely new to Dapper. Some people have heard about it, but never used it. And some people are running it in production. So this is like a good data point for me to capture over the years. So and sometimes people think it's the Orem from Stack Overflow, you know, but as far as I see no one is voting for it yet. So okay, some people completely new, okay, some people have heard of it but never used it and one is using it in production. Okay, well, who's using it in production? Okay, okay. Thank you, Alex. All right, I know everyone by my name who is using it in production, so cool. All right, okay, good to know, thank you. So Dapper is the distributed application runtime. It's a project that's about six years old now and it's specifically designed to help developers build distributed applications. It's part of the Cloud Native Computing Foundation. It's a graduated project since last November, which means it's a very mature project. A lot of big enterprises depend on it. And basically it offers all kinds of APIs to build microservices and these days also agentic AI systems. So I'll do like a very quick explainer about Dapper. So Dapper runs as a sidecar as a separate process next to your application. So you can basically use any language as long as you can communicate over HTTP or gRPC with the sidecar. And Dapper exposes a lot of APIs that you can use in your application. So if you want to do like async messaging or you want to do save some state, there's also like a built-in actor model. So it offers a lot of API's that make makes it really easy to build micro services or agentic AI In this talk however, I will only focus on the workflow because there's a built-in workflow engine in this in a sidecar and the conversation API which is relatively new API, but it's like an abstraction over different LLMs The nice thing about Dapper in general is that it decouples your application layer from your infrastructure layer. And I'll give an example about that later, but it makes it really, really flexible to use. Okay, next up workflow engines. And again, I have a poll for you if you are using or ever have used a workflow engine. So I've used like for instance, Azure Drupal functions quite a while before I started using Dapper. I've used Dapper workflow. There's also like a very old one, which is still used also by Microsoft. I'm not sure if it's in the list still. Oh yeah, BizTalk in it, yeah. I think BizTalk is still running. I mean, it's like 20 years old or something. That's amazing. Temporal is quite a well-known one. I think it's not so much a big thing in the .NET world. I think it's mostly Java, I think. But I think it has a .NET client. Yes, something else. Okay, who's using something else as a workflow? Custom solution on top of podcasting. Ah, okay, yeah, okay, yeah. I think a lot of organizations have like a custom solutions for these kind of things. So yeah, but thanks for sharing, nice. All right, and for all the people, they've never used it. Okay, that's also good to know, thank you. So Dapper has a built-in workflow engine in... the sidecar and it's inspired by the Drupal task framework. So if you're familiar with Azure Drupal functions, then you know what the Drupal task framework is. It's used in many places in Azure. You write your workflows in code. So you can use .NET, but also Java, Python, JavaScript. And these workflows are stateful and they're persisted to state store and the workflows should be deterministic and had it between brackets should be because it's up to you. The developer, if they actually are deterministic, it depends what kind of code you write in these workflows, but they should be deterministic. The smallest building blocks of these workflows are activities. And inside these activities, you can use any code, what you like. So those can be like non-deterministic code. So it typically means you call out to another service. In this case, you call out to an LLM provider or you save some state in a data store, et cetera. So just some some small examples on how it looks like when you use a Dapper workflow before I show the code which is a bit more involved. So if you want to start a workflow, here's like a post endpoint that uses the Dapper workflow client and that has a method called schedule new workflow async. You pass in the name of the workflow and the input. And what you get back is not the result of the workflow, but you get back an instance ID. So for every workflow you start, you get that unique ID for running that workflow. And that's because workflows are completely asynchronous, right? They can be running for days or weeks or months or years or even indefinitely. So you never get back the result of a workflow, but more like a key that you can use for, then you can query the state of this workflow. So here a very basic example of an actual workflow. So you inherit from a base type, you specify the input and output, and then you need to override this run async method. And then you use this built-in workflow context to do all kinds of operations within the workflow. So you can call different activities, the smallest building blocks of a workflow. Again, you provide a name and an input. And what you get back is then the result of... of an activity and you can pass that result again as an input for yet another activity. So this is an example of chaining of different activities. And finally an example of a workflow activity. So again it's used the same structure. So you inherit from a base type, you specify the input and output type, and you need to override this run async method. And here you can use whatever code you want. So in this case I'm calling out to an LLM, but you can use anything here. Right, okay now Some details on how this actually works a bit bit under the covers. So on the left hand side, we have your application Which uses the workflow Debra workflow client to actually schedule a workflow You have your workflow class definition and you have like several activity classes typically Then this is the Debra sidecar. So your workflow client will be communicating with the Debra workflow and the Debra workflow is communicating with both your app but also to a stage store So what happens when you use the Dapper workflow client, it sends a signal to the Dapper workflow engine to schedule a new workflow. The workflow will actually store some information about this workflow and the input to the state store. It returns an instance ID that you can use later on to get some more information about the workflow itself. Then the workflow engine will send some signal back to your application to actually execute your workflow because your application will actually run the workflow code. And then for each activity in your application, a signal will be sent back to the workflow engine to schedule these activities. And then if the activity is not yet executed, the workflow engine will send a signal back to your application to actually run the activity. The result will be returned to the workflow engine. That result will be persisted again to the state store. And what then happens is the activity result also goes back into your workflow. What happens in this, the workflow will not continue straight to the next activity, but the workflow actually replay from the top. So what happens then is for activities which are already executed, the state of that activity will be returned from the state store straight into the workflow. So activities will not be executed twice because the workflow engine knows that activities already have been executed. So this happens in a loop. So that's a replay functionality and finally the workflow is complete That signal is sent to the workflow engine and the workflow engine will persist the final workflow result in the state store. So there's a lot of A lot of communication going on between your application and the Debra sidecar the workflow engine and the state store So it's important to realize that but yeah because of all this communication and all the checkpointing It is a very reliable system because if you have any if it crashes anywhere, the state is always kept in the state store. So yeah Workflows, I think, really are the GOAT. They're really super reliable. They've been used for like decades already. So yeah, if you're not familiar with them, I would definitely say, okay, invest a bit of time in how workflows are working. I created some learnings on Debra University about workflows. I think it's good information to know regardless if you used for Agenda or not. Okay, moving on to the next Debra API that we're using, this is the Conversation API. So the conversation API is basically like an abstraction across many different LLM providers. So from our application, you can call to a conversation endpoints or you use one of the client SDKs. I will show you the .NET SDK and you can call to any of the major LLM providers. So this is a bit of a sample code, how it looks like. So you use a Dapper conversation client in this case and use the converse async method. So it's a bit verbose, but you basically have construct an input there that consists of an assistant message and a user message with some content. You can also add roles and names, et cetera. And also some options, which actually gives you back your response. So I will show some more elaborate examples later. But this code, it doesn't show what's the underlying LAM provider, right? Because that's the whole thing about DAPR. It's an abstraction on top of other things. So you actually can't tell what is the actual underlying LLM here. So the only thing that we see here is that we are using a component label or component name called myConversation. And Dapper works with component definitions in YAML files. And that's this thing. So this name here is the same thing that we saw in the source code in the previous slide. And this points to a conversation.openAI component. So if you would run this, it would use OpenAI to actually do the prediction. All right, moving on to actually combining workflows and LLMs because I think now it gets interesting. So I think combining them is really beneficial because workflow engines have been around for decades. They're super reliable. They provide like a good deterministic control over the activities. and they're also like very structured error handling and retries and management of their life cycle. So if you combine it with the flexibility of the LLMs and the reasoning capabilities, I think you have a very good solution then, which yeah, for I think many cases is already enough. You don't really need one of these agentic AI frameworks. So this talk is mostly based on a blog post I found on the Anthropic Engineering blog. So it mentions, we've worked with dozens of teams building LLM agents across industries. Consistently, the most successful implementations use simple, composable patterns rather than complex frameworks. So it's really about simplicity and knowing your tools and then just trying out the shiny new toy and just hoping that it works. And so the blog post mentions a couple of patterns. So it's prompt chaining, routing, parallelization, orchestrator workers, evaluator optimizer, and autonomous agent. And I've got demos for all these patterns. And I'm not sure if we can cover all of them, but I think we can do most of them. So as you can see, these are all the projects. I'm pretty big fan of Star Trek. So they're all in a Star Trek theme. So you can expect many more Star Trek GIFs. I grew up with the next generation. So I'll be running everything locally. I've got Dapper installed, Dapper CLI. I also have Docker installed. If you install Dapper CLI, you get some Dapper services running in some containers. You also get a Dapper, you can also get a Redis container for the state store. I'm also using the DiGrid Dev dashboard. That's probably something you've never heard of, but I'll show that a lot. That's very useful if you're developing Dapper workflows. all my applications or .NET 9 web applications and I'm running a local Ollama server with a Lama 3.2 model. So actually not even a large language model, it's pretty small. Okay, let me start all of my tools. So first the Ollama server, start my model. All right, and prove that the model is running. Yeah, all right. Start the Diabrid dashboard. I'll show that to you soon how it actually looks like. All right. And for all of the demos I've prepared, I'll be using the same configuration. So I'm using this YAML component that's using OLAMA. All right. So let me go to the first example, and that's the prompt chaining one. So it's also the most basic, but also one of the most reliable ones. So it's basically like sequential LLM calls where the output of one call is the input for the next one. So if I, if you visualize that it's like this, so this is my input for my workflow. I call an LLM. You typically have some kind of a validation or quality gate. If it passes, you pass your, for your next LLM call, again, you have a quality gate and so on. As you can, you can make this as big or small as you want. So when would you use this? Well, when you have like a big task that can easily decompose into smaller tasks, but like a fixed number of sub tasks. So that's, that's important. So yeah, you can use it for like multi-step analysis or like content generation or a translation with some refinement and evaluation as long as the number of subtasks are actually fixed and you all know them. So some pseudo code if you want to do this with DEPA workflow would look like this. So in case of a translation, we would first call a translate activity when we make an LLAM call, we get back a translation in this step and we use this as an input for The next one, the refined translation, and we use that output as the input for yet another one to evaluate it. So for the demo, it's called the spatial anomaly analysis system. I just realized it's actually a SAS now. So it analyzes spatial anomalies detected by the enterprise sensors through a sequential five-stage process. Well, if you look at the diagram, it looks like this. So we have an input with some spatial anomaly data. We first use an LLM to actually process this data. So this is the first quality gate. If it's valid, then we continue. We classify the anomaly, again, using an LLM call. We do some scientific analysis, get another LLM call, and we do a risk assessment. Again, an LLM call. Then another gate, so is the risk critical? If so, we have like an optional activity to alert the bridge. And finally, we always do this generate recommendation activity as well. All right, starting the project now. So let me show you the code. So this is the program CS file where we do the registration of all of our services. Since I'm also using the Dapper state API here, I'm also adding the Dapper client, but for the demo, it's not really, important. The ones that are important is the Depo Conversation client. We definitely are using that in most of our activities. I'm specifying a bit of a longer timeout here. It's not really that important for this demo because it's pretty quick. But of course if you do like a lot of like if you have a very large context and needs to do a lot of reasoning a lot of thinking and then you might run into the 30 second timeout with which the default so it's useful to know that you can actually specify a timeout on this conversation client so you don't run into that if you're using Dapper workflow you need to use this Dapper workflow extension method to be able to use the client in addition you also need to register your workflows and all your activities so For the people familiar with Durable functions, you don't need to do that in Durable functions. You don't need to register all of your individual types, but you need to do this with Dapr. Otherwise the Dapr sidecar doesn't know that your application has all of these types. So what I have here to start this analysis workflow is I've got a normally analyzed endpoint here. The Dapr workflow client is injected and also the Dapr client is injected. The Dapper client is used to save a bit of state, but we skip over that. It's not really what's important now. But we use the workflow client to schedule a new workflow, and that's the anomaly analysis workflow. In this case, I have created an instance ID up front. So if you don't provide one, Dapper workflow will generate one for you and you get back. But you can also create one yourself as long as you make sure that it's unique. So we scheduled here and we give back the instance ID so we can... use it later. If you have a look at the workflow, so this is the anomaly analysis workflow here. I first create a retry policy for my activities. This is optional, so by default, your workflows are already like persisted. So if the whole application crashes, your data is safe and you can spin up a new process and then the workflow will continue. But what if an activity fails, right? So if you don't configure a retry policy for activities. If an activity fails, then also your workflow will fail and that might not be desirable, right? So you can optionally also configure these retry policies. So whenever I am calling an activity, I will always use these default retry policies. So the activity will be retried in case it initially fails. And since I'm making a lot of LLM calls, which sometimes have a bit of like unpredictable behavior and doesn't return the exact things that I want, it's actually useful to have this retry policy. So this is the entry point of the actual workflows to run a sync methods. We have this workflow context and we have the input. So this is the first activity, the process sensor data activity. We use the raw central data from this input. We also specify the retry policy and we get back this process data. All right. So we use this process data first in this quality data. So do we actually have any process data from this first activity? If you do, then we can continue and we use it in our classified anomaly activity. We get back an anomaly type and then we use that as an input for the scientific analysis activity. So then we get an analysis back and we use that analysis for the risk assessment, which is a combination of both the anomaly type and the scientific analysis. So then we have a risk level then we check if the risk level contains some specific string that we're interested in so if it's critical then we do this optional activity and finally we do this generate recommendation activity so as you can see whatever happens in this workflow is mostly calling out to other activities and some if else statements so that's like always And so it's always deterministic. So what if you need to do in a workflow is never use like for instance, like random data. So don't create GUIDs, don't depend on date times, don't make direct calls to other services or to databases, put those things in activities. And so it also only contains like if else statements, switches, those kinds of things. All right, so this is the first activity that we're calling. I will only show this first one because all of them are structured in the same way. We inherit from this workflow activity. We have a string input and a string output. And this one we are using the Dapper conversation client. So that gets injected here. So we override this one, an async method. And here we are specifying the conversation options where we mentioned this conversation component name. So this is also the name that's in this YAML file. So Dapper will figure out that we are using Olama. Here we are using Conversation Client with the ConverseAsync method and here is what is actually being sent. So we have a system message, you are Lieutenant Commander's data, sensor analysis subroutine, et cetera, et cetera. And then the user message, which I process this sensor data and this is getting fed in with the activity. And this is how you get data out of the response. It's a bit convoluted at the moment. I think there will be some help methods to make it a bit easier to read. But at the moment, this is it. So let's start the workflow. I'm yet again using another VS Code extension called the REST client. I have this HTTP file and then you can create all of these requests here and I can actually execute them here straight in VS Code. So this is the payload that I'll be sending to the anomaly slash analyze endpoint. So let's execute this and then I'll jump over to the DIGREDEV dashboard. So we make this request. It's accepted. So now the workflow is running. So this is the Diagrid Dev dashboard and it's very nice because it visualizes your workflow. So you can see like the state of the workflow, the ID, what type it is, when it was started, when it was completed. Well, it's not completed yet. And you can drill down into this workflow. So in this case, it's completed because I already see an output. So this is ideal for like inspecting the workflow state, especially if things are going wrong. You can very easily like expand on these nodes and just to see, okay, what was the input for this scientific analysis activity and what was the output for this activity. So this shows the entire execution history. So if I scroll to the bottom, so it's in reversed order. So what happens first is at the bottom, we can see that the execution is started. Then we see the first execution of the activity, the process sensor data activity. which then moves on to the classify, activity, scientific analysis, and so on and so on. So yeah, if you're dealing with that workflow, this is a very nice thing to use. Right, it still says running here, but it's not. Yep, right, it's done. And it took like 17 seconds, that's pretty okay. So let's actually have a look at the result now. So what's the result? If we scan, look down here, this is all of the individual stages. So the anomaly type is a wormhole, so we have some scientific analysis here. formation theories, we have a risk level high, and then tactical recommendation. I think science fiction is also a great use case for LLMs, you know, because it doesn't really matter what the LLM comes up with. It's always like, okay, sure. Yeah, this is fine. Yeah. All right. Let's stop this demo. Right. So prompt chaining, it's really quite easy to understand how the flow of this workflow, it's also quite accurate, right? Because you can split up a task into subtask and you can get really like specific prompts for these specific tasks so I think that works really well and typically also add these validation gates so yeah you have a lot of control when you use this prompt sharing pattern. If you compare it to just making one call which does everything of course yeah you do like subtasks so you have multiple calls I mean higher latency because you make multiple calls and also means like yeah you probably have like increased costs as well because you make multiple calls but I think it's worth it because I think the value. the quality of what comes out of it is better. Okay, the next example is routing. So there you classify input and based on the classification, you route the request to a specific handler. So that looks like this. You first have an LLM to do the classification and then based on the classification, you send it to a dedicated handler and of course... this can be like, you can use like different LLMs here as well, right? So for one, you can use like maybe a large language model, for the other hand, you can use like a small model, or for the other hand, you can do like no LLM at all, right? Maybe you can use some old fashioned coding. So again, it's quite flexible. When you would use this, well, when really distinct categories are better handled, like separately, instead of just one thing that handles everything. a very common use case where this is like customer service routing, right? Different areas for customer service. So here's some pseudocode of how you would do this in DEPRA workflow. So first you would have the classify activity, right? To do the classification itself. And then you have the output of that. And then you can use like basically like an if else statement or like a switch statement with all of the options. So it's really, really quite straightforward how to do this. So the demo I have is called the Galactic Anomaly Classifier. So we classify and route different types of space anomalies to specialized analysis activities. So, okay, so the diagrams are getting gradually bigger as well over time. So we first do the LLM call to classify the anomaly and then based on the anomaly type, we have like different headers. So there's one for temporal rift, there's one for dark matter, alien artifact, stellar phenomena and dimensional tier. but only one of them will be called, right? So that's important to realize. All right, let's start this. And we only have a look at the workflow now. We don't have to go through the whole program CS file and all the activities, but I think it's nice to look at the workflow. So what we have here is we have the classify anomaly activity, which gives us back the classification, right? So we get back the type of anomaly from this classification. What we have is like a basic switch statement. It just checks for, okay, are you a temporal rift anomaly? If so, just run the analyze temporal rift activity for this. And we get back the result and we just extract the right fields from this result. And the same for all of the other ones. There's no real magic. It's quite easy to understand. What is very important though, is that you, of course, always use a default because the classification can result in something. that you didn't think of at first. But you always need a handler to handle the edge cases. So you'll run into this quite quickly if you don't. I'll skip the activity because it's the same setup as we saw before. Okay, so let's do this temporal rift request. So we actually already know the answer. It should end up being posted as a temporal rift. Let's call this endpoint and let's see what is happening. Okay, so... classification workflow is running. So let's scroll down here so we can see that the workflow is started. The classification is running. So this doesn't live update. So I have to hit refresh a couple of times. Oh, I just promoted someone else's book. It was called Hit Refresh, right? Satya Nadella's book. Yeah. Yeah. Okay. So, and yeah, the classification is done here and we see that the the analyzed temporal rift activity is indeed called, which as it should, so it's good. And here the workflow is indeed completed. So if we inspect the output, we can indeed see that there's a temporal rift classification. This is the result of the specialized analysis. And then the recommendations here. Deploy advanced chronal dampeners to mitigate causality loop effects and protect against tachyone radiation. Brilliant. Yeah. LLMs are great for this. Okay. Right. So let's have a look at the pros and cons for this. Yes. So the process you have your specialized handlers per category, right? Which again, it improves the accuracy because you're not doing like a catch all thing, but you can be very specific for these things. So you can use it as well for like a cost optimization, right? Like I mentioned before, maybe for some handlers you want to use like an expensive model for some other categories, you can use like a very simple model or for some other things you don't need like an LLM interpretation at all. Of course, it's all very dependent on the actual classification, right? So the most important thing is to have the first activity that you're calling. So that's really important. What's also good to realize is if I would really use this in practice, I would route to a completely new workflow. So what you can do with Dapper workflow, but also other workflow engines is instead of calling an activity, you can call a child workflow because a child workflow can consist of individual activities again it's very likely that you don't have like one function to handle something but it again is like a chain of functions so i think that's the most likely thing that you would do if you want to go this routing area all right the next one parallelization so here you can execute independent calls concurrently so let's have a look at this so you have your input you break that down into separate pieces and you can execute all of these subtasks in in one go And the nice thing about it is that the workflow will then wait until all of these tasks have been completed. You can then aggregate the results and give it back as an output. So this exists, it's like two variations. So one is called like sectioning and they actually break that one thing down into smaller tasks. You can also do voting and then you can actually execute the same task, but then with different inputs, or maybe even with the same input and then compare all of the results to see if you reach some kind of consensus. So use cases, multi-source analysis or some kind of a code review step. So here's the pseudocode for doing this. So it's now quite a bit different. So what you see here is that we prepare a list of tasks of result and the result is actually the output of all the activities that you're calling. So it's very important that you can only do this when you, if the output type of the activities is the same for all of them. So what happens here is we create a list of all of these activity calls, but you notice that we are not awaiting these individual calls, right? We just are building up a list of tasks that you want to execute later. And that's what's happening here. Here we do await task when all and then providing all the tasks there. So at that moment, the Dapper workflow engine will actually schedule all these calls and will actually wait until they've all completed. And then the final step is you probably want to do some kind of aggregation or summary over it. All right, so the demo for this is the Starship Diagnostics one. So it performs like a diagnostics on Starships and it runs like independent scans simultaneously. And then in case of critical findings, it also does a round of faulting on different aspects. So this is how it looks like. So the inputs are some Starship criteria or some metrics. Then we start all of these. So we do like a whole integrity scan, reactor core scan, navigation, life support and weapons. We combine that in a result. If any of those are deemed critical, then we do a round of voting and then we have different voting activities for different aspects. So we have an aspect for like safety voter, a severity voter and a recommendation voter. So it means like a cost benefit analysis. So when those are completed, we also combine those and then we combine both sets so both all of the analysis results and the voting results so this is quite a bit more more involved so let's run this and have a look at the code so the parallel diagnostics workflow so here we go the run async method so what comes in is a starship input so here you can see that we are preparing that list of task scan results right so we are building up this list of all of the activity calls to different activities and here is the place where they are all scheduled and the workflow will wait for the execution until all of these activities have been completed then we check if any of them have critical findings and if they do then we loop over these critical findings and then we do the same thing but now for the voting so we have these three different voting activities we also execute them then we aggregate those votes and findings out of it And then we aggregate both the scan results and the votes and then we create a final report in this aggregate results activity. All right, I'm skipping over the activity. Okay, let's make this request and we will do like an analysis on the user's enterprise. But this telemetry and it should have like multiple issues. So let's see what it comes back with. All right, it's running. Let's scroll down. So as soon as it started, you already see that all of the All of the specialized things are scheduled at the same time. So if you now refresh, some of them are completed. So the life support scan is completed. The weapon scan is completed. You can actually inspect already what it says for the weapon scan. Replace power coupling in phaser array 3, of course. Yes, okay. All right, and at this moment it's done, right? So what it did, it completed all of these individual activities. I don't see... the recommendation voter activity so yeah so it did found a critical one because otherwise these these voter activities wouldn't have triggered so it did find a critical one and here we did the aggregation of everything so let's let's expand this and if I have a look if we scroll down here there's a little output okay so here are the critical votes so the critical thing was the whole integrity apparently they were very unanimous it needs urgent repair Yeah, yeah, you're correct. Yeah, it could be even more in parallel as you mentioned. Yeah, true. Yeah. In this case, yeah. So yeah, that's also a good use case to start these child workflows then with their separate sections. So yeah, in this case, I'm keeping it sort of simple in one workflow, but that would be a good use case of putting it in the child workflow. So yeah, pros are need high throughput, can be even higher indeed, but also high confidence if you use this voting aspect. The cons is you might not be able to use it because you can only use it when tasks can really be subdivided and be really independent. So if you don't have independent tasks, you cannot use this approach. All right, next one is orchestrator workers. So now it gets a bit more dynamic. So with orchestrator workers, you first have an orchestrator LLM that defines what needs to be done. And then it decides, okay, here are the workers that need to do some work. So the task comes in that will be broken down in some smaller tasks. But for one request, it might only need this worker, but for other requests, it might need these two workers. And for yet another request, it might be all of the workers. You don't know upfront, but things need to be decomposed and handled by workers. But it might depend from case to case which one you need. So at the end, of course, we gather back all the results from these workers because you never know which one will be used. And finally, we do some summary again. So when would you use this? Well, we need some kind of dynamic decomposition of tasks. So depending on the request that you get, you might need some kind of different decomposition. So it's usually for more complex, slightly unpredictable task structures. And not completely unpredictable because you still need to write all of your workers, right? I mean, it can be completely new. You still need to think of these things, but at least you have like more variations in it. And so you can use it for coding agents to handle different types of files or different types of languages. Also research tasks, which can be quite different or complex planning. And that's actually the topic that we're gonna do. So some to the code, how to do this in Debra workflow is you would have like a decompositive T where you use an LLM to give you instructions on what you want to do. And here you could iterate over these instructions and then based on the instructions you have like workers in this case it's this is not very realistic but now the same worker will be started for all the instructions what will happen is that based on the instruction you will have like a different worker and and execute that worker you wait until all of them have been completed and then you do some consolidation on the output So what we have here is the demo is the space colony planner. It's a good example of like a more complex planning tool. So it does dynamic space colony construction planning. So what we do first is there will be like an analysis of a planet's conditions. Based on that, there will be different construction work that you want to do on the planet, which requires specialized workers. So the end result is like a colony construction plan. And by the way, this is the only gift that's not the next generation. This is from the very first original Star Trek. All right. So again, our diagram grows bigger. So the thing I hope is still visible for the people at the end. What you first have is like an activity that analyzes the planet, then an activity that determines what kind of structures will be required on the planet. And then there's like a dynamic routing happening. So in case we need habitats, then we have a handler for that. If we need a power plant, we have a handler for that. Agriculture, mining, research and defense system, right? So it really depends on the planet, what kind of structure we need and which other workers or handlers we need. And the workflow will wait until... All of these which are required have been completed. Then we create a construction plan and then we do a refinement step yet again on the plan. Okay, running this. Okay, so let's have a look at this workflow now because it's quite a bit different. So first we have the sequential part analyzing the plan activities. So what we have is the plan analysis and we use that as an input for the determine structure activity. Then we have the structure requests. So in this case, we do like a for each of the structure requests. And here we have a switch where we just map the whatever key we find here to the corresponding activity. And so if the key finds, if you find habitat dome, then we want to run this activity and so on. And again, it's very important that you have like a default. switch as well, right? Because probably there will be some cases that some new or undefined structures will be found by, in this case, the LLM. So yeah, you always need to make sure that you have kind of an edge case handling for this. So that's why I introduced the unknown structure activity. So all of these tasks are added to the worker tasks. And here we are executing all of them. And so for the end result, we have this master plan. all the plans you go into that so then we have the first plan and then there's an optimization step which actually splits it out into different phases to help us sort of with the logical ordering of things i'm skipping over the activity all right so let's let's run this and this planning for uh for this planet with all this data let's go so this probably takes a bit a bit longer now let's have a look because at first needs to do two sequential things then it needs to do things in parallel and then we end up with a sequence of two things again. So now it's determining the required structures for this planet. I'll hit the refresh again. Okay, so here, so it found quite a bit of things. That's probably everything, right? Oh, it's interesting. So it also found something that I didn't expect. So we also have like an unknown structure activity. Interesting. Power plant, mining, yet another unknown. Interesting. Okay, so now it's synthesizing the plan. And then it was one more step for a refinement. I'm waiting for that one. See, no. Yes, okay, it's completed. I just have a quick look at what's here because I'm quite curious what kind of things. Okay, so apparently a shielding facility I did not think of. So that's like an unknown one. Okay. What else is unknown? Well, I think that's the only one. Okay. But it gives us a very nice estimate of... 2,550 construction days, excellent. A whole list of materials required. And then this is the refinement step. So it gives you actually a timeline with phases and what you need to do, et cetera. So of course, this is all like completely imaginary, but you can use it like this if you want to colonize your own planet. All right. So the probe of this orchestrated worker pattern is, well, it's very flexible and adaptable, right? I mean, I did not think of a shielding facility to add, but now I've seen it appear. So now I can add my solution with a shielding facility, you know, so it's also very extendable. So that's great. So it can really scale to more complex problems, right? And each handler can of course be very, very dedicated and focused on its own task. Again, if I would... make this for real, I wouldn't just do activities to handle this, but I would create like entire child workflows to handle these things. There's also high complexity, but I think it's also like quite manageable, right? I mean, it's still sort of overseable what it's doing, even though it's more dynamic now. And again, the most important part is the start, actually the orchestrator, which defines what the output is of this dynamic classification. Right, let's do one more, the evaluator optimizer. So this is like an iterative loop. to generate things and to evaluate and then do a refinement step in a circular way. So now the diagrams become a bit more basic, but we're actually looping it now. So when you start this, then you first check, okay, is this the first time running? If it's the first time you run this generator to do some kind of analysis or generation of something, and then you evaluate the output. If it meets your criteria, well, then you're done, but... typically that's not what happens. Something is not good enough yet, it doesn't meet your quality threshold. So then you continue, you always need some guardrails. So in this case, since it's iterating and looping, you need to specify some max iterations that you want to do, because otherwise this workflow will never complete. And you also need to pay a lot if you're not running it locally. So then you continue this iteration. And then we are not in our first iteration anymore so then we do a refinement step of this initial generation part so we do a small tweak and then we evaluate again and then we do the same thing so we evaluate do we need my criteria no max situations no okay let's do a loop again all right so this can continue if you don't meet your your your quality criteria until max situations is reached So when you use this, well, when you think there's value of doing small increments and these iterative refinements, you do need very clear exit criteria, right? So you need to be very specific on when is something good enough and also how far and how long do you want to continue with this. So you can use this for doing content quality improvements and also code generations with validation and linting. So here's some pseudocode where I first specify some quality threshold that I have, the max situations that I want to do. I call this generator activity, which gives me some output. I evaluate this thing that I've created. Then I check, is this quality good enough or have I reached my max situations? If so, I exit. If not, I do an optimization step and then I instruct the workflow to continue as new. So what's... important to realize in your workflow you never have really a main loop so you don't have a while loop or a do loop in your workflow. So if you want to loop your workflow you instruct it to continue as a new workflow and that means it actually it forgets its history. You give it some new input and it starts as a fresh instance. So in this case there's this alien translator that we're using. And we use it for like a translation refinement, right? So we do an initial translation, we evaluate, if it doesn't meet our criteria, we do a refinement of translation and then we just keep looping until we have either met our quality standard or we have the max situations. So I had a diagram, it's pretty similar to what I just shown you. We do a translation, we evaluate, if it's like, if the quality is okay, then we stop. If the quality is not okay, then we haven't reached our max situations, we continue as a new one. we do a refinement of a translation and then we just keep keep looping all right so let's start this so the workflow looks like this now we have like a max iterations so this is our guardrails we never do more than five iterations and we have our quality threshold and so the first thing is we do here is okay we first we we do a check okay yeah are we already in site and iteration if not we do the original one we do the translate activity one we have a translation as an output otherwise we called the refined translation one and then we do the evaluation on on the translation i'm also keeping some some checks and balances to also see the incremental changes here so that's why you see i'm appending things to two lists that's pretty optional so this is then the first exit criteria so if it's like a quality based thing if i met quality okay then fine return this output The other option is, okay, if I reach my max iterations, I didn't meet my quality criteria. So it's like a different kind of output. So that's why it says, okay, manual review still required. And otherwise I update my inputs with all the translations and evaluations that I already did. And then I restart the workflow with this continuous new method with this updated input. Okay. I'll skip over the activity. Let's run this. So we'll use this Vulcan diplomatic request that we're going to send. So let's send this and let's go back to this. So the translation is running. Now I've seen it go multiple ways with the same input. So sometimes it continues in just the first iteration already. And sometimes it takes like two or three times. So I'm very curious what happens now. So of course, now it's the first time. So this first time we call this translate activity call. And yeah, it's already completed. So already after the first evaluation, the quality is already high enough that it stopped. So it actually didn't even loop yet. Let me just do it once more to see if it actually triggers another one. If not, I'll just continue. I'm just running the same one now. Okay. I'm using the same ID. I shouldn't have done that. Okay. Let's move on to the next. Or actually, no. We don't have time for the final one though. Okay, let's go for the pros and cons and then I'll close. So, Evaluator Optimizer, I mean it's great if you want like small quality improvements. I couldn't demo that unfortunately, but still, the demo works. Of course, you can have like unpredictable iterations, right? So, in this case, I didn't actually... I was sort of keeping track of the score, but yeah, you can have like a lower evaluation on the next round. So in this case, it might actually be worthwhile to also keep the previous one and then figure out, okay, are we going in a good direction or a bad direction? Again, this is looping, so you will have like a higher cost and latency compared to just making one call and hoping for the best. Okay, I will not do the autonomous one. That's, let's say, the least deterministic, but yeah, I will share all the code so you can run this yourself if you want. Okay, final stuff. So I hope you realize now that agentic systems are distributed systems, right? And we can actually use all of our existing tools. I mean, if you want to use like shiny new toys, please go ahead. But you don't need to use the shiny new toys in order to build agentic systems. Please look into Drupal execution if you didn't do so already. I think workflow systems are really very, very powerful. I'm a big fan of Dapper workflow. Please, please try it out. Yeah, like the conversation API that Dapper offers because I can really easily switch from one LN provider to another LN provider without changing code. So it's convenient. And I think it's really useful if you understand these basic agentic patterns, right? Because it's very nice to just mix and match these patterns and see how far you can go without going like all in into like a really agentic framework. So in general just start simple and add complexity where it's needed. I think in general in IT I think that's always a good mantra to follow. You've seen this, I think this is very useful. This is just like a free download, runs in a container, super, super useful. This was something else if you really want to run a Dapper workflow in production. The Diogat has a tool called Catalyst. It's really like an enterprise platform for workflows and AI agents. It offers very nice visualizations of your workflow and inputs and outputs and more management of the workflow. So that's definitely a no-brainer if you're into that. And finally, please leave me some feedback. great decision and access all of the resources there. So there's links to Dapper University to learn more about Dapper and workflow. There's a link to the original Anthropic post that was the basic of this talk. Also links to like Catalyst and how to run and install the dashboard and also ways to connect with me. So thanks very much.

This talk was recorded at NDC London in London, England. #ndclondon  #ndcconferences #developer #softwaredeveloper    

Attend the next NDC conference near you: 
https://ndcconferences.com
https://ndclondon.com/

Subscribe to our YouTube channel and learn every day:   
/        @NDC 

Follow our Social Media!

https://www.facebook.com/ndcconferences
https://twitter.com/NDC_Conferences
https://www.instagram.com/ndc_conferences/

#dotnet #ai #microservices 

Everybody seems to be building agentic AI systems these days. But can these systems be put in production easily and work reliably under a heavy load? Agentic systems are essentially distributed applications, involving communication across LLM providers, services, and data stores. Luckily, we have been building distributed systems for decades, so let's apply this knowledge!

In this session I'll show how Dapr, the Distributed Application Runtime, a graduated CNCF project, helps to build and run agentic systems reliably using the durable execution principle that is provided by Dapr Workflow. I'll demonstrate various patterns in .NET for building effective agents such as Prompt Chaining, Routing, Parallelization, Orchestrator-Workers and Evaluator-Optimizer. I'll also use the new LLM Conversation API in Dapr to interact with different LLM providers.

By the end of the session, you'll have a good understanding of how to build agentic systems with Dapr, and when to apply the different agentic patterns.