Marc Duiker - Failure is not an option: durable execution + Dapr = 🚀

All right, welcome everyone. Thanks for joining my session. Failure is not an option. Duable Execution plus Dapper is rocket emoji. Yeah, when I wrote this title a long time ago, I didn't really know how I would actually pronounce this title. So I don't know. Duable Execution plus Dapper is amazing, I think. That closely reflects my thoughts. But maybe next time I won't use an emoji, but something else. But yeah, at least it got me through the call for papers selection. So that's good, I guess. I think we all know that failure is inevitable, right? I mean, we're all like software developers doing something in IT, and we probably see failing IT maybe every day. I don't know. It's every week it's in the news. Most of the time it appears to be DNS. But sometimes it can also be something very small. And maybe some of you are like as old as me and have seen this information message in the wild. Task failed successfully, an information message in Windows XP. Yeah. I've seen it. It's, of course, very silly to show this to your users, right? Task failed successfully. But that's actually the essence of this talk, right? Because things will fail. We know that. But yeah, if we have tasks that are failing, let's fail them in a successful way, which has the least impact for users as possible. So my name is Mark Duiker. I'm a developer advocate at Diagrid. It's a company founded by the co-creators of Dapper Open Source Project. And I'm also one of the Dapper community managers. And we do like every other week we do a... community call where we invite people who are using Dapper to share their story. And of course, when there's a new Dapper release, such as recently, we'll show you the new releases and new APIs in Dapper. And we also share all of community content. So if you're writing about Dapper or if you have made a Dapper video, we'll also highlight in those community calls. I also do a lot of creative stuff. If you want to know that, go to markduiker.dev. And as you can see, I'm using VS Code for my presentation. So everything I'll do today will be done in VS Code. And that's partially due to this extension. It's called Demo Time. And that allows me to do a guided tour through the session. So I think it's a bit 50-50 markdown slides and demos. But it can also execute all kinds of commands. So I will be running some .NET applications with Dapper, but everything is sort of scripted with demo time. So if you're interested in that, please, please look it up. It's made by Microsoft MVP Elio Strijf. It's a really cool extension for people like me who do a lot of presentations. So when I was researching IT failures, I came across this IEEE Spectrum blog post. It's already quite old, from 2015, so it's like 10 years ago already. But it's a really cool blog post. I definitely recommend you if you take a look at it later. And by the way, everything is in Git, and I will share the link with you at the end of the session via QR code, so no worries if you... cannot take a picture now or whatever. You'll get access to it later. But it's like an interactive blog post, because all of the diagrams in this blog post are interactive. So it's really cool to play with those sliders and see some data. took a screenshot of the first one because that highlights the amount of cost over a decade of IT failures. And if you see here the legend, the legend is like 10 billion in cost. And you can see here there are much larger circles here. So we're probably looking at like hundreds of billions of costs across the entire world from 2005 to 2015. I really hope they will create like a new of the latest 10 years as well. Let's see. One of the quotes there I found very interesting. So a rogue algorithm let the Knight Capital Group to lose 440 million in average trades in about 45 minutes. So can you imagine losing that much money in such a short time? That's really quite scary. So I'm not saying that the tips and tricks I'll be showing in this session will prevent this, because IT failures is more than technical failures, right? So there's also project management failures and all kinds of other stuff. I'll focus a bit more on some technical side and not so much on the process side. But yeah, there were some very interesting quotes in that paper. So in general, things are complicated, especially in larger organizations, because in larger... your organizations there are many many different development teams and when you have like a lot of development teams doing different stuff you usually end up with distributed applications such as this this is still a very small one but there are organizations who have like hundreds of these microservices development environments and this is very good tricky right because there's a lot of communication going on either synchronously between these services or asynchronously via message brokers a lot of different state stores are involved so this is really really a complicated matter And we know it's complicated because a lot of people have studied it and there's a list of the fallacies of distributed computing. It's already from 1994. Who's heard of this list, fallacies of distributed computing? Just a few hands. Okay, this is not a theoretical session, so I won't cover these. But you'll see there are like three links here at the bottom. I definitely, I would recommend you to start with the bottom one. That's a YouTube playlist where Udi Dehaan from Particular Software explains all of these fallacies in a quite, quite short video. definitely look into this it's very interesting and not only if you're building distributed systems but maybe general IT knowledge and it's really cool to see so have we noted yeah building distributed systems is very hard we've known it for many many decades and that's but sometimes said that there are groups of people who get together and create very clever solutions and that happened about five years ago in Microsoft and a team got together and created an open source project called Dapper the distributed application runtime so it originated in Microsoft research but then after a couple of years and it went into the CNCF foundation so it means that Microsoft is still part of it because Microsoft is part of the steering and technical committee of the DAPR project but it means that there is no single company owning DAPR because it's in the cloud native computing foundation But the whole purpose of this open source project is to make it very easy for developers, but also for people in operations, to build and run distributed applications at scale. And why is that? Well, if you develop and run with Dapper, Dapper runs as a sidecar next to your application. there's a lot of responsibility that you can actually hand over to that sidecar. So there's things that's like resiliency is built in, there's security built in, there's observability built in. And Dapper also offers a lot of different APIs that you can see here. And well, this is not a Dapper one-on-one session, so we're not going to cover all these APIs. In this case, we're going a bit deep into workflow. I'll also show you a bit of certification and a bit of state management as well. So as you can see, the Dapper sidecar offers a lot of different APIs. But all of the APIs are an abstraction over the underlying infrastructure. So for instance, when we look at state management, so Dapper has a key value pair API. But Dapper itself is not a state store. But you can configure dozens of different state stores with Dapper. So the only thing that Dapper offers is it offers you an API that you can use as a developer to save some state or to get some state or to delete some state. And then you can configure what kind of state you use with a YAML file. And I'll show you that later, how you can configure that. So you're really flexible when you're using Debra. And as a developer, you only need to learn one API, and it's the Debra API. And then you can use whatever stage store or whatever matches broker or whatever secret store, whatever you want. So it's a really, really very powerful solution. Again, I'm not explaining Debra completely. If you want to know more about Debra, if you're really new to Debra, I can recommend you to. to this online course it's called Dapper University there's one lesson now Dapper 101 that explains well the principles behind Dapper and it explains about certification and state management and pop sub messaging and I'm now working on a Dapper workflow course but I'll give you the most of it in in this session Alright, so what kind of problem are we solving here? Well, we know that systems fail, so we need to make sure that these systems can recover from failure, ideally automatically, and also limit the impact of the failure, right? Because we want to fail successfully, right? So this is where the durable execution principle comes in. What does it mean to execution? Well, it's running code in a stateful way. So that sounds a bit weird, but it will make sense later. So in case the process that runs the code, if that crashes, another process can come up, and it will continue running the code until it runs to completion. So if that sounds familiar, maybe you've heard of workflow engines. So who have used a workflow engine here? Maybe like logic apps or Azure Durable Functions, maybe? Yeah, saw some hands there. OK, yeah. So if you're familiar with especially Azure Drupal functions with that API, you'll also be very familiar with the Depo workflow API because it's built by the same person. So what you have in a workflow engine, well, first you need to, of course, you need to author your workflow. So either that looks like a visual designer or you have workflow as code. So you have some kind of process that has your workflow. Then there's a separate workflow engine. So that takes care of scheduling all of the tasks in your workflow. And you usually have a state store which captures the state of your workflow. So I've made a small animation to see what's going on there for most of the workflow systems. So what happens when you schedule your workflow? Well, first, all the information, so the inputs will be stored in a data store. And then the workflow will continue with scheduling the first activity. And that activity is maybe calling something to a database. Once that activity is completed, also the output is stored. Then the workflow will replay. from the top. So it will not immediately continue to activity two. No, the workflow will completely replay. It will first check activity one. It already executed that, so it will retrieve that information from activity one from the state store. Then it will continue to activity two, and it will do the same thing. It will execute, put everything in state store, and then it will replay again until all of the activities have been completed. And then it will do a final run through through all of the steps. And it recognizes that everything is really executed in the state store. So it will retrieve that. And then the workflow is completed. And also, that is put into the state store. So as you can see, there's a lot of I.O. going on between the workflow engine and the state store. And then this is how these job execution engines work. So in case something goes wrong with the process that's running the workflow, that's OK, because all of the state is persisted in the state store. And so the next time a process starts up, Again, it just reads all of the events from the stage to work in. So this is important to realize if you start working with these workflow solutions. So some of the workflow solutions, including Gapworkflow, are based on workflow as code. So the people who have been familiar with Drupal functions and know this principle as well. And I really like it. I think most of the developers prefer it to visual systems because, you know, What I like about it is it's part of your code. It's part of version control. So hopefully, someone, when you make a pull request, someone also checks what you've written. You can also write unit tests for your workflow, which is great, because our workflows are essentially your business processes. So it's great if you can fully unit test that. And there are quite some workflow as code solutions available. So we already mentioned Drupal functions. Dapper workflow is also based on code. There's also Temporal, which is quite big, not really in the .NET world. dozens and dozens more. So if you're interested in these workflow as code solutions, there's an awesome list on GitHub called Awesome Durable Executions. And you can check that GitHub repo. For every language, there are really dozens and dozens of options. OK, I'll quickly share some of the patterns that you can use when you are developing workflows. So the most basic one is task chaining. So here the order of execution is important. And so maybe the output of activity A is used as an input for activity B and so on. So that's why you have to execute it in a certain sequence. So this is the most basic one. Then fan out, fan in. So here the order of execution is not important at all. So the workflow engine will try to just schedule it and run it. as much as possible in parallel. And then you can instruct the workflow to either wait until all the activities are done or when the first one has been done. So that's totally up to you. But it's a very nice thing that you can actually wait in the workflow until everything is done because then you can still aggregate over all the outputs of these individual activities. You can still aggregate over it. The monitor pattern that's typically used when you want to do some kind of a task on a regular basis. So for instance, maybe you want to do like a nightly cleanup job of some cloud resources that you want to get rid of. And so you can use that first activity to make your call to Azure and remove some resources. And then you use a specific timer in a workflow to wait for another 24 hours, for instance. And once the time has passed, you instruct the workflow to continue as a new instance. Now, this is something different than the replay, which I've shown you earlier, because when you do a workflow replay, you don't actually do it. The runtime does it for you. But every time the workflow replays, all of the historical information is kept. But when you instruct the workflow to continue as a new instance, it just forgets all of the previous executions. So this is basically doing a do-while loop, but then in a workflow fashion. The final pattern I want to mention is the external system interaction, also known as the human interaction pattern. It's typically used for when you need a human in the loop, you need some kind of approval. So let's say you want to order an expensive item and you need manager approval for that. So in one of these first activities, you can send a message to another system that will alert your manager that you want to buy something expensive. And they use that system. They either approve or disapprove. They send a message back into the workflow, and that workflow is waiting until they receive that message. Once that message is received, it will inspect the payload. And based on a payload, you can make a decision in your workflow. Is it approved? OK, go to this branch. If it's not approved, you can use another branch in your workflow. And what typically happens when you are developing these workflows, you usually combine different elements, the different patterns. You never just use one of these patterns. you usually combine them. And the nice thing about Dapper is with Dapper, you have all of these APIs available. And you can very nicely use all of these other APIs in these activities, because it's very important to realize that your workflow code will mostly contain some business logic, so some if-else statements, and chaining or fan in and out patterns. But all of the other stuff that you do should be done in Dapper. in activities. And so when you are communicating with another endpoint or with a database, that should all be done in activities. So DAPR is really nice because you can use all of the DAPR APIs in all of those activities. OK, so talking a bit about the DAPR workflow engine. So here on the far left, we have the workflow app. So this is the workflow that contains your workflow as code definition and also your activity code. And we will go into a demo in a couple of minutes. And your workflow application will communicate with the Debra sidecar. And the Debra sidecar contains the workflow engine. So your workflow is, of course, executing the workflow and also executing running all the activities. But the scheduling of the workflow and when these activities are being run, that's all being handled by the workflow engine. And, of course, the workflow engine is communicating with the state store because it needs to persist all of the in and outputs of all of the activities. All right, so the first demo I'm going to run is a small demo with a workflow application and a shipping service that's next to it. And the workflow application is also interacting with an inventory. So this is the visual representation of the workflow. So we will start here. It's called a validate order workflow, so we're going to validate an order. And what we'll do first is we'll do a check with the inventory. system to see if we have enough stock available to actually complete this order. So if there's no sufficient inventory, we'll immediately go to the end. If you do have sufficient inventory, then we are going to check which are the shipping providers we are currently dealing with. So we get back an array of shipping providers and then for each of the shipping providers we will check what are the shipping costs for each of the providers. So we'll do some kind of a loop here. So this will be a fan out fan in that we're going to use here to get the shipping costs. And then we're going to in our workflow, we're going to check which one is the cheapest and we will use that to actually register this shipment. And well, this part is in a try catch block. I will show you a bit later. And if this is successful, then we're done. If this is not successful, we can do something that's called a compensation action. So we can run another activity to undo the mutation that we did all the way in the beginning, because we were very optimistic here and already checking the inventory, but also already updating the inventory. So in case something goes wrong here, we can actually undo this inventory mutation right here. All right. And moving on to the code. So I'm running a workflow app. It's a web service running on .NET 8. The only dependency we have here is the Dapper workflow package. And we're using the latest version, Dapper, Dapper 1.15, which was released two weeks ago, I think. So this is the workflow definition. So a workflow is just a class. And since we are using the Dapper workflow package, we have some base classes available. So we need to inherit from the workflow class. Here we specify what inputs we expect for the workflow and what output we want. Once we inherit from this class, we need to override the runAsync method. Again, we specify what output we want back. This runAsync always starts with a workflow context, and we use that context to interact with activities, and we specify the input for this workflow. What happens then is, of course, it's all business logic in here. Here we specify just the output type again and then the first activity that we're calling is to update the inventory and this is both a check and an update and i will go into the activities later but what we are providing is the order so we provide an order item and using the context to call this activity this is the output that we want back and you always specify activities by name so that's why you see name of the class So for people familiar with Azure Drupal functions, this looks probably like exactly the same, right? Because it's exactly the same syntax that we're using here. So then we have an if statement that asks, okay, is the stock sufficient? And if that is the case, then we continue and we are going to schedule another activity call and that's the get shipping providers. Now, as you can see here, this is, I've made a bit of a... a more extensive example here, what you can do, you can optionally supply some workflow retry policies. So if you expect that this call to this activity, because in an activity you probably do another call to another servers or to a database or whatever, if you expect that something might go wrong there, you can apply some retry policies. In this case, I have applied a retry maximum of three times, and we do a first retry interval of two seconds. There are many more options you can specify here. You can also do like an exponential back off and stuff like that. There are a lot of options here, but this is good to know, right? So once we get this back, we get back an array of shipping providers. So that's what you see here. So this is what we get back after this activity call. And we're going to iterate over these shipping providers. And what we're going to do is we're going to check the shipping costs for each of these providers. And in this activity, we will call out to another service. to get that information. But what's important to realize here is that we have this call to the activity, to the getshipping calls, but we're not awaiting this call. So whenever we do an await, the workflow will stop executing. The activity will actually then run. And as soon as the activity is done, the workflow is replayed. But that's not happening here at this point yet. So we are not awaiting the calls to the activity, but we are actually We have a task defined that actually will call the activity, and we are adding that task to a list of task shipping cost results. So this is the output of that activity. So at the moment, let's say we have three different shipping providers. So after this, for each, we have three activities lined up that we want to execute. So this is where the... fan out fan in part comes in and at this line here is where the fan out but also fan in part happens so now we do await task when all and here we provide all of the tasks that are going to call the activity so at this moment the workflow will will stop all of the activities will be scheduled they will all be run and the workflow will only continue when all of the activities have been completed So then we have in this case three different shipping cost results and we are simply checking which one is the cheapest and we're gonna use that when we do the registration of the shipment. So that's the next part. So here we are going to use the context again to register the shipment for the cheapest shipping service here. We are awaiting it again. But what's different now is this is put in a try catch block. This is just to illustrate if this activity fails and we catch the exception. In this case, the exception, it's wrapped in a workflow task failed exception. So that's why I'm catching this one here. Of course, if you want, you can check what the inner exception is of this exception. But here you can do the compensation action. And the compensation action is nothing more special than, again, writing your own activity that you want to act as a compensation action for something that you did earlier in the workflow. So this undo update inventory activity will just make a mutation to the state again to compensate for our very first activity that we did here. All right. These are some examples how you can write such a workflow like this. Let's have a look at some of these activities. Again, activities are also just classes. In this case, you inherit from the workflow activity class. Again, you specify the input and the output. In this case, we are using the Dapper client because we're using the state store, the state API to interact with Redis in this case. So that's why we are injecting a Dapper client here. You also need to override the run async method again, and now you're working with a activity context instead of a workflow context. And here's the input again. So in this case, we are doing a get state to see if there's a state for a certain product ID. If there's no product at all, we just give back a response that indicates I don't have any inventory for this product. If the product inventory is known, then we check if the quantity is more equal to whatever we want to order. And then we do a mutation of that inventory and we are saving it again using the Debra client. The other activity that we'll check for the shipping code. So this activity will actually make dual serviceification to another service, to our shipping service. So here we are injecting HTTP clients. I will show you a bit later in the program CS how this is configured. And we are doing a post to the SAP client to the calculate cost endpoint. And providing the shipping request, and we get back a result. And we just pass it back to the workflow. The same is true for the calculator. So this is the endpoint in the shipping service. So this is the endpoint that we're calling. As you can see, I'm using some demo code here, which involves like a thread sleep. So I have enough time to actually do some stuff later during the demo. And in this case, we'll be you just get like a random cost back. You have a similar approach for the register shipment activity. Again, we use an HTTP client here and we're doing a post to the register shipment endpoint. And this is then the register shipment endpoint right here. In this case, it always returns back an OK. I can tweak the code later to return it a 500. If you have enough time for that, we'll see. Now, one thing that is... Different compared to to jubile functions in this case if you use depa workflow you need to register the workflow and all of the activities in your in your startup. So here's your iService collection there's an extension method here add depa workflow and you need to provide some options and here you need to register the workflow and all of the activities that you're using. Because if you don't do this, the DEPA runtime will be unaware that there are any workflow activities or workflows itself there. So it's going to do any of the scheduling. So this is an important step. So this program CS file also contains our starting point for our demo. So what we're going to do, we're going to make an HTTP request to the validate order endpoint. we get the Dapper workflow client from our service provider and we're going to use the schedule new workflow async methods to actually start our new workflow. So it's important to realize that it's named schedule new workflow because it's the responsibility of the Dapper runtime to actually start it and execute it. So it's called schedule. So what you get back is you don't get back. the immediate response of the workflow, because a workflow can be very long running. It could take days or weeks or months or whatever you have implemented, of course. So what you get back is you get back the instance ID of the workflow, and then you can actually query the workflow itself by ID to actually get the state of the workflow. So I'll demonstrate that as well. And there are some additional endpoints in this application to make sure I can manage my inventory. All right. Well. I told you that the workflow are depending on the state, because otherwise, if the process is gone, the runtime cannot rehydrate any of the state anymore. So workflows need a state store. And Dapper uses state store configuration via component files. So these are YAML files. And in this YAML file, you specify that we're using a certain type of state. In this case, we're using a locally running Redis container that gets pre-installed when you install the Dapper CLI. But of course, when you're running in production, you probably run in Azure, so you probably use something like Costos DB. What's interesting to know is that Dapper workflow is built on top of Dapper actors. So that's why this actor state flag should be set to true in your state store component file. So that's just a very bit of an implementation detail. You don't have to really know or understand how Dapper Actors work if you want to use it for workflow. But you do need to know that you have to enable the Actor State Store because otherwise workflows don't work. All right. So finally, I'm going to run this sample via a terminal. And I will use the Dapper CLI with something called Dapper Multi-App Run. and we will point the Dapper Multi-Ap run to this configuration file. So it points to a resource path which contains the component specification to indicate if you are using Redis as a state store. And it contains two applications, our workflow application and our shipping application. So when I do Dapper run dash f dot, it will point to this file so Dapper knows it needs to spin up two applications and also each application will have the Dapper sidecar. All right, so what will happen now is I will start the application, both applications. I will start the workflow application, and I will start the shipping application. So that's happening now. So this is the equivalent of dapper runf, but I've scripted it using this demo time extension, so you don't see me running it. But our applications are running now. We see some debug logs already happening here. So both services are running. call a couple of endpoints to make sure I have enough inventory. So I'll make an HTTP request to my workflow application to slash inventory restock. So this will make sure I have enough stuff available. All right. I can actually check it if I have enough. Yeah, I've got five items of my rubber duck 1000. So it's all OK. OK. And the next thing, what I'm going to do is I'm going to start the workflow via this slash validate order endpoint. And then this is my order payload. I'm going to order two of my rubber duckies. And once this is started, I will have it run for a couple of seconds, but then I will stop all processes. So this is to simulate a very serious error, and everything will be stopped. So our workflow app will stop, our shipping service will stop, and both of the Dapper processes for each of these servers will also stop. Then I will restart everything again using Dapper run dash F, and then we'll see what happens. I'm calling devalilator response now. And I'm going to check the logs to see what's happening, because at a certain moment, I want to simulate an outage. So then we'll stop all of the applications. OK, so it's running. It's running. It's not very far. OK, yeah, now I'm going to stop. So there's a lot of logging going on because there's logs of both the applications but also of the type of sidecar. But the final piece of logging that I saw is that from the shipping application, we are logging that we are getting the shipping cost for a certain order. So this is the last information that we've seen. So our workflow application is communicating with our shipping app. So the shipping app is now getting the price, but it hasn't been... completed yet, so it didn't send the price yet back to our workflow. But now everything is dead. So this is, let's say, a serious outage. OK. What I'm going to do now is I will restart the applications. So this will restart both the workflow application with the Dapper Sidecar and the shipping app and the Dapper Sidecar. And let's see what happens. So everything is starting up again. And then we see exactly where we were. So we see shipping service C, shipping service A. So things are continuing. without us actually making the call again, because all of the state was stored in the Redis state store. And they had the Dapper runtime, had a Dapper workflow engine, got all that state again, and just reprocessed everything. And it will now actually continue to completion. And indeed, here in the logs, we see workflow completed with status. Workflow status completed. So this is a good example of sort of whenever You can imagine the worst case happening. So you have an outage, your whole application surface is down. Whenever the process restarts, or maybe you redeploy it again, it will read all of the states back, and it can run to completion. I can also call this endpoint again to make sure that our workflow instance is actually completed. Oh, don't do that here. Oh, I should click the right thing. All right. So when I'm using the header Dapper workflow and management API to actually get me back the status of this workflow, it also says completed. We see the input and we see the output again. All right. So you see like, yeah, drupal execution I think is really a powerful way to write your business process and to really make sure that they always run to completion. OK, I'm going to stop this. I'm not going to do the other issue with the workflow. I will continue with. Another part, and that's resiliency. All right. So it's important to realize that not all technical issues are the same, right? So we have transient failures, which are temporary, and they usually are resolved. automatically. It actually means that someone else was actually smart enough to implement a solution, right? So in case some firmware or software written by someone else resolved the issue, so that's great. And there are also permanent failures, and those require really human intervention, right? So maybe some code is broken and you need to fix the code and redeploy again, or you maybe need to restart a server or something. So those are permanent failures. And both of these types of failures also need a different approach to actually fix them. And so in case of transient failures, it's totally fine to, hey, if you could request fails to a service, yeah, it's totally fine to retry the request again, right? Because maybe there wasn't even network error, and you can just retry again, and probably it will work now. But in case of permanent failures, it doesn't make any sense to just, yeah, retry and retry over and over again if there's a serious server error going on. So in case of permanent failures, you're probably better off with timeouts. So you specify a maximum time that you want to retry something, and then you just stop processing and take an alternative route. And also circuit breakers are very valuable there. So in case of a circuit breaker, if a dependency is failing constantly, you just stop making a request to the service. And once in a while, you just check once or twice if it's still breaking. If it's breaking, you don't do anything. If it's back again, you just open it up again. So these things are all baked into Dapper, right? So you don't actually have to do anything. So retries and timeouts and circuit breakers are all baked into the Dapper sidecar. So you don't have to write them yourself. So probably many people have used or known about Polly, for instance, who can also do like all of these policies. So in that case, you still have to include a package and configure that. Dapper already, yeah. has that built in. You can still override it, and I will show you how. So in this case, and by the way, you can do like resiliency between like service communication, like I'm demonstrating here. You can also have resiliency policies between a service, but also a component. So in this case, I want to demonstrate resiliency between application A and B. So application A, we will call it to make an HTTP request. This will also make a request for application B, and application B will actually save something in a state store. So what we have here is application A, and this is the endpoint that we are going to call. And we are using an HTTP client here, and we've configured the HTTP client with a certain header, so we know that we are talking to application B when we use this HTTP client. And we are going to call the slash profile endpoint on this application B, giving it the profile details, and just returning the results. And if you look at... application B. So this is the endpoint that we're calling and here we're using the Debra client to save some state. So nothing very spectacular happening here. This is the state store definition. Again it's the same definition as we saw for the workflow component. So this is the multi-up run configuration file. So now we have again two applications, application A and application B. Okay so what we're going to simulate now is let's simulate an issue with application B. So we are going to start application A. So application A is now running. Let's split the terminal. And here we're going to navigate to application B. So I can't make use of the Debra MultiApp run file now because I want to actually decide myself when to run A and B. Okay, so here we are in B. So A is running on the left here. And I will copy the commands to run application B, but I won't start it just yet. Okay, so A is running. B is not yet running. And now, so I've done this. And now we are going to make the call to application A. So this is the endpoint we're going to trigger in application A. But application B is dead. So this will not work. So let's see what happens here. You're going to make the request. And what we see here in the logging is error posting operation endpoint app B. appB profile. So this is dapper log showing that, hey, there's something wrong. We cannot reach the profile endpoint. But it also says retrying. So that's good news. We didn't have to do anything for that. But dapper is retrying for us. OK, let's start application B now. OK, and now we see that the response has gone through. It took a while, of course. But what you see here in the dapper logs is recovered processing operation after eight attempts. So here you can see the power of dapper. It will retry these things for you. OK, stop these applications here. So I told you that there are resiliency policies that are built into Dapper, but you still have the option to configure them yourself, right? Because you might not like the defaults that Dapper provides. So what you can do, you can always add one of these YAML resiliency files to your configuration of Dapper, and you then can specify policies in this YAML file. So policies can be timeouts, retries, or circuit breakers. So in this case, we have a couple of retry policies. I'm not going over all of them because there are many different demos here. In this case, we are using this one, the my constant policy, which is like a constant retry policy, which is useful in demos. This means I'm going to wait two seconds between every retry, and next retry minus one means just continue indefinitely. Again, that's not something you want to do in production, but great for demos. You can see you can also specify exponential policies as well. And once you have a policy defined, you also need to... apply it to a target. So that's what you do here. Here we have targets. Targets are either applications or components. And here you specify the application ID of the application that you want to apply to. So in this case, we have applied this my constant policy as a retry policy for application B. So this is how you do it. So of course, I mean, who's a fan of editing YAML files? I don't think a lot of people are right. So I'm also definitely not a fan for that. There is something that's called Conductor, which is more like an operational tool if you're running Dapper in production. One feature that was added recently is a Resiliency Builder. So this is more like a developer-oriented tool if you did that, yeah, sort of a wizard for making these YAML files. So you can specify a name here, and then you can specify what kind of policy you want to build. So in this case, we want to do a retry policy. You give it a name. You can say, OK, it's a constant or exponential. And here you can specify all the values that you want. You can then confirm it. Oh, and by the way, you also saw that you can actually also use HTTP status codes now. So that's something new. So you can really, really be specific when you talk about circuit breakers or retries or timeouts. You can actually specify, OK, I want to only do this for certain HTTP status codes. So this actually helps you to create these YAML files in a decent way without spending much time on this annoying YAML format. Almost out of time, so I'm not going to do the other policies. Like I mentioned, this is all in a Git repo, so I will definitely recommend you to try it out yourself. go to the final closing part. So I just give you a very quick overview of Deppr and particular Deppr workflow. Like I mentioned, use the left QR code if you want to try these demos yourself. It comes with a dev container, so you can either use a dev container locally or just spin it up in GitHub Codespaces. The right hand goes to Conductor, the tool I showed you that can help you build component files or resiliency files. So yeah, please give it a try. And I'll go back. I'll go back. Just one sec. I'm still peaceful. OK, yeah. Let me go back. Yep. Only for Eric. Yeah, all right. OK, and finally, yeah, please review my session. I always like to improve. So please let me know how I can improve. And if you want some stickers, some Dapper stickers or rainbow stickers. Please come over if you have any questions. Now is a good time or come over. I'll be around here for like another hour or so. So thank you.

Applications break all the time, there could be a network issue, a cloud provider outage, or just a glitch in the matrix. But as a developer, you really need your applications to be resilient without the need to recover databases and restart services manually.

In this session, I'll demonstrate how Dapr Workflow provides durable execution, which enables you to write reliable workflows as code. In addition, I'll show how resiliency policies in Dapr improve reliable communication across services and resources when developing distributed applications.

I'll go into specific workflow features, such as scheduling, sequential and parallel execution, and waiting for external events. I'll show many code samples (in C#) for each of these features and will run the applications using the Dapr CLI to demonstrate their resiliency.

By the end of the session, you will have a good grasp of how durable execution with Dapr workflow and resiliency policies can help you build resilient applications.

#futuretech #dotnet