On .NET Live: Shaving the outbox pattern yak

good afternoon good evening welcome to 60 minutes of unscripted.net entertainment this is on the net life our mission is to empower the donna community to achieve more i'm your host maya wenzel with my co-host katie savage and frank boucher and i'd like to welcome today's guest joan antonis welcome joan hello thank you how about you introduce yourself sure so my name is joan tunis I'm Fort Portugal. I'm currently working as a lead architect at OpenView Mobility, which sounds like a big fancy thing, but it just means I do a lot of different bits of stuff, which hopefully makes me a good person to talk about the subject we'll be talking about today. Which will be the Outbox pattern. You have a project you'd like to show us. Yes. I'm going to use the project as an excuse to further discuss the Outbox pattern. why it's relevant, but yes. All right. All right. Take it away. Let's see what you have to show us today. All right. So I'm going to freestyle a bit. So as you can see, there's no slides. This is like an Excalibur. I'll focus on Excalibur, showing some stuff, showing some code. I'll start with some theory. So when folks get bored of the theory, please complain in the comments. Hopefully not too much. and feel free to to feel free to ask any questions interrupt that's the point that's why i'm freestyling it so we we have a discussion if it makes sense so the outbox pattern i think many people know about it but let's ensure everyone that's watching knows about it so what's the problem related to the outbox pattern, or better yet, what specific problem does the outbox pattern try to fix? Imagine we have like, I'm keeping it simple. We have a client application. We have an API or producer application. I'm calling it producer because it will produce messages to a message broker. And it also has some persistence in the database. And then at the end, we have a consumer. So what's the, potential problem here. Imagine that the client does a request to the producer to do some operation, like create, post an order, something like that. This service will want to persist something in the database. So over here and then notify if we are building an event-driven system, notify the consumers that something happened. So produce the message here. As these are completely different systems, the database, let's say it's a Postgres database and the message broker, let's say like Azure Service Bus, they don't interact with each other. So the producer orchestrates all of that. What may happen is that when the producer does something, for example, writes in the database and then produces a message, something may happen along the way. For example, let's imagine that the producer was able to write to the database, but then fails producing the message. So then there's an error here. What happens here is that the system as a whole becomes inconsistent because we wrote something in the database. So it's become a fact. But then the consumers don't know about it because it failed to produce a message. And it may fail for different reasons. We'll drill down a bit in a minute. But it may be because the message broker was down because the producer went down mid midway because something in the network went wrong something like that the important part is that because there are two things that don't directly depend on each other one may fail and what might succeed causing the system to become inconsistent so first i'd like to say something that is because i've heard that doesn't happen it doesn't go down and like in the previous company i worked When we first introduced messaging and I was talking about such a problem, the architect at the time said, oh, but the broker is never done. And I was like, really? So, okay, so it is. And so I'd like to share some pointers on it is documented that things go down. Let's start with Azure SQL. I don't know if many people notice this, but there are some docs, a lot of docs. about things in Azure. And maybe there's some relevance in the docs. There is a section about transient errors. So it is in the docs saying that Azure SQL, because as we know, we don't care, or as developers, we don't think about the servers where Azure SQL is working, but they're still servers. Azure SQL is still running on one server. And at some point, maybe... it needs to failover, it needs to be upgraded, it needs to whatever. So there are transient errors and it's documented. And for example, it's saying here, most of these configurations event finish in 60 seconds. To handle them, you should implement RetryLogic. So it's documented that these things happen. So, and then you can say, oh, but it's an Azure SQL thing. So then we can go to a different example of a different cloud provider. Don't know if... Folks have heard of the SLA, know what that means, which is a service level agreement. And any service you use in any cloud provider will have something like this, which means they guarantee they have a certain uptime, but that guarantee just means that if they aren't able to fulfill it, for example, in here, it's talking about giving credits because they failed. the that compromise so in that case we can see that for this particular service it's 99.99 95 so it's basically admitting it will go down at some point even if for a very low amount of time and just what they try for as the level of of of service but it doesn't mean that it's always there um but yeah i'm i'm now getting super curious about your talk because uh one of the services that i inherited here at microsoft it does have messaging and we are going through like things that are failing in one of the points so it's like you call an api from one service and then you go and you have to save on your database like we have a dynamics urm and like that thing in the middle broke right so it's like how do you fix that i'm super curious on on on your talk now Exactly. And just to make sure that it's not, oh, but it's the big cloud providers. So I have an example. Many might not know, but Etsner is like a small cloud provider from Germany. And they have an SLA because everyone needs to have an SLA. And they say that their cloud servers have 99.9% uptime. Or otherwise, they refund a certain amount. So, yeah, no one can claim a hundred percent. They will never up a hundred percent. Yeah. So, it doesn't happen. Oh, it doesn't happen and it's documented. And I could even talk about RTOs and RPOs, but that's like more for disaster recovery. So, let's not go down that path because it's even worse. So, now that we have established that this happens, because sometimes people still are like, but why does it happen? so let's drill down a bit more and i'm i'm going to over drill down so sorry in advance but let's consider this you have the producer we mentioned above i'd say it's a dot net api and so we have a process running so now imagine let me just choose this this process let's imagine you like me i work with Kubernetes, so I put things in containers. So this is a container. And then we go, but the container is where? The container is running in the VM. And we can go further. The VM is inside. Doing it live. It's a VM inception. A physical machine. And then, okay, so we have all this. I'm probably forgetting some stuff. But then this physical machine has a network interface card. So when the producer wants to communicate the DB, it goes here. And again, I'm forgetting abstractions. And then we have probably a switch. We have a switch. And we have another switch. So then when we want to make a network call, we do this. And we do this. And this. And this. And let's not forget that normally in the cloud, we have all of this inside the virtual network. So besides the physical part of things, we have software-defined networking. So when folks think, oh, how will it fail? Well, it will fail because we are working with like 3,000 layers of abstraction. And at some point, something will fail. So things that can fail and a non-exhaustible list. So, our .NET API, it can crash because of a programming error. Maybe it hits a resource limit, like we put the container to use 256 megabytes of RAM and it wasn't enough. There's another thing that maybe folks don't think about it too much, which is graceful shutdown. For example, if we, I don't know how many folks use continuous integration and continuous delivery, that means normally you do a lot of deployments per day. If you do a lot of deployments per day, it means that some services are being scaled down and up, scaled down and up. Are your applications ready for that? Because it means that the service that was responding stops responding. And it's graceful, but it still happens. So scaling, autoscaling, it's the same thing that you can scale up, but then you can scale down. So the service that were responding, now it's not responding again. Similarly, the nodes, it can crash because of hardware, because of the operating system. working a graceful shutdown because of maintenance like os updates kubernetes upgrades hardware maintenance like replacing part of the machine scaling and auto scaling i included this here because it's interesting when we want to save some some money on the cloud we use something like spot instances that are cheaper but the cloud provider can take it away from us at any time and of course networking like the switch can fail the The network card can fail, ESP can fail, DNS, the classic that we always like to joke around, and the software defined, so networks. So there's a lot of stuff that can happen, probably too much. So I overdrill down, but I hope to make a point that is really something will happen. It's just a matter of time. So I think mostly theory is kind of done. this is this ties in well with what Maeda was saying that in this in this session i will focus on the hotbox pattern because it's enough for us to talk about but the problem of doing things that cannot be a single transaction because they go to the database because they call an api because they call they call the broker it's like a general problem that that's not simple to solve In this case, we'll focus on the hotbox pattern, but it's a very specific instance of that problem, which is kind of like a solved problem because in .NET in particular, we have a lot of good libraries to handle these kinds of things. I will talk about them in the end. But keep in mind all of this that I said, it's not just about messaging in APIs. It's about everything that's not a single transaction. Let's look at the demo. I'll start with no outbox, so we'll see things in action. Some context. So, I created this here. It's basically the same as we saw above. So, what we'll have is a client application, which I created a CLI using Spectre Console. And it will make a lot of requests to the producer. This producer is writing in the Postgres database and in RabbitMQ. and which will then consume and write here to make things interesting i created something i called container disruptor so the idea here is like following some ideas from chaos engineering and we will introduce failures to see how the system behaves so what will happen is that this container disruptor will randomly kill containers like the producer api the message broker the database etc In fact, I put this, I started this before we started the session. So we can see here, hopefully it's big enough for everyone to see. I have this Docker running and you'll see that sometimes some containers stop. Like now Postgres stopped and eventually something else will stop. RabbitMQ now died. So this container disruptor here is just killing things randomly. So you have your own little monkey behind just plugging and plugging. Oh, that's cool. So it's nice to see it live, too. You know, yeah, it's true. It's cool. And it can. So because it's random, the demo may not go as well as I want it to. So hopefully it cooperates. That's live. Yes. So no, but that's cool to see it like. Because it's a good way to test those failures is to actually have random things to break, right? So that's really cool. Exactly. So the endpoints are the most basic thing ever. So I just have a post where we'll post some body. And in the first demo, which is without Outbox, we will add it to the database. So we're using Entity Framework here. As you can see, I'm adding the entity, saving changes, and then calling the producer for RabbitMQ, and in the end, returning a 204. So let's see how well this goes. I have this little application. So it asks me what I want to run. I want to run without Outbox right now. I can choose. Let's make it 2,000 requests to see if we see it working. So we have this here. It takes a bit. In the meantime, we can even see the logs in the container disruptor. We can see the logs what it is killing. So Postgres, producer, producer, producer. It's likely to kill a producer. But it is random. RabbitMQ. So you can see things going. What the color mean? Is it like now it's kind of yellowish, orange? It means that, so ideally, What we want is that the producer and the consumer entities are the same, which means that we could write them to the... Let me check. We wrote them here, we produced a message, and then the consumer writes to the database in the end. If in the end of the run, the two databases are not the same in terms of entities, it means that messages were lost. Okay. So if we go back to your screen, so... Now they are red, I'm assuming because the discrepancy is too big, right? It's 500. It's red because it ended. So during, it was yellow because it's different. But because it's like eventual consistency, it may take a while to get there. Yeah. So I was green when it was matching, yellow when it wasn't matching, and now it's red because, okay. So what happened is... And we were targeting to have the same value. Exactly. I get 110. So we have requests. We did 2,000 requests. We only got 1,009 with a successful response. This just means that the client got a successful response. And 991 failed. So the producer persisted in its database, 1,509, while the consumer, 1,009, which means that 500 messages were lost. In this case, because it's random, it could be more, it could be less. We could even be lucky and it worked without an issue. And one thing interesting that I am not sure I probably won't be able to demonstrate, but I think it's interesting to mention is that in this case, the requests succeeded and the consumer entities are the same because every time it was able to produce, it was also able to return the 204 response but that's not a fact because we could for example have a timeout between producing the message and returning the response to the client so even this is not certain but that is a client may think it failed but it actually succeeded but i don't think we because i didn't know if i had a timeout or something like that in the the end point maybe i could show this happening but i wanted to avoid the timeouts so it wasn't cheating too much to to to make a point so this is the classic we have this um a lot of places i've worked including where i am right now we have this problem where we do something in the database then produce later and then the message never went and then we have another scenario which I think is less common, but I've been asked that. What happens? So, if we are afraid to write in the database and then produce a message, what if we inverted it? Let's produce the message first and then write to the database. So, I can run it in a demo as well, but I think most people can guess what will happen, which is mostly the same, but the other way around, which we will have more entities on the consumer than in the producer and i would say that from a both both are bad but i would say from an understanding perspective this one is worse because when we produce an event in an event-driven system where we where the event is a fact something that happened so we are sending a message saying that there is a fact But then it's not true because we didn't persist. If we don't persist, it's not a fact. So at the end of the day, it's kind of the same. The system is inconsistent. So in the first case of these problems going wrong, the fact is there, but we basically haven't told the consumer yet. And that's where the issue arises. And you're saying in this case, we've told the consumer that something happened, but it didn't actually happen. It's not registered in our database. Exactly. okay and for example imagining someone who has this problem maybe it's easier to to fix the first one fix like after for example okay we have all the data in the database we know that something went wrong let's just produce the events again or something like that but the other way around is like okay we produce the events now how will we change the database and put the right ids and stuff like that it's a bit I'm thinking of like a purchase scenario where it's like you're trying to purchase and then tell the consumer that he got the item. So if you tell them they got the item without the actual purchase, like without securing the item, it's worse than securing the item for them, but not telling. But both are bad, right? They should match. Yeah, exactly. Both are bad, but it's like one it's easier to fix or something like that. but okay it finished and as we can see the consumer consumed more than the producer wrote so we got into this inconsistent set so now let's see what differs from the outbox i think everyone will will know given all the the intro is that what happens is or better yet maybe not everyone will know because i didn't mention that specific bit is that what changes particularly just to clean the public instead of the producer writing directly to the message broker like this in one shot it will in one transaction it will write the entity plus a message so the that's maybe if folks will see or remember what an outbox is, is where you put the letters that someone will take later. And if you go to the emoji selection on your computer and write outbox, it's there. There's an emoji for an outbox. And it's exactly that, a place where you put your letters. So we are putting our entities and our messages, which are our letters, in the database. And then the producer behind the scenes will read those messages. And only then it will write them to the message broker so if you had to to say a definition of the uh of that pattern like it would be like the entity and message are creating one transaction and until they are validated they are public like you know what would be the definition of that pattern i was it was my english is somewhere else i was really starting to talk in french uh it was someone in the in the chat on twitch was asking was the fuel was asking like what's the outbox pattern and i'm just trying to get like a short clean definition of that pattern let me check if i have it here because i understand what it does but i'm just trying to put words on it sorry so let's name it better because the complete name is actually transactional outbox pattern and that's important because that's the the pain is that the we need a transaction way the transactional way to ensure that everything works so what happens here let me here so what would happen is in this case i'm using an example as a relational database but we got a request we have something that handles that request And it stores both the entity and the message. And that's where comes the outbox naming, which is we create the outbox table in this case, or if it was a document database, it would be a collection or whatever. In this case, it's a table in the same database. Because as we established, we cannot have something transactional across different systems, but we can have something transactional in the same system. be it a relational database, be it non-relational, be it like MongoDB, Cosmos DB, all of that supports you write things in a transactional way. So this ensures that if this command succeeds, both things were persisted in the database. Never just one, always both. If it fails, everything fails. Then what happens is that we have something it can be in the same process can be out of process that is reading this outbox table grabbing the the messages and then sending to the broker and then when it succeeds sending to the broker then they can delete market process whatever in the outbox table okay it's really leveraging the the transaction concept exactly um then because we are talking about it it has pros pros and cons so the pro is we mean we ensure system consistency every every time we write to the table the entity we write also to the outbox the bad part it adds latency because before you were just writing directly to the broker now you put an interaction you are right into the database and then something else is right into the broker but depends on Each situation, if you have a situation where this is a problem, you need to take a look at another pattern. And because we're talking about it, the outbox pattern, I don't think is like the only way to solve this kind of problem. There are other ways, but it's the more generally applicable. So it's easy enough to use. And most of the time solves the problem without really thinking about it. If you are using the right libraries, you don't think about it. It's solved. Otherwise, you need to think about it a bit. But generally, you can put it in place and it solves your problem. So, let's get back here. And let's go to the demo. But looking at the code first. Let's put the demo running and look at the code while it is running. So, we optimize time. So, without Box, let's make again 2,000 requests. So, what happens here without Box? The main difference here, you'll notice, is that I'm not producing any message to the broker. And this is because I'm using the library we talked about, OutboxKit, that we'll mention later. And in this case, and for folks who are not so used to entity framework, what we're doing is adding the entity to the DB context, so the change tracker. We're adding the message as well to the change tracker as well to another collection. And in the end, we call safe changes. What this does is with entity framework, only when we do safe changes is when data is persisted to the database. And when there is more than one operation to be done, entity framework automatically creates a transaction. So if entity framework didn't do this, we would need to create the transaction explicitly to persist everything at the same time. But Entity Framework does this for us, so the code is a bit simpler. But as you can see, that's all. We are adding the entity and getting the message and saving everything at the same time. Where does OutboxKit come in? So, OutboxKit and the kit in the name is important. It's just, it's not a complete Outbox implementation. It's a toolkit to help build Outbox implementations. In the part that we store messages in the database, OutboxKit doesn't care yet. OutboxKit works from reading the database to making the messages available for production. So in terms of configuration, we only need to do two things if we want to use the default. There are other options. But I'm just adding OutboxKit. And then I'm saying I want to use Postgres SQL polling and pass it the connection stream. And this will make OutboxKit go to the Outbox table and read things. Then the other piece of the puzzle is the production. OutboxKit doesn't implement message production. It implements making them available so that you can implement whatever you want. For example, at work, I don't use RabbitMQs, but I just implemented a producer for this demo, which needs to implement this iBatchProducer interface. If we take a look at the implementation, what it's doing, implementing the producer sync method. It's going through the messages and publishing each one. At the end, it adds this message to a list of process message, which OutboxKit will use to mark them as completed. All the messages that aren't marked as completed will be still in the Outbox because they may fail because the broker is down and stuff like that, and will be retried later. Let's see how the demo went. And it went as expected. So what this does with the Outbox is not that it makes the things not fail. They still fail. But the producer and the consumer entities are the same. So let me put it running again, just in case it was a fluke. You got so lucky that everything worked. It has happened. Of course. There was one question in the chat. about just like you know what if uh and the question was um about uh i'm trying to uh yeah thank you i was like what about the design where we write from a transaction in the database and then publish the message but then it fails like how do we roll back since the transactions in the database and not necessarily in the code no the you create you start the transaction in the code but the transaction is actually in the database right But the thing is, that is basically the same as the second demo I did. Because only when you commit is when you persist to the database, right? Yes. But you already produced the message before committing. So it seems that you started in the database before, but you didn't. Now you can say, it's less likely to fail because if you already started the transaction... it's the database is up so you produce the message and then when you're going to commit the likelihood that the database is up is high but it's still the same problem which is the database can go down in that moment your process can go down at that moment and the message already went and you cannot roll back the message yeah that's what i was thinking because things can fail in between when you're trying to roll back as well it could also fail so you still need uh the outbox pattern in this design someone to to avoid um any failure at any point right exactly there i can say something that's not relevant today i think but it might some people that were here like 20 years ago might understand what i would be saying if people recall If anyone remember MSMQ, which was like a Microsoft message queue technology, it had an integration with transactions. So we could create a transaction scope and persist in a SQL server database and send a message in MSMQ. And then the distributed transaction coordinator or whatever it was called would come in and do like a two-phase commit stuff. But we just don't use that. kind of thing anymore because it's all heterogeneous we might be able we might be using azure service bus and azure sql or we might be working with kafka and postgres so it's just not used anymore so we do stuff like that the outbox button for this kind of stuff great we had also another question related to uh like i'd like exactly the honor so just like can we use the outbox outbox pattern between thread in a WPF application, for example? I'm assuming the pattern is applied, you can apply it in every code, every language, right? It's a bit like I was saying before, that the outbox pattern is an instance of the overall problem when you do things that aren't transactional. So, I wouldn't call it the outbox pattern, but you can use things that will look pretty much like the outbox pattern. So, I did a presentation the other day and I called it leaving breadcrumbs, which is like you do something and you leave a breadcrumbs saying, I did this. Or better yet, you leave a breadcrumbs saying, I'm going to do this. And then you do it. And then you leave a breadcrumbs saying, I did it. And then you go to the next step. So if something goes down in the middle, you can always resume. You know exactly what your point is. Follow your breadcrumbs to follow it. That's great. I like that image. Yeah. So the Outbox pattern is like a concretization. It's like simplified. So instead of making the whole thing, we're like, okay, we're applying this to database and messaging. So we use the Outbox pattern. But if we want to do like a... long-running transaction that spans multiple services and stuff like that we kind of do something similar we probably just call it another name but it's just have a different name okay because like the outbox patterns for when you're talking about these distributed systems with where you have transactions happening where you need to like have the state and where things cut yeah the thing is you can build like us like a saga pattern on top of this for example you do one stage of your work and you persist it put the like a status and persist the event and then you receive a message to start the next stage of the side and do the same thing again and do again so we are kind of using the outbox pattern as like a building block for the rest on top so yes it's kind of the same, but then, for example, I think it's nServiceBuzz that implements like a saga pattern, and it's kind of like this, if I'm not mistaken. I never use it, but I read the docs a couple of times. I think they do something like this, like it proceeds to start to publish a message, and then starts the next stage, stuff like that. Okay, so, but in terms of demo, that's it. If anyone has any question about BoxKit, feel free, though, I'm just using it as an excuse to... show everything else. That's great. Awesome. And talk a little bit about your project in terms of the Outbox Kit. Where can we find, is it open source? Can people contribute? Yes. So maybe talk a little bit about that. Yeah, so Outbox Kit, you can find it in this URL. I think it's in the show notes, outboxkit.yakshavefx.dev. Maybe you understand why the name of the presentation, you know, where it came from. We did have a question about that. Could you explain for us that the name of the project? The Outbox Kit or the Yakshave part? The Yakshave part in particular. Yeah, the Yakshave. Maybe not everyone is aware, but it's a common thing that we say that when you in the start of the project, we need to do some boring stuff. But it's important for the project to succeed, but we need to do the boring stuff. And normally we call that yak shaving because it's boring, but we need to do it for yak needs to be shaved. So when I was trying to figure out the name for the projects, so the YakShaveFX is actually like an organization that I created to put my open source project. OutboxKit is the main one. So the effects comes from CoreFX. which is like the for dotnet yakshave comes from there so i didn't know how to call it so i just said yeah this is boring stuff that i want to use in every project yakshave fx so that's funny and now we know i was curious like i i was uh checking what's coming up on friday and i was like yakshave like what what i i i i heard it a few years ago and i i had the same reaction the first time i i heard the term yak shaving like why are you shaving a yak but yeah no that's very clever and so you have the the hub as well that i'll close yes on the comments it currently supports mysql posterior sql and mongodb because that's what i needed for work so this was basically we didn't have it at work so i needed to do it i didn't have time to do it during work hours so it's like okay i do it after hours but then it will be open source because you're not paying me to do it so that's why it's open source uh and yeah if some if folks want to contribute like sql server implementation and stuff like that feel free but currently i just did what i needed uh so but the it's it's on github it's already running tests on dot net 11. so it's eight nine ten eleven it's gonna be ready when i think when it goes live yep exactly uh and yeah so i can where it was i can say a couple of potential interesting thing or not about top box kit so the kit in the name is there for a reason like i said it's it isn't a full box you need to do work it just Yak shaves some part of the boring stuff. Like boring and not so easy. It might seem easy to implement an Outbox, but there's a... And it's not rocket science, but it has some annoying quirks. Like if you want to maintain order, how do you ensure you don't duplicate messages as best as possible, stuff like that. so that's why it's the kit i didn't want to implement everything i didn't want to implement like integration with all possible brokers it's just the base the the foundation and then you build what you need on top interesting stuff like or not only one instant producers at a time that's one important thing at work we use kafka so we we want to use we want to retain order so the only way to retain order is to ensure that they aren't like uh competing publishers reading the database and producing there are other ways to to do this we could do partitioning and stuff like that but it was wasn't needed and i don't think it's needed for majority of cases then there are other things that are annoying for example retries as i mentioned we could we can persist to the database and then fail to produce or we can succeed in producing and then failing marking it as completed So we need to do retries, but we need to take care to try to avoid producing too many repeated messages. And this is, I almost forgot this one. Outbox kit, or better yet, the outbox pattern as a whole ensures at least once delivery. It doesn't ensure exactly once. What this means is the same message might be sent more than one time. So the consumer needs to be ready to... to duplicate the messages or something like that, like if you receive the same message multiple times. For example, in the demo, of course, this is demo code, so it's not exactly as you would do. But for example, the insert in the database for the entities, where is this? This is a raw SQL on conflict with nothing. So maybe you cannot do exactly the same thing in your services. Maybe you have some more complicated logic. But in this case, what I'm doing is I'm trying to insert it. But if it's the same message, I ignore it because I already have it. Sometimes it's as simple as that. Other times, not as much. So definitely, yeah, for many cases, you have to have another pattern to manage duplicate or conflict. Exactly. And there is like the inbox pattern, which is like... oh where did that thing come from yeah makes sense exactly so yeah but that's one important thing but that's the thing we already needed to do it because the same way that we could produce the same message multiple times the broker can give the consumer the same message multiple times for example because you you took too long to respond to the broker saying it's done so the broker times out and sends you the message again so you need to ensure that you can handle it So it's a bit like a lot of small patterns that need to all work together to achieve this. And all those issues or possible crash, they're for sure they will happen when you're working in a distributed system anyway. So it's great to have pattern to manage those situations. Well, one thing that I mentioned, like the graceful shutdown. And if we were doing this in a room, I would ask people to raise their hand to tell me, do you handle graceful shutdown? Because I'm pretty sure nobody does. But at the end of the day, if you handle non-graceful shutdown, you can survive also graceful shutdown. If you don't handle one, then no. In the chat, people can send like a smiley face if they are doing it or like a sad smile, you know. the opposite can we do polls yeah exactly like side sad smiley face if you're not managing it it's always funny to see the the result of those i know for most i don't so fuse me was asking when you pull the electric plug yeah that's not so graceful what if you you know even if you're unplug it with grace the shutdown is not graceful exactly all right awesome to wrap up and doing like the opposite of what someone talking about the library they did they did should do i'm going to say don't use it what are those logos uh i i recognize the masters that one but he has the name yeah but these are all the others yeah these are all dotnet community projects that do messaging and not just messaging and i think most of the times it's a better fit than using outbox kit or even rolling your own i did outbox kit i think it's useful in two situations one you already have everything built and you don't want to touch much of your service and you just want to add something in there to solve the problem that's the reason i built outbox kit because we already have dozens of microservices, and I was like, okay, how do I add something without having to rewrite everything? The other scenario where I think OutboxKit might be useful is if you really like the way you do stuff, the way you structure APIs, and you already have like a messaging library, stuff like that, and you want to continue doing things that way, because OutboxKit is somewhat customizable for those scenarios. Okay, then maybe use it. Other than that, this is Wolverine, which is a more end-to-end project, which does HTTP endpoints on top of minimal APIs, has the Outbox pattern, and Inbox has a schedule, delivery, stuff like that, a bunch of stuff. Brighter, as well, is messaging-focused. It's like a mediator on steroids hooked into messaging. It has a bunch of integrations with different brokers, different databases, stuff like that. And then we have N-ServiceBuzz and Mass Transit. These are all open source from the dotted community, some with commercial licenses or not. And they have been at it for a lot of years, so they're great projects. So normally, use them. If you really want something very custom and you cannot use them, then consider OutboxKit. But otherwise... Stick to these. That's our first rule. You can talk about your project and go use these instead. No, that's awesome. And we should maybe reach out to some of these project owners as well to come and present there. I believe we've had mass transit on the show before, but I don't think that Wolverine or Brighter have been. We might have covered in Service Bus as well, but not. I think we spoke about Wolverine at some point. okay cool yeah but we could reach out and see these different projects um and have different shows as well it'd be awesome as well to have um any other resources that you want to leave us with or uh not really i think okay those are awesome that was awesome thank you joel thank you All right. That was great. Thank you, folks, for joining us today. It was great. Another great show. Thank you, everyone, for watching. If you want to check past shows, you can go to dot.ant.live, and you'll see the title for our show and the upcoming streams on YouTube and Twitch. and tuning i think in two weeks from now so we don't have a show next week uh but we have one uh on may 18 uh with david mccarter uh he will talk uh about coding standards to supercharge.net quality and performance um so i'll see you then um and i'll end with uh janice who's made the fourth be with you it's true we are may the fourth yeah so thank you folks until then bye

Everyone's probably already bored of hearing about the transactional outbox pattern, but clearly not enough, as systems continue to crash, and messages continue to be lost.  In this session, João Antunes presents his unambitious OutboxKit project, a toolkit to implement the transactional outbox pattern.  More than presenting the project, the goal of the session is to talk about the ideas behind it, reiterate the importance of this pattern in distributed applications, go over some of the lessons learned while building such systems, and even mention (better) alternatives to OutboxKit.

⌚ Chapters:
00:00 Countdown
03:15 Welcome to On .NET Live
04:26 The outbox pattern overview
08:20 Errors - they happen for everyone
11:54 Drilling down on why errors happen
17:35 Demo context
18:50 Demo
29:50 That is the Transactional Outbox Pattern
33:15 Demo
37:22 Q&A
43:30 OutboxKit
55:06 Thank you for watching!
56:12 Wrap

🔗 Link:  https://outboxkit.yakshavefx.dev/

🎙️ Featuring:  João Antunes (@joao.antunes.dev), Maira Wenzel, Katie Savage, Frank Boucher

#dotnet #dotnetdeveloper