SAGA Pattern Explained: Fix Microservice Transactions

Today we are going to discuss a problem. You cannot share one transaction across microservices. But banks still do. With two-phase commit protocol. Money in transit cannot be disappeared. This is the reason from banks. But you pay for it. Latency, locks, availability. These are the issue with two-phase commit protocol. And model stack like Kafka, Goz. Redis cannot even speak the two-phase commit protocol. So modern microservices replaced it with something else. Let's see how modern microservices do it. Let's start with monolithic. One process, one DB. One database, one transactions. You write one table, then write on another table, then you commit. And database automatically does four things for you. Atmosity, consistency, isolation, durability. Every transaction, no effort from you. This is the world where we grew up while writing the code and it just works. Now we move to the microservices and immediately something changed. Three services, three database, no shared transaction. So we asked obvious question. How do we keep them in sync? The classical answer is two phase commit. And if you watched the last video, you already know how this ends. Blocking, availability, collapse are the issue with two-phase commit protocol. And Kafka, Mongo cannot even join this protocol. Two-phase commit doesn't survive modern microservices. If this part is new to you, please pause here. Watch the two-phase commit video. Link is given in description. Then you come back here. Database engine that enforce it disappeared. The business doesn't care what we changed in architecture. Payment still cannot be lost. Order still cannot be doubled. Requirement survive. What change? Who is responsible for uploading it that's used to be in the database sitting under your code. Now it is the responsibility of code. The discipline the database gave us for free. We have built it by design every time. We name it what we actually building. We call it Saga which sound like an epic movie but it is really just a story of small transaction. Told in sequence across services. Each step is a normal local transaction completely happy with fully acid inside its own database. The trick is what happens when step 3 fails. There is no magic rollback. Database cannot reach across the network. To undo the what already committed. It has no idea those other services even exist. So we do the next best thing. We manually write our control jet. For every forward step, we write a compensating step that reverse it. Reverse inventory. Release inventory. Charge the card. Refund the card. It is not a rollback. It is a brand new transaction. Moving backward. Basically cleaning up our own mesh. And that gives us back 304 asset guarantee. Just in slightly different shape. Atmosity achieved by our manual compensating transaction. Consistency reached eventually. Durability kept by local commit. plus a saga log that tracks our progress three letters rebuilt entirely by hand this is the saga not a workaround not a weaker transaction a discipline we are doing the database old job but now we have to do it in our code if we boil down all these explanation into one clean line pause this video read it carefully this is definition and drop your question in comment section Atmosity handled by our clean up code. Consistency eventually reached. Which is just a polite way of saying it will sink when it gets there. Durability we save our progress in a saga log. That is a saga. It is not magic. It is just good planning. So you do not have to apologize for lost data later. Every saga is built from three specific kind of step. First is compensable. Any step that we can actually undo. It is a step where we have already written the cleanup code. Think of reserving inventory. If the order fails, we can just release the stock. Like you have an undo button. It is compensable. Next is pivot. The point of no return. Once this step commits, the saga is committed to finish. No matter what. like shipping a package once it is on the truck and halfway to the customer you cannot really compensate like you cannot hit undo button every well designed saga has retrieval retrieval those steps happen after the pivot there is no going back now so the only way out is to complete the process if it fails we keep retrying until it works confirm the order, send the receipt, update those loyalty points. At this stage compensation is not an option. Persistence is in one line and keep it simple. Compensable before pivot, retrieval after pivot and pivot is the single hinge that holds the whole things together. Knowing exactly which step is which, this is the secret to handling the failure without waking up at all. Once we accept is Sada as our model. We hit the next big architectural question. Who is actually driving the bus? It is not about which one is correct. It is about how much control you want to trade for simplicity. We have two paths, choreography and orchestration. In choreography, nobody is the boss. Every service is independent. They just listen for the events and react. Order services finished and says, Order created. Payment service hears it, charges the card and says, Payment succeeded. Inventory hears that and grabs the stock. It is peer to peer. It is decentralized. It is beautiful. There is no single point of failure. But good luck, trying to figure out what happened and when the process dies in the middle of 10 services, it is totally mistreat. Moving to orchestrations. In orchestrations, one component runs show. Orchestration is the conductor. It knows the exact steps, call the services, listen for replies and decide what to do next. If a step fails, orchestrator already knows the plan. It just trigger the compensation. Every command flow through one brain. You have one single place to check the saga state and yes, it is also one single place that can crash the whole thing. So, which one do you pick? Simple event-driven flow go with choreography. Complex logic with a dozen decision trances go with orchestration. But debugging a choreography-based saga across 10 services and 20 different event type. when it is just for the right of passage for any distributed system engineer you will only have to express that level of complexity once before you realize that building an orchestrator is the far more maintainable path forward keep it simple for choreography use orchestration for complex stuff in reality most production systems are messy and use both of them choreography and orchestration now in real production environment things gets bit more involved you need timeout for every steps and idiom potency is not optional it is mandatory if there is a transaction to add 10 dollar then it will add 10 dollar irrespective how many times same request come and my advice do not try to hand roll this use a better testing framework But there is a honest truth that most blogs tend to skip over. Even your compensating transaction can fail. Company has documented this extensively in their pattern and it is a reality. You have to accept it. So you do not just plan for success. You have to plan for the failure of your failure handling project. Now there is a bigger catch. With saga pattern we regain Atmosity, consistency and durability. But there is a fourth pillar, we lose isolation. Since your saga is broken into local transaction, other services can see your intermediate half-finished data. Let's see, order is completed, but payment service is in progress and inventory is in pending state. Other transaction can see that order is completed. before order status is placed it is visible in mid of saga it is a significant challenge but there is a path forward we use we call countermeasures but before we dive into those patches we need to build the foundation in our next video we are going to implement how to build the choreography and orchestration then we will see the countermeasures

Distributed transactions failing? Two-Phase Commit (2PC) is dead in modern microservices. Learn how the SAGA Pattern guarantees data consistency across multiple databases using compensating transactions, choreography, and orchestration.

In this masterclass, we break down exactly why 2PC fails at scale and how to build resilient systems. We cover the honest truth about ACID properties (and the loss of Isolation), how to handle Pivot and Retriable steps, and the production realities of what happens when compensating transactions fail.

🔗 Resources & Links:
* Watch Episode 1: [https://youtu.be/A5aRyNfH_TM?si=luJceScaDH1Kj3cD]

#sagapattern #microservices #distributedsystems #systemdesign #thesmartstack #springboot #java