Let’s understand first what the distributed transaction is and how the distributed transactions work in the micro-services. Suppose you have a client and the client in turn connects to the microservices and the services in turn connect to the database right. So, here the database is co-located, and you can do all the transactions. You can lock the table you can create update delete the records because it is located at same location. Now, if you have a database that is located in different locations, let's see how it can be done.
Now considering the scenario where a case like you have a client which is connected to the payment services and payment services are connecting to a database. Now you can see in the below image that there is another service called Notification microservice which in turn connects to a different database and this notification service and the database can be located at different locations or it can be out of a different network. In this case, there is no way we can do a transaction between two different databases or two different tables which are located at two different databases. We cannot apply any of the ACID transactions here.
So, here in this is how we are going to achieve this. In this case, we have a very interesting design pattern called “SAGAS” where we can use that pattern to apply the distribution transaction to a table that is located in different places. Now how the distributed transactions using Sagas design pattern works in the micro service-based architecture.
Suppose you have a client. The client intern connects to the micro services, which is a Payment micro service. A payment service, in turn, connects to the database where you maintain all your payment details.
Now I want to do payment and the payment request goes into the payment service, and it gets created in that database. So, now we are going to maintain a state in the table where I call it a “processing” state where payment can either succeed or failed state after receiving the notification from the bank.
Now can we fulfill this payment here? No, because we have not received the notification from the bank whether that payment has been a success or failure. As I told you the notification service is in a different database and the service is also located at a different location. So, how do I make sure that notification is available for this payment request?
So, what I'm going to do here is I am going to send a message from the payment microservice to the Notification service. The Notification service, in turn, checks for the customer’s account balance information and makes sure the payment is done or not. If the payment is not done, then it sends again back to the payment services with the message, which we call the Payment Failed. So, the process state gets changed into the "Failed" state in the payment info Database.
The order is completely canceled as payment is failed, which is similar to what we call is to the rollback. Let's take in the happy path, where users the user does the same payment request where he places a payment request, and it goes to the payment service and if the payment is successful in this case and the payment database got updated to "SUCCESS" state. Now in this happy path case, we noticed that there is a payment done by the customer, which is confirmed by the Notification service. So, the notification service sends the reverse message to the Payment service saying that Payment is successful and approved.
So, what happens over here is the "processed" state becomes "Success" and in this case, you can fulfill the payment process. It is how the whole transaction works, and this is how we can achieve the distributed transaction.
In the payment service, there are two scenarios new possible: either a successful transaction or a decline of the transaction due to any reason. In both the scenario, the request should move forward to the Notification service if the transaction is successful or it is a failure. Both kinds of states should we notify the user. From the Notification service, the user will get the response in the UI service whether the stock purchase is successful or not. If you look at the simple use case scenario, we have four microservice and a single-use case of purchase of stock, the request is going through different microservices and involving data of different microservices.
Here, you can see a single request is spanning over multiple microservices. When a single request expands over multiple services needs for the automaticity of the transaction and isolation of concurrent transaction is required. By the term automaticity of the transaction, means that the local transaction could either be successful or should not happen. It means if the transaction is failed, it would require to be rollback. By the term isolation of concurrent transaction, means it may happen a situation like a user is logging in by a laptop or by an app and from both the devices user is trying to purchase the same stocks or different stocks.
Usually, services expose APIs that are invoked by Saga. The transient state of the request process between microservices as I mentioned earlier, and the payment service let's imagine the transaction is going on and money is deducted. But due to some reason, the transaction comes to failure. In this case, your money is deducted but the transaction is still displaying failure in UI.
In this case, the amount that is deducted should be returned to the user. This transient state of the request processed between services needs to be persisted in case of failure also. Moreover, in case of failure, there should be a rollback.
These points all are related to data inconsistency. To maintain data consistency, we use the SAGA design pattern.
In Simple terms, SAGE design is nothing but a sequence of the local transaction. As I have mentioned earlier in the request for the purchase of online stock is going through four of the microservice. All the combination of those local transactions is called SAGE. In the Sage design pattern, we have each transaction to a particular microservice is treated as a Local transaction. So, we have here a sequence of local transactions and each local transaction updates data within a single service. The transaction could be sequential or parallel that means it can be synchronous or asynchronous. This design pattern is preferred for long-running Transactions.
If you have only two microservices or three microservices in your entire backend application, then the Saga design pattern is not a preferable approach. In case your request is going through five microservices, then data consistency can look at the SAGA design pattern.
There are different approaches for the implementation of SAGA design patterns but mostly two are used.
There are different approaches to the implementation of SAGA design patterns but, mostly two are used, which has been explained by the Spring Boot developers.
In Orchestration or command-based pattern or approach, the microservices do not directly interact with each other. Instead of that, we have a common orchestrator to perform such action. Again, taking the same case of online stock purchase. The user invokes the request for online purchase of stock it will go through UI service. UI service will generate a command purchaseStock(). Instead of directly going through the StockManagement service, the request will go through a central Orchestrator.
The orchestrator on receiving of purchaseStock() command will generate the command selectStock(), and this command will be listened to by the stock management service in the backend, and the stock service will reply, with the next command saying stockChoosen(), and again this will go to the orchestrator.
Upon receiving of stockChoosen() command, Orchestrator will further generate another command saying makePayment() and it will be listened to by your payment service. When the payment service will process the payment transaction then on success or failure of the transaction, it will generate a command called “paymentNotification()”. If the payment is successful and this response will be listened to by the Orchestrator and upon receiving this response, the orchestrator will further generate the next command called PaymentReceived() for the notification service. This payment confirmation will be listened to by the notification service and the notification service give a response to the orchestrator.
If you have a database, which is located in different locations we can do a transaction between two different databases or two different tables, which are located at two different databases. It is called distributed Transaction.
In java microservices, this is a very common scenario where distributed transaction takes place during the data flow through microservices. I have explained to this blog how to handle this using the Saga pattern in two ways with Java Saga code taking an example of Online Stock Purchase.
The saga design pattern is for data consistency in a microservice architecture. There are different approaches for implementation of SAGA design pattern but, mostly two are used one is an Event-based Approach & the other is an Orchestration-based Approach.
In Orchestration or command-based pattern or approach, the microservices do not directly interact with each other. Instead of that, we have a common orchestrator to perform such action. In the case of multiple microservices, this pattern is always the best.
First, you have to create a saga of the desired functionality. I have explained through an online stock purchase example where I have to create a saga of stock purchase and also initialized the state machine. Please go through the sample code for a better understanding.