Implementing Saga Pattern in Microservices with Node.js
How to Implement Saga Pattern with Node.js
Traditional monolithic applications heavily depended on ACID properties to maintain data integrity. However, as applications became more complex, the shortcomings of this monolithic model became increasingly apparent. While the microservices architecture effectively tackled many of these limitations, it also introduced a significant challenge in managing transactions and ensuring data consistency across multiple independent databases and services.
Saga pattern offered a structured solution for addressing this challenge. It provides a systematic approach to managing transactions across multiple microservices. This resolves the intricacies of distributed transactions and seamlessly aligns with the microservices architecture principles, characterized by loose coupling and independent deployability of services.
What is Saga Pattern?
The Saga Pattern is a design pattern used to manage transactions and ensure data consistency across multiple services in a distributed system, especially in a microservices architecture. Unlike traditional monolithic applications where a single database transaction can handle consistency, microservices often use different databases, making it challenging to maintain data integrity across the entire system using standard ACID transactions. The Saga pattern addresses this by breaking a transaction into smaller, local transactions handled by different services.
There are 3 main components in the Saga pattern you need to know to understand how it works: local transactions, compensation transactions, and communication.
- Local Transactions — Each step in a business process is executed as a local transaction in its respective service.
- Compensation Transactions — If one of the local transactions fails, compensating transactions are triggered in the services where previous steps were successfully executed. These compensating transactions are essentially undo operations, ensuring the system returns to a consistent state.
- Communication — The services communicate with each other through messages or events. This can be synchronous or, more commonly, asynchronous using message queues or event buses. The Saga Execution Controller triggers these events in case a failure happens to ensure the system’s stability.
How to Implement Saga Pattern with Node.js
There are 2 approaches to implementing the Saga pattern: Choreography-Based Saga and Orchestration-Based Saga.
- Orchestration-Based Saga: A single orchestrator (arranger) manages all the transactions and directs services to execute local transactions.
- Choreography-Based Saga: All the services that are part of the distributed transaction publish a new event after completing their local transaction.
For this example, I will use the Choreography-Based Saga approach and a real-world hotel room reservation scenario with 3 microservices: Booking Service, Payment Service, and Notification Service.
- Booking Service: Starts the process by reserving a room. This is the first local transaction. Once successful, it sends a message to the Payment Service to process the payment.
- Payment Service: Receives the message and processes the payment. If the payment is successful, it commits its local transaction and informs the Booking Service and Notification Service.
- Notification Service: On receiving confirmation of successful payment, it sends a booking confirmation email to the user.
- Handling Failures: If the Payment Service encounters an issue (e.g., payment decline), it returns a failure message to the Booking Service. The Booking Service then executes a compensating transaction to cancel the room reservation, ensuring the system returns to its original consistent state.
Prerequisites
- Node.js project with necessary dependencies (express, amqplib, nodemailer, mongoose, dotenv) installed.
- RabbitMQ server running locally or remotely.
- Email server or service for the Notification Service (e.g., Nodemailer with SMTP or an email API service).
Step 1: Create an API Endpoint to Initiate Bookings
// booking-service.js
const express = require('express');
const amqp = require('amqplib');
const app = express();
app.use(express.json());
// Connect to RabbitMQ
const rabbitUrl = 'amqp://localhost';
let channel;
async function connectRabbitMQ() {
const connection = await amqp.connect(rabbitUrl);
channel = await connection.createChannel();
await channel.assertQueue('payment_queue');
}
// Booking endpoint to reserve a room
app.post('/book', async (req, res) => {
// Save booking to the database and attempt to reserve a roomconst
booking = { /* ... */ };
// ... booking logic
if (bookingReservedSuccessfully) {
await publishToQueue('payment_queue', booking);
return res.status(200).json({ message: 'Booking initiated', booking });
} else {
return res.status(500).json({ message: 'Booking failed' });
}
});
// Start the server and connect to RabbitMQ
const PORT = 3000;
app.listen(PORT, async () => {
console.log(Booking Service listening on port ${PORT} );
await connectRabbitMQ();
});This Booking Service handles HTTP POST requests to create a new booking. It attempts to reserve a room and, if successful, sends a message to the Payment Service via a RabbitMQ queue named ‘payment_queue’.
Step 2: Create an API Endpoint to Listen for Bookings and Process Payment
// payment-service.js
const amqp = require('amqplib');
const rabbitUrl = 'amqp://localhost';
let channel;
async function connectRabbitMQ() {
const connection = await amqp.connect(rabbitUrl);
channel = await connection.createChannel();
await channel.assertQueue('notification_queue');
await channel.assertQueue('compensation_queue');
channel.consume('payment_queue', async (msg) => {
const booking = JSON.parse(msg.content.toString());
// Insert logic to process payment
const paymentSuccess = true; // Replace with actual payment success condition
if (paymentSuccess) {
await channel.sendToQueue('notification_queue', Buffer.from(JSON.stringify(booking)));
} else {
await channel.sendToQueue('compensation_queue', Buffer.from(JSON.stringify(booking)));
}
channel.ack(msg);
});
}
connectRabbitMQ();The Payment Service listens for messages on the ‘payment_queue’. It processes the payment, and based on the outcome, sends a message to either the ‘notification_queue’ if the payment is successful or the ‘compensation_queue’ if the payment fails.
Step 3: Create an API Endpoint to Listen for Successful Payments and Send Email
// notification-service.js
const amqp = require('amqplib');
const nodemailer = require('nodemailer');
const rabbitUrl = 'amqp://localhost';
let channel;
async function connectRabbitMQ() {
const connection = await amqp.connect(rabbitUrl);
channel = await connection.createChannel();
await channel.assertQueue('notification_queue');
channel.consume('notification_queue', async (msg) => {
const booking = JSON.parse(msg.content.toString());
// Insert logic to send email notification to the user
console.log(Sending booking confirmation for bookingId: ${booking.id} );
// Setup nodemailer transport here
// Send email...
channel.ack(msg);
});
}
connectRabbitMQ();The Notification Service listens for messages on the ‘notification_queue’. When it receives a message, it sends an email confirmation to the customer.
Step 4: Create an Compensation Service to Handle Faliures
// compensation-service.js
const amqp = require('amqplib');
const rabbitUrl = 'amqp://localhost';
let channel;
async function connectRabbitMQ() {
const connection = await amqp.connect(rabbitUrl);
channel = await connection.createChannel();
await channel.assertQueue('compensation_queue');
channel.consume('compensation_queue', async (msg) => {
const booking = JSON.parse(msg.content.toString());
// Insert logic to cancel the booking
console.log(Compensating transaction: cancelling bookingId: ${booking.id} );
// Update booking status in database to 'CANCELLED' or similar...
channel.ack(msg);
});
}
connectRabbitMQ();The Compensation Service listens for messages on the ‘compensation_queue’. When it receives a message, indicating a payment failure, it performs a compensating transaction to cancel the booking and revert the system to a consistent state.
That’s it. You have sucessfully implemented 3 microservices with Saga pattern. Each service performs its tasks and communicates with other services through events.
What if I use Orchestration-Based Saga?
If you plan to use Orchestration-Based Saga instead of Choreography-Based Saga, you need to use a central coordinator to tell the participating services what local transactions to execute.
The key differences in an orchestration-based approach would be:
- A separate Orchestrator service would manage the overall process.
- Each service would communicate back with the Orchestrator after completing its local transaction.
- The Orchestrator would decide the next step and send commands to the appropriate service, including any compensating transactions required due to a failure.
The choice between choreography and orchestration often depends on the complexity of the business process, the degree of coupling you’re willing to accept between services, and the need for central control over the business transaction. Choreography is more decentralized and requires less setup, while orchestration can provide more control and is easier to manage and monitor for complex sagas.
Conclusion
The Saga pattern effectively addresses the complexities of distributed transactions in a microservices architecture, ensuring data consistency without compromising service independence. As microservices continue to dominate the software landscape, mastering the Saga pattern is an essential for building robust and resilient applications.
