Event-Driven Architecture 101
The basic concepts of EDA: events, event sourcing & CQRS, streaming, logging & monitoring, and implementation strategies
Event-Driven Architecture (EDA) is a trending software architecture pattern nowadays. Many organizations and systems adopt this pattern due to its scalability and flexibility. Let’s discuss the basics of events, event sourcing & CQRS, streaming, logging & monitoring and implementation strategies through this article.
Events
This is the cornerstone of event-driven architecture. The EDA has evolved from microservice architecture while it replaces legacy architectures such as monolith and SOA. The classic communication between services is command and query. The services either send commands or queries for data.
💡 Treating microservices as composable building blocks is key, and it’s made easier with a tool like Bit which allows your teams to independently publish, version, document, test, and share individual components such as functions, UI elements, or data models, that can be reused across multiple microservices.
Learn more here:
The main characteristics of this communication are
There are 3 main problems with command and query,
- Performance: Due to the synchronous nature
- Coupling: The calling service calls a specific service
- Scalability: The calling service calls a single instance of a service
These drawbacks can be overcome by using events. Event indicates that something happened in the system. The main features of event are
- Asynchronous
- Calling service has no idea who handles the event
- Never returns a response
- Something happened
There are two types of event data, complete and pointer.
Complete is the better approach since it makes the event completely autonomous while pointer can be used for large sets of data.
Components of EDA
There are three main components such as producer, channel and consumer. Let’s discuss the characteristics of those elements separately.
Producer
- The component/service sending the event, the producer sends the event to the Channel
- Often called as publisher
- Usually sends event reporting something the component done
- Exact method of calling the channel depends on the channel
- Usually using a dedicated SDK developed by the channel vendor
- Utilizes some kind of network call, usually with specialized ports and proprietary protocol
- There are many libraries based on several programming languages
Channel
- The most important component in the EDA, channel distributes the event to the Consumers.
- Responsible for distributing the events to the relevant parties
- The channel places the event in a specialized queue, often called
Topic or Fanout - Consumers listen to this queue and grab the event
- Implementation details vary wildly between channels
- The channel’s method of distribution varies between channels such as queue, REST API call or proprietary listener
Consumer
- The component that receives the event sent by the producer and distributed by the channel. The Consumer receives and processes the event
- Sometimes reports back when processing is complete (Ack)
- Consumer gets the event using either: push or pull
Orchestration and Choreography
These are two architectural styles which usually align with EDA.
Orchestration
- Flow of events in the system is determined by a central orchestrator
- Orchestrator receives output from components and calls the next component in the flow
- The next component sends the output back to the orchestrator.
Choreography
- No central knowing all component
- Each component notifies about the status of events
- Other components listen to the events and act accordingly
Event Sourcing and CQRS
Events can be used as the basic building blocks of data too. Event Sourcing and CQRS offer a pattern to store data as events. The issue with traditional databases is they only hold data about the current state of the entity. There is no way to see the historical data of entities. Simply data is a snapshot of a point in time. This will be solved by the event sourcing and CQRS.
Event Sourcing
- A data store pattern in which every change in the data is captured
and saved. - Database stores a list of changes for the entity, not the entity itself
- No updates or deletes, just inserts
- Every row documents a change in a property/properties of the entity
- In this pattern, the database is called Event Store
CQRS (Command and Query Responsibility Segregation)
- Separating the commands (updates/inserts/deletes) from the queries
- Each one of them is in a separate database
- Commands database is implemented as Event Store to improve
performance and simplicity - Queries database stores entities
- Database are synced using a central synchronization mechanism
This is recommended when access to historical data is extremely important.
Stateless and Stateful EDA
Related to consumer behaviour there are two main patterns in implementing EDA.
Stateless EDA
- Each event handled by a consumer is completely autonomous and is
not related to past/future events - Should be used when the event is an independent unit with its own
outcomes
Stateful EDA
- Events might be related to past/future events
- Should be used mainly for aggregators and time-related events
- Current state is stored in specific consumer(s).
- Should be used when events are part of a chain of events.
Example: Send an email if more than 5 failure events were received in a single minute.
The main drawbacks of this approach are load balancing and scalability. Since the state is stored in a specific consumer, subsequent events must be routed to the same consumer. No load balancing is possible and since the state is stored in a specific consumer, additional consumers cannot be added to handle the events.
Event Streaming
Event Streaming is another event-oriented pattern. The examples are
- E.g. Telemetry from sensors, system logs etc
- The events are published to a “stream”
- Events are retained in a stream for a specified amount of time
This is recommended when the system needs to handle stream of events from the outside such as sensor data. Apache Kafka is the most notable one for event streaming.
Implementing EDA
There are two main approaches for implement events.
- Events are retained
- Events are not retained
Retaining Events
- The channel retains the event for future handling
- A retention period is defined in which after it expires the event is removed
- Great for streaming events and when the channel is the source of truth
Not Retaining Events
- The channel publishes the events and does not store them
- If a consumer missed an event it can t be replayed
- Used mainly for in-system events
Most EDA systems are not pure EDA because UI Clients need responsiveness and use Web API to call the backend. If the client only asks for data, EDA will probably not work.
Synchronous EDA
EDA is asynchronous by nature. Normally the producer does not wait for a response to the event but sometimes it does, as a separate event. This is difficult to implement. To make the process easier you can create a wrapper around the producer. The wrapper exposes a synchronous Web API.
Saga Pattern
Transaction management in a distributed system is difficult. The Saga pattern strives to solve this problem. A sequence of service-scoped transactions, triggered by events. When a transaction fails, a compensating transaction is triggered.
Conclusion
Implementing Event Driven Architecture shouldn’t be too difficult. The most important part is the design therefore make sure to select the best tools for the tasks. Always try to use existing tools and cloud services do not reinvent the wheel. There are many libraries and tools to implement EDA with the support of different programming languages. Logging, monitoring and security are the most important factors that need to be addressed when you are designing an EDA.
From monolithic to composable software with Bit
Bit’s open-source tool help 250,000+ devs to build apps with components.
Turn any UI, feature, or page into a reusable component — and share it across your applications. It’s easier to collaborate and build faster.
Split apps into components to make app development easier, and enjoy the best experience for the workflows you want:
