Improving Performance of AWS Serverless Applications with CQRS
CQRS (Command Query Responsibility Segregation) is a pattern designed to separate read and update operations for a data store. This pattern helps applications maximize performance, scalability, and complexity. This approach also allows the system to be more flexible as the system evolves by separating concerns.
Problem Statement
Performance, scalability, and complexity are critical problems that face applications when reading and writing data. Firstly, performance can suffer it you must wait for the data to be updated and returned in the same endpoint call. Secondly, scalability can become an issue if there is not an elegant way for the call to handle spikes in usage. Lastly, complexity can result if a simple CRUD(Create/Read/Update/Delete) model is implemented and business requirements including different query fields, business validation.
Solution Overview
This is where the CQRS design pattern can applications improve on performance, scalability, and complexity.
The CQRS design pattern involves separating read and write endpoint meaning that write endpoints do not return any data.
All write endpoints do not actually make the data updates to the end data store for which data is read from. The write endpoints will submit their data update request directly to an event-driven workflow which will eventually update the datastore that the readers read from. This makes the write calls quicker and the event-driven workflow service can scale more proficiently. Additionally, endpoints can be better tailored to specific business needs as CQRS does not limit itself to simply 4 CRUD endpoints
A sample AWS serverless CQRS architecture is presented below:
Background
The CQRS design pattern is an application of the Command-query separation principle developed by Bertrand Meyer as part of his pioneering work on the Eiffel programming language and the concept of Design by contract. In his paper, “Eiffel: programming for reusability and extendibility” (Meyer, 1987), states that “every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer”. When you think about CQRS and read Meyer’s statement it makes sense. Additionally, this also makes sense when you think about it in the context of Event-Driven Architecture.
Solution Review
Approaching queries and updates in the basic CRUD(Create Read Update Delete) approach can become over complicated. Reads often require many different types of query operations including different fields and conditions to return the required data. Writes can require complicated validation and business logic. Data contention can occur when many different operations are occurring in parallel on a given datastore.
Advantages
- Simplifying development by separating commands(updates) and queries(reads) (Reducing cognitive load)
- Independent Scalability for commands and queries
- Flexible approach which can evolve from a simple model to a more complex model as business rules evolve
- Enables datastore optimizations with respect to both reads and writes
- Security is simplified by clearer delineation between read and write operations simplifying the security aspects
- Loose coupling improves the maintainability
When CQRS might not be a good choice
- If a simple CRUD approach will suffice
- If business rules are simple
Considerations for CQRS
- Eventual Consistency should be acceptable
- Messaging is a key component of the CQRS implementation, handling message failures and retry scenarios should be considered
- With an event-driven model, care must be taken to ensure that the event model does not become to unwieldy or overcomplicated
Review of CQRS in Santa’s Workshop
Santa’s Workshop is a serverless CQRS Implementation written in Java and deployed with AWS CDK. Santa’s workshop demonstrates the separation of the query handling and command handling of the CQRS Pattern.
In this example, letters represent the letters children wish to submit to Santa with the list of toys and the amount that they wish to request. These are handled by submitting the request as an event. The client will receive a confirmation that the request was successfully received. The actual writing of the data to the database will not have been completed at the end of the request.
Santa’s Workshop also illustrates how CQRS can update multiple datastores with one call. The first data store is a single table design Letters database which helps him manage the toys he must deliver. The second PostgreSQL RDS data store is a relational database to do some analytics as a relational database lends itself to analytics queries. The event driven workflow implemented with SNS, SQS, and Lamba allows the Command model to update different datastores in a flexible manner.
Query operations are completely separated in that they do no data updates at all. Queries in this case are simply reading from a DynamoDb table and a PostgreSQL RDS instance. In this scenario, the DynamoDb is there to meet a higher performance use case and the RDS instance is there to meet a SQL query use case. This is an example of where the separation of concerns between read and writes allows more flexibility in system design.
Command operations submit the request and does not retrieve any data and only an acknowledgement that the update request has been successfully received. This means that the client can know that their request has been successfully received and know that the data will be updated in an eventual consistent manner. The Lambdas will scale independently to the events that they receive.
In this example, write operations are processed via some event routers including AWS SNS & SQS. Other event routers could include MSK, EventBridge, and Step Functions. Additionally, event producers and consumers are implemented as AWS Lambdas. Events could also be produced or consumed by AWS EKS microservices.
Obvious performance benefits for the Command(write) operations include:
- Improved response times as the data update operation is simply submitted and does not have to be executed during the call
- Improved scalability as the event driven system can dynamically scale to process each request. (Avoiding overloading/failures to better handle high usage periods)
The CQRS pattern is often associated with Event-Driven architecture and more sophisticated workflows can involve the Saga Pattern.
Conclusion
The CQRS (Command Query Responsibility Segregation) Pattern is a valuable pattern to help improve the performance, scalability, and security for database operations. Separation of concerns of writes and reads can better enable the system to deal with high usage periods.
Do not hesitate to reach out to us at Accolite Digital for more information about your organization’s cloud and digital transformation needs and how we can help you.
