Sample Architecture using Amazon AWS, Microsoft Azure, Google GCP, MongoDB and Couchbase.

A drawing should have no unnecessary lines and a machine no unnecessary parts.

                William Strunk Jr., Elements of Style

In the book Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann has written about traits and trade-offs for data infrastructure while designing modern applications. He has given an example architecture for a data system that combines several components. I used this example for the article Example Architectures for Data-Intensive Applications. That article explored just the Couchbase features and functions.

A recent twitter thread talked about how AWS has many databases, but most “half baked” services (not my characterization) compete for the same business as DynamoDB!  Could this be a paradox of choice? Theory is, more choices do not usually lead to better outcomes. However, customers do vote with their dollars and cents. So far, they’re picking AWS for the potential savings in the short run. Could this lead to complexity and drive up the cost in future?

In this article, I explore how we can architect that example using products and components from various Cloud database vendors.

1. Amazon AWS

2. Microsoft Azure

3. Google GCP

4. MongoDB

5. Couchbase

Example reference architecture:

Amazon AWS

Amazon has choices and a reddit user has created a decision tree for choosing the databases! Here's that graphic.

[Source: Reddit]

Using this guide, let's look at the architecture to make our choices for various components.

For the core operational database, you can use DynamoDB (NoSQL), DocumentDB(NoSQL) or Aurora/RDS (RDBMS). Add either Elasticache or MemoryDB for caching. Use Amzaon Neptune or Elastic Search service (using opensearch). Or use Elastic itself directly. Either AWS SQS or Kafka provide queuing function for downstream processing. Amazon Redshit and Amazon Quicksight provide the data warehouse and BI function. You'll have to make choices about sizing and workload about each one of them and find a way to stich them together either by built-in functionality, connectors and customer application code. The lines from the reference architecture diagram are intact. Choice does address your need even if you have to stitch them together.

Microsoft Azure

Microsoft has a grown from a single (Sybase) SQL Server on OS/2 in 90s plethora of product and services on Azure. This page gives you the Azure databases list and matches them with the right workload requirements. Just like AWS, the choices are plenty.  Interestingly, they don't list the "search" in the database category.  This is typical of database vendors even though most applications that use databases require search. Microsoft has the Azure cognitive search service, not tightly integrated with Cosmos or other services.  You also have the option of deploying Elasticsearch itself.

In the end, this is same as to AWS diagram above, replaced with Microsoft Azure products.

Google GCP

Similar to AWS and Azure, GCP has plenty of choices for databases from Cloud SQL to Cloud Spanner, Memorystore to Firestore, and Oracle to Bigtable. 

Different Cloud, same result. You can pick up from a number of databases, components for cache, queues and search from GCP and others.  The architecture is same and challenges are the same you saw with AWS native solutions.

MongoDB

MongoDB is a scale-out, highly available documented oriented database that meets a number of requirements for a modern database. MongoDB provides simple get/set API, complex aggregation framework and distributed transactions. MongoDB reduces the number of components because it has built-in MongoDB Realm functions which can execute serverside logic.  You'd still need a caching layer to meet low latency performance objectives,  While the product is advertised to have built-in text search, its functionality is limited.  So are its capability for complex analytical queries. 

MongoDB Atlas

This improves the situation because it has a Atlas search. Atlas search is built using Apache Lucene. This search capability is only available on Atlas and not on within the MongoDB product.

Couchbase and Couchbase Capella

Couchbase is a shared-nothing, scale-out, highly available, high performance document oriented NoSQL database that also meets a number of requirements for a modern database. Couchbase provides simple get/set API, SQL inspired N1QL for both operational and analytical query processing, full-text search, and event processing. It's distributed and supports distributed transactions both for document get/set APIs and N1QL and their combinations. Couchbase FTS is built from the ground up and uses the bleve search engine. Couchbase Analytics is an MPP database engine designed for analytics.  And finally, it's the same core product on-prem and on-cloud. 

SUMMARY

On-prem and on-cloud, there are plenty of choices for databases to design data intensive applications. Each architect and company have different philosophies: Best of the breed or integrated solution? The answer, of course, is "it depends".  Even in this simple example, each database may have optimizations and would be good to hear from respective experts.