Skip to main content

Example Architectures for Data-Intensive Applications


 

 ...a drawing should have no unnecessary lines and a machine no unnecessary parts. - William Strunk Jr., Elements of Style

In the book Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann has written about traits and trade-offs of elements of data infrastructure for modern applications. In that book, he has drawn "Figure 1-1. One possible architecture for a data system that combines several components". This is an exploration that example and possible architecture with Couchbase.

Figure 1 below is the architecture from the book. 

Sample architecture

Let's use a modern database like Couchbase which simplifies the infrastructure by providing scalable CacheDatabaseSearch, and Event processing within the same product. One bye one, we can replace each independent component with a scalable feature in Couchbase feature ( in red  ).

Modified architecture

Let's remove the unnecessary parts to get the new architecture.

Revised image

Example 2: I've added an additional common use case: data analysis and DataViz.

Adding data analysis

After using the same technique to use Couchbase and remove unnecessary components, we get this:

Adding Couchbase

Again, let's remove the unnecessary parts to get the new architecture. Here, we've exploited the Analytics service available within Couchbase.

Revised architecture

This doesn't mean everything can be simply rearchitected to use one or two products. The tendency of any infrastructure is to increase complexity. More components will increase complexity. This is a guide to continuously add, remove, and refactor the components to meet your business objective and to fight complexity.

Comments

Popular posts from this blog

Swami Vivekananda: The Monk That Nobody Sent to Chicago

  There’s a saying in Chicago: “We don’t want nobody that nobody sent.” This was the cold reception Swami Vivekananda faced when he arrived in the windy city in July 1893, determined to attend the World Parliament of Religions that September. He belonged to no organization, carried no letter of recommendation, his countrymen were nobody, and represented an alien religion to the Western world. As the days passed, his hope of attending the parliament dwindled. With money running out and the odds stacked against him, he left the Windy City and went to Boston, praying for a glimmer of opportunity.  Swamiji came to America to share India’s most profound gift: the wisdom of the Hindu sages, preserved through centuries of oral tradition and embodied by its monks. This was 1893, not 1993—India was under the British grip, its resources drained, and its spirit subdued. Swamiji’s mission was not just a cultural exchange; it was a bold step toward envisioning a future where India could re...

Why Should Databases Go Natural?

From search to CRM, applications are adopting natural language and intuitive interactions. Should databases follow? This article provides a strategic perspective. Amid the many technological evolutions in software and hardware (CISC/RISC, Internet, Cloud, and AI), one technology has endured:  Relational Database Systems   (RDBMS), aka SQL databases. For over 50 years, RDBMS has survived and thrived, overcoming many challenges. It has evolved and adopted beneficial features from emerging technologies like object-relational databases and now competes robustly with   NoSQL databases .  Today, RDBMS dominates the market, with four of the top five databases and seven of the top ten being relational. RDBMS has smartly borrowed ideas, like JSON support, from NoSQL, while NoSQL has also borrowed from RDBMS. NoSQL no longer rejects SQL. From a user perspective, all modern databases have SQL-inspired query language and a set of APIs. All applications manage the respective data...

Accelerating Insights With Couchbase Columnar

Insights come from observing and analyzing data from all sources. Couchbase Columnar helps quickly analyze data from a variety of data sources with zero-ETL. The purpose of computing is insight, not numbers. -  Richard Hamming Capella  Columnar  is an advanced real-time analytics database service from  Couchbase , targeted for real-time data processing, offering SQL++ for processing  JSON  (semi-structured) data and more. This service enables data to be managed locally and streamed continuously from both relational and NoSQL databases, or simply process data on S3. The columns or fields of the source are directly mapped to a field in the JSON document at the destination automatically. This is really a zero-ETL operation. A key feature of this system is its ability to continuously stream data, making it immediately available for querying, thus ensuring near real-time data processing.  At the heart of Columnar's architecture is a Massively Parallel Proce...