Relational & NoSQL Databases

Many startups rely solely on NoSQL databases, assuming they are a one-size-fits-all solution for all their needs. However, even good products can fail if not designed and executed properly. Reading books and following best practices can only take you so far; true expertise comes from experience. Consultants are often focused on optimizing solutions to maximize profits for their companies.

The optimal path forward is rarely as clear-cut as "use relational databases nowhere" or "move to NoSQL everywhere." Finding the best solution often requires a nuanced understanding of application design and architecture, gained through extensive experience. That's why Arc offers architectural blueprints as part of our solution. There is no single silver bullet that works for every situation.

At our company, we don't have a bias against any particular technology. We believe that every technology has its strengths and weaknesses, and it's important to use them appropriately. By choosing the right building blocks from the start, you can ensure that things run smoothly even as your company grows to unicorn status. Otherwise, growth will lead to constant stumbling, and you may have to sacrifice new features because of a massive technical backlog.

Relational databases excel in certain areas, such as referential integrity and support for atomicity, consistency, isolation, and durability - all of which solve important problems and are still necessary for most datasets. The issue arises when people try to use relational databases for everything, even when it's not the best fit.

Relational databases are incredibly valuable for storing data and maintaining its integrity, which is crucial. However, today's data is often messy and requires extensive cleaning. As a result, data scientists spend most of their time cleaning data rather than performing their actual job.

While relational databases are powerful, they do come with their own set of issues. For example, the schema can be difficult to maintain, and it can be challenging to write a Data Access Layer, even with frameworks like Entity Framework. As usage grows, maintaining performance can become costly, and hiring DBAs is not cheap. However, even NoSQL databases are not a complete solution.

NoSQL databases do address some of these issues, but they do so by compromising on key areas such as referential integrity, in order to improve in other important areas. Today, we have a situation where NoSQL databases are attempting to provide some of the features of relational databases, while caching services are trying to become NoSQL databases.

We believe that every software solution should focus on its own strengths and integrate with other technologies that excel in their respective areas. Rather than relying on a one-size-fits-all approach, it's important to carefully select the right tools for each task to ensure optimal performance and efficiency.

Our recommendation is to adopt a hybrid architecture that combines the strengths of both relational and NoSQL databases. Specifically, we suggest storing primary entities and their strong relationships within a relational database, while keeping all metadata related to these entities within a NoSQL database. This approach allows you to maintain data integrity while also taking advantage of the benefits that NoSQL databases offer. It also ensures that each database is being used for its intended purpose, rather than trying to force NoSQL to handle tasks that require relational database features.

By adopting a hybrid architecture, you can leverage the strengths of both types of databases to create a solution that is optimized for your specific needs. You'll be able to store and retrieve data efficiently, maintain data integrity, and scale effectively as your business grows. Additionally, by keeping metadata separate from primary data, you can further optimize performance and simplify data management.

At the heart of our application architecture is the Fusion Core Framework, which is designed to simplify the development of your Data Access Layer. Our framework builds on top of Entity Framework, adding an advanced layer that streamlines development and enhances performance.

One of the key features of the Fusion Core DAL is its deep integration with caching. By leveraging caching, we are able to bypass many of the performance and architectural issues associated with relational databases, allowing you to work directly with objects instead of SQL query results. With a Fusion Core DAL, your business logic is no longer tied to SQL queries, which require constant and expensive tuning to maintain performance. Instead, a significant portion of the query load is handled by the cache, which reduces the load on your relational database and improves overall performance.

Because the Fusion Core DAL reduces the amount of time and resources required for query tuning, your relational database can remain small and focus primarily on data updates. This, in turn, reduces the need for expensive DBA tuning and allows you to allocate resources more efficiently. With the power and scalability of caching, combined with the ease of use and performance of our Fusion Core Framework, you'll be able to create highly efficient and effective solutions that meet the needs of your business.