Onebridge Data Engineer Bradley Nielsen serves up insights andresources about the data analytics and BI world in every issue of the DataPlanet. He gets straight to the point to give you the real story.
NoSQL databases were invented around the turn of the decade to avoid the limitations of traditional relational databases (SQL Server, MySQL, PostgreSQL, Oracle).
Since then, the hype around them has waxed and waned. In this article, I’ll talk about two specific types of NoSQL databases and when and why they’re useful.
Document databases
Let’s start with document databases. In a traditional database, data is stored in tables with rows and columns. Each row has the same number of columns and the same kind of datatypes.
Document databases store rows as JSON-like objects. In theory, each row can have a completely different schema.
The obvious advantage is flexibility. Developers can add a new attribute without having to worry about modifying the structure of the database.
These databases also allow for complex data types like maps, arrays, and nested objects. This makes it far easier for applications to convert data into programmatic objects (known as Object Relational Mapping).
When should you use a document database?
Selecting the most appropriate database is one of the most difficult decisions an architect can make. Most traditional databases have since added JSON data types (like VARIANT in Snowflake) that allow them to mimic document databases, while also retaining the advantages of relational databases.
However, if you are dealing with truly massive scales (billions of rows), then the performance advantages of document databases might be appealing.
For example, document databases are very fitting for blogs and video platforms where content management is needed. Or they can be used for real-time analytics or statistics. Document databases also tend to be easier for software engineers to use as opposed to traditional databases.
Examples of NoSQL databases are Azure Cosmos DB, AWS Dynamo DB, Mongo DB, Couchbase. Here are further resources for more information.
Work with JSON inside Azure SQL Database
Semi-Structured Data inside Snowflake
Graph databases
In mathematical terms, a graph is composed of vertices and edges. Put more simply, graphs are about things and the relationships between those things.
The classic example is social media. A person has many friends, and those friends have many friends of their own. Often people will share mutual friends. In theory, in a traditional database you could have a person table and a relationship table. To get a person’s friends, you simply join the person to relationship table.
But problems start popping up when you need to probe the data, like, “Show me the friends of all my friends’” or “What is the shortest path between these two people?” Graph databases were invented to quickly answer those kind of questions.
When should you use a graph database?
Use a graph database when you have complex and unbound relationships between your entities.
For example, graph databases are a good choice for providing a personalized customer experience, for inventory catalog management, financial services, real-time fraud detection, or the Internet of things (IoT) and sensor data.
Relational databases obviously can handle relationships (it’s in the name, after all), but there are certain kinds of relationships that they struggle with. That’s when a graph database is a good alternative.
Some examples of graph databases are Neo4j, Azure Cosmos DB (graph mode), ArangoDB, Dgraph, and Azure SQL Database graph tables. Here is some further reading to help you learn more:
Introduction to Graph Databases
Graph Processing with Azure SQL DB & SQL Server
Conclusion
Over the last 10 years, the debate over relational vs NoSQL databases has been one of the most contentious arguments in the developer community. The comforting news is that most database platforms, whether relational or NoSQL, are stable and mature.
There are plenty of examples of successful software projects running on all kinds of databases. As with any software choice, the key is knowing your options, having a firm understanding of your requirements, and a good handle on the team’s capabilities.
Resist choosing more “exotic” options unless the requirements clearly call for it. New technology may be exciting, but be aware of the limitations.
