Database

Database

With the growth of the user base, one server is not enough, and we need multiple servers: one for web/mobile traffic, the other for the database. Separating web/mobile traffic (web tier) and database (data tier) servers allows them to be scaled independently.

Which databases to use?

You can choose between a traditional relational database and a non-relational database. Let us examine their differences.

Relational databases are also called a relational database management system (RDBMS) or SQL database. The most popular ones are MySQL, Oracle database, PostgreSQL, etc.

Relational databases represent and store data in tables and rows. You can perform join operations using SQL across different database tables.

Non-Relational databases are also called NoSQL databases. Popular ones are CouchDB, Neo4j, Cassandra, HBase, Amazon DynamoDB, etc. Join operations are generally not supported in non-relational databases.

These databases are grouped into four categories:

• Key-value stores: a key maps to a value you fetch by key, nothing more. Examples: Redis, Memcached, DynamoDB. Used by GitHub and Stack Overflow for caching and session storage, since a lookup is one fast hash hit with no query planning.
• Document stores: each record is a JSON-like document (JSON, BSON, XML) you can also query by its fields. Examples: MongoDB, CouchDB. Used by The Guardian for its article store and by eBay for parts of its product catalog, since one document holds everything a page needs and there is no fixed schema to migrate every time a field is added.
• Column stores (wide-column): think of one giant table where every row is found by a row key, but unlike SQL each row decides its own columns. Row 1 might have name and email, row 2 might have 900 columns none of which row 1 has, and you never declared them upfront. Related columns are grouped into "column families" and stored together on disk, so reading just a few columns across billions of rows stays fast and empty cells cost nothing. This layout is built for very high write volume and queries you can plan in advance. Examples: Cassandra, HBase, Google Bigtable. Used by Discord for chat messages and Netflix for viewing history, since rows partitioned by a known key (channel id, user id) spread writes evenly across the cluster and any node can fail without an outage.
• Graph stores: data is nodes and edges, and queries walk the relationships directly. Examples: Neo4j, Amazon Neptune. Used by eBay to compute delivery routes and by NASA for a "lessons learned" knowledge graph, since "what connects to what, how many hops away" is answered by walking edges in milliseconds while the same query in SQL would be dozens of self-joins.

For most developers, relational databases are the best option because they have been around for over 40 years and historically, they have worked well. However, if relational databases are not suitable for your specific use cases, it is critical to explore beyond relational databases. Non-relational databases might be the right choice if:

• Your application requires super-low latency.
• Your data are unstructured, or you do not have any relational data.
• You only need to serialize and deserialize data (JSON, XML, YAML, etc.). "Serialize" means turning an object in your code into text you can store; "deserialize" is turning that text back into an object. If all your app does is save a whole object under some id and later fetch that exact object back, and it never asks questions like "find all users in Delhi" or "sum every order from March," then you are not really using what a relational database is for. A document or key-value store that just keeps the object as-is is simpler and faster.
• You need to store a massive amount of data.