MongoDB is a free and open-source, cross-platform document-oriented data store. It is a document database storing strongly typed objects with optional levels of schema validation. Ad hoc queries, indexing, and real time aggregation provide different ways to access and analyze the data. It is a distributed database at its core that provides high availability, horizontal scaling, and geographic distribution.
The software company 10gen began developing MongoDB in 2007 as a component of a planned platform as a service product. In 2009, the company pivoted to focus on maintaining MongoDB instead. It shifted to an open source development model under the AGPL, with the company offering commercial support and other services. In 2013, 10gen changed its name to MongoDB Inc. In 2016, MongoDB returned to being a data platform as a service product with the launch of MongoDB Atlas, and in 2018, it became a publicly listed company on the Nasdaq. MongoDB changed its licensing to the SSPL in 2018 a non OSI approved license, which specifically requires those who offer MongoDB database as a service (as opposed to as-a-service products built with MongoDB) to open source their infrastructure code.
MongoDB uses a combination of consistent checkpoints and a write ahead log known as the database journal. Every 60 seconds, a consistent version of the data is written to disk. Between these checkpoints, all changes are recorded in a write ahead log, allowing the database, (on recovery) to automatically roll forward from the last checkpoint to the last journal flush. Writes can wait for WAL (write ahead log) journal flush to ensure 100% durability.
Two-Phase Locking (Deadlock Prevention) Optimistic Concurrency Control (OCC)
Data in MongoDB is stored in the underlying WiredTiger storage engine. Whilst WiredTiger implements optimistic locking for concurrency control, the MongoDB databases operations above it use automatic retry to present pessimistic concurrency control for individual documents. Pessimistic locking also exists at the collection/table and database level for some administrative operations. MongoDB also includes traditional ACID transaction semantics for writes with begin, commit, and rollback commands. These can be used when an update needs to modify more than one record, although in a document model database, this is less common.
Fundamentally, indexes in MongoDB are similar to indexes in other database systems. MongoDB defines indexes at the collection level and supports indexes on any field or subfield of the documents in a MongoDB collection. MongoDB provides a number of different index types to support specific types of data and queries. The unique indexes will reject insertion or update with existing value. The compound indexes are useful for queries that specify multiple predicates. For fields that contain an array, each array value is stored as a separate index entry. Time to live indexes allow the user to specify a period of time, after which the data is automatically deleted from the database. Geospatial indexes optimize queries related to location data within a two-dimensional space. Partial indexes only index the documents in a collection that meet a specified filter expression. MongoDB uses b-tree for the data structure of the index. MongoDB also includes the ability to create a clustered index on the primary key of tables to organise the underlying record storage.
Read Uncommitted Read Committed Snapshot Isolation Repeatable Read
Reads from MongoDB are only of fully committed transactions by default, providing true ACID properties. However, the isolation level can be adjusted by setting the appropriate “read concern” level when performing read operations. The default is to show transactions fully committed that are persisted on the current primary node, with a read concern of "majority", meaning the transaction is replicated to a majority of high availability copies. “linearizable” read concern in MongoDB ensures that read operations reflect the latest committed state of the database, providing a linearizable view of the data.
MongoDB supports left outer joins via the keyword $lookup. $lookup performs a nested loop join with internal invariant caching. In document databases such as MongoDB, it is common practice to use a JOIN only for fairly rapidly changing dimensions and the additional document modeling features to reduce the number of joins required. For example, a repeating subgroup of attributes can be represented without need to use an additional table. Joins can also be performed using a combination of $unionWith and $group.
MongoDB maintains a fixed sized, on-disk circular buffer - called the oplog - of all logical database operations. This is used for replication and can also be observed to provide an event stream of database operation using the watch() command. The oplog is the only data structure that has write ahead logging enabled so in the event of failure, a Write Ahead Log (Journal) rolls the oplog forward from the latest full consistent disk snapshot. Then, the oplog operations are replayed on the data to bring the whole database up to sync with the point of failure. At this point, if the instance is part of a highly available replica set, then additional transactions are recovered from the current primary node and reconciled by rolling back to the last point of common truth and playing transactions to catch up. All this is fully automatic.
Queries in MongoDB are not parsed from text like an SQL query but are supplied from the client in the form of a binary object representation of the required operations and parameters. At its simplest, this could be described as query-by-example, although the object model allows for arbitrary complex comparison functions. These structured query objects are traversed at run-time to build a processing tree for the underlying query engine.
Within a single instance, MongoDB performs linear execution of simple retrievals but can perform parallel chained operations when using a pipeline query. When querying a collection (table) that has been partitioned across multiple hosts, each partition can be queried in parallel.
MongoDB by default is a disk-oriented database, with all reads and writes going via a database cache. There is no practical limit to database size, with only the most recently used elements being held in RAM in the cache and older data being auto-evicted as required. The database cache is configurable but typically approximately half the RAM on the server, with the rest being used for query process and OS tasks, such as the underlying OS page cache. MongoDB has the option of running as an in-memory database, in which case, the maximum storage capacity is equal to the configured database cache size - normally 50% of RAM - and can be adjusted to be larger.
MongoDB stores data by default on disk using the BTree/B+Tree hybrid WiredTiger storage format and engine. WiredTiger represents each BTree as a single file, with the file containing both a consistent data snapshot referenced from the root node and any blocks written to disk but not yet included in a consistent snapshot. Collections in MongoDB are organized as BTrees (technically a variant B+Tree where data lives in leaf nodes but there are no links explicitly between leaf nodes) where the key is an auto-assigned incremental key called an "Identity" and the value is a BSON document. Indexes are also WiredTiger trees, with the key being the index key and the value being the identity of the record it points to. Individual blocks in WiredTiger data files are compressed by default and can optionally be encrypted for protection at rest.
MongoDB stores BSON serialized objects (documents) in the database, making it a row-based system. It also has a Time Series option, where specific collections of individual row-like documents are converted to and stored in specially formatted hidden documents in a clustered, columnar format, providing greater compression and read performance for time series values.
MongoDB systems consist of replica sets: groups of identical instances hosted on different underlying servers, serving the same data and replicating continuously to ensure high availability as well as providing additional query capacity. Larger data sets can be range or hash partitioned over multiple replica sets in a process known as sharding. In a sharded cluster, the values of one or more fields known as the shard key determine which replica set hosts that specific record.
MongoDB supports read-only, non-materialized views, defined as an aggregation and a collection or view on which it will be run at query time. When a view is the source collection from another view, it allows us to execute a chained aggregation. Views are read-only. Write operations on views will error.