Derby is a lightweight embedded relational database implemented completely in Java. It is an embedded database for any Java applications.
In 1997, Cloudscape Inc., a start-up in Oakland, California, developed a database engine called JBMS, which was later renamed as Cloudscape. From 1999 to 2001, Cloudscape was acquired first by Informix Software, and by IBM, and its name was changed to IBM Cloudscape. In 2004, IBM contributed the code to the Apache Software Foundation as Derby. The Apache DB project, supported by Apache Software Foundation, aims at creating and maintaining open-sourced, high-quality databases. In 2005, Derby exited the incubator and became a Apache DB subproject.
Two-Phase Locking (Deadlock Detection)
There are two scopes of locking (table-level and row-level), three types of locks (exclusive, shared and update) and four different types of transaction isolation levels. The locking strategies for different combination of scopes, lock types and isolation levels are different.
In general, although not explicitly stated, Derby implements strict two-phase locking. Exclusive locks will be held until a transaction aborts or commits; shared lock, instead, will be released after the reading of the rows finish (except for specific isolation levels) Derby also supports deadlock detection. When a deadlock is detected, the transaction that holds the least number of locks will be aborted.
Derby implements Write Ahead Logging (WAL) similar to the ARIES design. One of the differences is that instead of saving Log Sequence Number (LSN) in the page data, it saves the page version number in both the page data and the log record, and compare them during recovery.
Derby implements page-level physical logging. For queries that involves more than one pages, the operation will first be converted to loggable actions for each page involved. Then the loggable actions will be used to generate physical logging on that page.
Derby provides two types of join strategies -- nested loop and hash join. Nested loop join is more preferable in most cases. Hash join is preferred when inner table values are unique and outer table have many qualifying rows. Also, when the system estimates that the amount of memory required for hash join exceeds the amount available, nested loop will be used.
N-ary Storage Model (Row/Record)
Derby implements row-based storage model. Rows corresponds to records in data pages.
Read Uncommitted Read Committed Serializable Repeatable Read
Derby supports four level of isolation: serializable, repeatable read, read committed and read uncommitted for both table-level and row-level locking. The isolation levels can be set by either JDBC methods or SQL statement. One thing to highlight is that for both repeatable read and serializable, the entire table will be locked by either shared or exclusive lock depending on the statement, and the lock will only be released at the end of the transaction. Therefore, there is no phantom read under repeatable read.
Subqueries can only be materialized if they not correlate with outer queries, and return one row. For subqueries that cannot be flattened (DISTINCT), optimization can be made on subqueries such as using Hash Join.
Derby support Java stored procedures.
Derby implements standard B+ Tree algorithms with a few features:
It only uses exclusive latches on pages regardless of reading or modification of the page;
Node split is always left to right;
The system holds at most 2 latches simultaneously. During insertion, if there is no space for node splitting, all latches will be released, and Derby will do a split pass from top to bottom. After the split pass, Derby will redo the Insertion operation again.
Derby mainly support on-disk database. It also provides in-memory database for testing and developing applications. By following backup procedures, in-memory database can be stored and be used as either an in-memory database or normal on-disk database at a later time.
Derby supports fuzzy checkpointing, with slight variances from the ARIES implementation. Instead of storing active transaction table and dirty page table in checkpoint, it instead stores a few timestamps, which include the checkpoint start time and the earliest start time of ongoing transaction when the checkpoint starts. For example, if transaction T1, T2 and T3 are not finished when the checkpoint starts, then the earliest start time of the three will be recorded in the checkpoint.
When it comes to recovery, the system will first find the nearest checkpoint. Using the earliest start time of ongoing transactions, it will iterate through the log and find all the active transactions and dirty pages at the checkpoint, and redo or undo accordingly.
Foreign key is implemented as one of the CONSTRAINT clauses. There are two levels of CONSTRAINTS, column level and table level. Foreign key constraint in a column level enforces that the values in the column must corresponds to the values in the referenced column marked as primary key or unique key. Table level constraint works similarly, but it is for multiple columns.
Insert, update or delete instructions will return an error if the foreign key constraint is violated. The constraint check can be at statement execution or commit depending on the constraint mode.
Derby is a relational database that supports SQL syntax.
Code Generation JIT Compilation
Derby parses the prepared statement using Javacc and generates the Java binary code directly. JIT complier is supported, so that after several executions, JIT compiler will compile it to native code for performance improvement.
Using ij, Derby can also run ad-hoc statements. The exact compilation process is unclear.
Derby stores data and index in containers, which has a one-to-one mapping with files. Within each container, there will be three types of pages -- header page, data page and allocation page. Data pages hold data in row order.
JBMS, Cloudscape, Java DB