QuestDB is an open-source time-series relational database. It uses a column-oriented approach and supports heavy parallelized vectorized execution using SIMD instructions. QuestDB implements a SQL interface and augments it with extensions simplifying the semantics of time-series queries. It supports high-throughput, schema-agnostic ingestion using the Postgres wire protocol and the InfluxDB line protocol. QuestDB is written from scratch in Java and C++ with no external dependencies and zero garbage collection.
QuestDB started as a side project in 2014 by Vlad Ilyushchenko who applied principles from low latency techniques found in the financial services industry. QuestDB is backed by YCombinator.
QuestDB uses a single writer model. The writer re-orders timestamps on the fly. Multi-publisher, single consumer queue provides concurrent write access.
QuestDB uses a relational model with column-based storage model.
SIMD optimised analytics.
Indexing is available for symbol columns. The symbol datatype in QuestDB is used to store repeatitive strings. Internally, it is stored as a dictionary of integers and corresponding string values. The Index support for other types will be added over time.
QuestDB uses the READ COMMITTED isolation level.
Nested Loop Join Hash Join Semi Join
QuestDB supports INNER, LEFT OUTER, and CROSS join types. FULL joins are not yet implemented and are on our roadmap. In addition, QuestDB also supports ASOF, LT, and SPLICE join types particularly useful for time-series analytics.
Joins in QuestDB are internally implemented as nested loop joins and hash joins.
Starting from version 7.0, QuestDB supported using WAL to ingest data. With the addition of WAL-supported tables, clients can write to disk independently without holding the lock of the table writer. Transaction numbers are provided by a central sequencer protecting a single source of truth. After applying the WALs, the data is applied to the table asynchronously by another job.
Multi threaded SQL execution
QuestDB uses a JIT compiler to improve performance by compiling the predicate evaluation part of the query. After determining whether a filter is suitable for compilation, the compiler frontend would transform the abstract syntax tree (AST) of the filter into an intermediate representation (IR). The IR is then processed by the compiler backend to emit vectorized machine code with the AVX2 instruction set.
QuestDB uses the vectorized execution model and takes advantage of SIMD operations for fast aggregations and predicate evaluation.
QuestDB has a SQL interface mostly compatible with the PostgreSQL dialect. It also extends the standard SQL with constructs simplifying the semantics of time-series queries (e.g.
QuestDB uses a disk-oriented storage architecture with memory-mapped file data access.
Decomposition Storage Model (Columnar)
QuestDB uses a column-based storage model. Data is stored in tables with each column stored in its own file and its own native format. New data is appended to the bottom of each column to allow data to be organically retrieved in the same order that it was ingested.
InfluxDB, PostgreSQL, Prometheus