Kdb+

Kdb+ is a column-based relational time series database (TSDB) with in-memory (IMDB) abilities, developed by Kx Systems. The goal of Kdb+ database is to serve as a central repository to store time series data and support real-time analysis for large amount of data.

History

Kx Systems originaly released kdb in 1998. It was a database built on the K programming language developed by Arthur Whitney. In 2003, Kx Systems released kdb+ as a 64-bit version of kdb.

System Architecture

Shared-Nothing

Kdb+ uses Lambda architecture has the following properties:
- Data currently using stores in memory, while historical data is stored on disk.

  • New data come in from streaming sources.

  • Event-engine distribute data to downstream subscribers, including real-time database engine and streaming query engine.

  • Real-time database projects its content down to on-disk historical database for analytic use at daily basis, controlled by event-engine.

Logging

Physical Logging

Kdb+ uses physical logging. When new data comes from sources, in-memory event-engine will log to on-disk log file to ensure durability.

Hardware Acceleration

Custom

Kdb+ doesn't rely on any hardware acceleration.

Indexes

B+Tree

Kdb+ supports secondary indexes.

Storage Architecture

Hybrid

Kdb+ has both in-memory and on-disk storage. New data is held in memory and is flushed to disk at daily basis under control of the event-engine.

Storage Model

Decomposition Storage Model (Columnar)

Kdb+ uses DSM both for in-memory and on-disk storage.

Data Model

Relational

Kdb+ uses relational model.

Stored Procedures

Supported

Kdb+ supports UDF.

Compression

Naïve (Record-Level)

Kdb+ supports compression in following ways:

  • A built-in WebSocket server which supports WebSocket compression standard.

  • In-flight compression between hosts.

  • On-disk compression, which supports 4 algorithms:

  • kdb+ algorithm

  • gzip

  • Google Snappy

  • lzh4c

Isolation Levels

Serializable

Kdb+ only supports SERIALIZABLE isolation level.

Query Interface

SQL HTTP / REST

Kdb+ supports following interfaces:

  • ODBC/JDBC

  • C/C++

  • Java

  • C#

  • Python

  • R

  • Kafka

Kdb+ Logo
Website

http://kx.com/products.php

Source Code

https://github.com/KxSystems/kdb

Tech Docs

https://code.kx.com/

Developer

Kx Systems

Country of Origin

US

Start Year

1998

Former Name

kdb

Acquired By

First Derivatives plc

Project Type

Commercial

Supported languages

C, C#, C++, Go, Java, JavaScript, Lua, Matlab, Perl, PHP, Python, R, Scala

Operating Systems

Linux, OS X, Solaris, Windows

Licenses

Proprietary

Wikipedia

https://en.wikipedia.org/wiki/Kdb%2B