ActivePivot is a distributed, in-memory DBMS designed for HTAP workloads, which supports real-time data streams and continuous queries. It uses a columnar storage architecture with dictionary compression and binary representation of Java objects. The system can also materialize OLAP cubes, constructed with compressed columns, for analyzing multidimensional data.
ActivePivot is one of the core products of the French company ActiveViam -- once known as Quartet FS, which was founded in 2005. The companies primary customers are banks and other financial institutions, so there is an emphasis on support for real-time data streams and continuous queries.
ActivePivot supports queries through Multidimensional Expressions (MDX) either through ActiveUI or any another MDX-compliant interface.
The system supports materialized views for both OLAP cubes and continuous queries. Views can be materialized on a subset of dimensions for the OLAP cube. For continuous queries, the materialized view of the result is updated alongside changes to the underlying data.
Multi-version Concurrency Control (MVCC)
The system allows for multiple versions of a database using delta storage to keep track of different versions. For example, there can be one version that continuously gathers incoming data, and another version from an earlier point in time for analysis.
Decomposition Storage Model (Columnar)
Data is stored in columns in memory, DRAM or flash, with no additional row-based storage.
ActivePivot uses bitmap indexing for multidimensional cubes and supports primary and secondary indexes for relational tables.
The system takes advantage of non-uniform memory access (NUMA) by partitioning the tables based upon the number of CPU cores available on servers.
Both relational tables and multidimensional OLAP cubes are used for organizing the data.
All servers act independently and do not have access to the information contained on other servers. Data can be distributed between servers based on the values of a given attribute. For example, if one of the attributes is city then the data can be distributed based on geographic regions.
The Java JIT compiler is responsible for handling the compilation of Multidimensional Expression (MDX) queries. MDX queries are similar in syntax to SQL queries, but they serve a different purpose. While SQL queries operate on tables which are two dimensional, MDX queries operate on OLAP cubes, which can support many dimensions. Some other operations are compiled directly into Java byte code.
ActivePivot uses both dictionary encoding and Java primitives for compression.
Dictionary Compression: When there is a small number of distinct values for an attribute, the attributes can be encoded into an integer from 1 to N where N denotes the total number of distinct values by using a dictionary that stores the distinct values.
Java Primitives: Instead of storing integers as Java Integer objects, ActivePivot stores these as primitives. The number of bits stored for each number is based on the range of possible values. For example, if the values in a column are given to be between 0 and 256 then the system would only store 8 bits per number. This form of compression can be combined with dictionary compression
ActivePivot provides support for multi-core processing and non-uniform memory access architectures. SIMD instructions can be used in accelerating operations such as column scans and aggregations.
When a server receives a query, it also sends the query out to the other servers. Once it receives a response from all the servers it aggregates all of the information and returns the result.