ARRAY STORAGE ARCHITECTURE WITH COMPACT INTEGRATED INDEX

Abstract

Increasing the amount of data generated daily requires effective storage, a quick request for such data. Typically, such data are multidimensional and can be presented using the array data model. Along with the addition of more and more powerful processors and accelerators, most modern computing systems contain an increasingly complex input stack, ranging from traditional disk file systems and ending with heterogeneous accelerators with individual memory space. Effective access to such a complex input stack when processing arrays is important for the use of high computing capacity of modern computing platforms. One of the key to achieving such efficiency is to determine the place of generation or storage of data, as well as the corresponding selection of appropriate representation and processing strategies.

In this work, it is focused on optimizing the processing of arrays in such complex stacks of input-output by studying two fundamental questions: what kind of data should be used, and where the data should be stored and processed. Thus, the problem of efficient processing of data of the array is considered, a compact storage facility for disk data is presented, integrating it in its indexation based on values without loss.

The architecture of the storage system of arrays with integrated support of the value index has been developed. It achieves the reorganization of the elements into a series of custom bits and the effective coding of indices according to the bits and the corresponding values. This generates an indexed presentation of the array, which adds little additional storage costs.

The areas of further research are to improve the architecture of the storage system of arrays with integrated maintenance of the value index regarding the optimization of storage of indexes.

The experiments confirm the possibility of practical implementation of the proposed architectural decisions.