Demystifying Object-based Big Data Storage Systems

Abstract

Today’s era is the digitized era. Managing such generated big data is an important factor for data scientists. Day by day, it increases the demand for big data storage systems. Different organizations are involved in providing storage-related services. They follow the different architectures or storage models for storing big data. In this survey paper, our target is to highlight such storage architectures which provided by different renowned storage service providers. On an architectural basis, we divide the big data storage systems into five parts, Distributed file systems (DFS), Clustered File Systems (CFS), Cloud Storage, Archive Storage, and Object Storage Systems (OSS). Also, we reveal a detailed architectural view of the big data storage systems provided by the different organizations under these parts.

1 Introduction

Figure 1 shows the interest in Cloud, Object, File, Archive, and Block storages (y-axis, in percentage share) for some of the selected countries across the globe. We have selected the top 59 countries (x-axis) such as the United States, Australia, India, Canada, China, United Kingdom, Russia, and more on the basis of their search trend survey. Figure 1 clearly shows the popularity of Cloud storage over the other discussed storage systems.

Figure 1: Country-wise plot of the popularity of storage systems as retrieved from Google Trends. (The readers are requested to view the online version of this colored image.)

Cloud storage is a service hosted by cloud providers such as Infrastructure-as-a-Service (IaaS), and Storage-as-a-Service (STaaS). Data can be stored in logical pools built atop physical storage. The physical storage is not actually hosted in one machine/server, rather, it can span multiple servers. Physical storage is virtualized for efficient usage and management of huge amounts of data. Generally, when we talk about cloud storage, we refer to object storage hosted by cloud providers across participating servers. Although other types of storage (such as block storage) are also being hosted by cloud providers as STaaS and used for Compute-as-a-Service, these are not as beneficial as object storage systems from the perspective of cloud services and Big un-structured data.

The concept of cloud storage, centered around distributed resources, is suitable for implementation using object storage due to its features such as abstraction (from application and users), durability (through versioned copies), and fault-tolerance (via distribution and redundancy). Object storage in the cloud is preferred due to its cost-effectiveness and reliability.

Motivation and Uniqueness of this survey:

The performance analysis between the storage systems always has been a debatable issue as present in [16, 27, 33]. In [33], performance analysis between cloud computing storages (distributed/scale out cluster file-systems) Compuverde [33], Gluster [33] and Swift [33] has been discussed. In every case mentioned above, architectures have been explained in detail and main focus is on performance which is compared on the basis of read, write and delete operations. The choice of cloud storages is purely based on allocation or data distribution technique i.e., Compuverde uses Multicasting for Distribution, Gluster [33] and Swift [33] uses Distributed Hash Algorithm (DHT). However, emphasis on architectural characteristics comparison between these storages could have been an interesting point to discuss.

To maintain the increasing popularity of Cloud Storage systems and services based on Object Storage architectures, different storage service providers have launched their products. Each of these has different architectures based on their supporting properties. For example, the target of the Amazon Storage service provider is to support the storage data volume and telnet service request handling. For this purpose, they launch different storage platforms: (Simple Storage Service (S3), Elastic Block Storage (EBS), and Elastic File System (EFS)). The focus of the Microsoft vendors is to provide the storage platforms (Blob storage, Table storage, Queue storage, and File storage) based on the application demands. Without the knowledge of architectures, storage service consumers will not be able to consume the needed storage system. Therefore, our aim is to point out the architectures of diverse commercial cloud storage systems which will create awareness for the storage service consumers as well.

Organization of the survey: This paper is organized as shown in Figure 2, it represents the section headings of this survey article. This article begins with a background study (Section 2) followed by the mechanism of object storage management. In the next part, Section 3 discuss about the architecture of Big data storage systems included with distributed file system, clustered file system, cloud storage, archive storage, and object storage systems. At the last Section 4, concludes this article with referring the overview of this article.

for tree=rounded corners, draw, align=center, top color=white, bottom color=blue!20, edge+=-¿, l sep’+=1pt, baseline, qtree [Organization of the article [Section 1
Introduction ] [Section 2
Background Knowledge ] [Section 3
Classification ] [Section 4
Conclusion ] ]

Figure 2: Organization of the article

2 Background Knowledge

For storing data as objects in locations and retrieving the same most efficiently, Object Storage Device or OSD [14] performs the task of management of some services [10]. The services are discussed below in the following.

• Management of free space: For efficient storage of objects, free space information must be maintained properly. Also, proper data structures must be used to hold objects. These are two basic functions of any storage system. In a typical object storage system, we obtain discrete units called objects by breaking the data into pieces. Then it is kept in a single repository, unlike a file storage system where it is kept as files in folders or as blocks on servers in a typical block storage system. Object storage system architecture resembles a flat structure where files (the actual data) are broken into pieces (objects) and spread out among hardware. All user objects are mapped to files (not like we see in a file storage system). Attributes of root objects, partition objects, and user objects are also stored as files but in a flat structure unlike hierarchical structure of a conventional file storage system. However, some of the stores like Ceph Bluestore and IBM Cloud Object Storage goes for block device rather than file system.

• Security: Security is another main functionality of OSD. As shown in the Figure 3, in order to access the objects, clients or users must request for credential information from the security manager. The security manager is structurally composed of modules which maintains the credential information. This information is shared with an OSD via a capability key. Thus, the OSD check whether a user has requisite right to perform a particular operation.

  

  Figure 3: Security mechanism for storage and management of Object Data.

• Command Interpreter: Command Interpreter is responsible for converting object command to a format that is used for storage system that lies underneath [10]. Actually, it converts a command to a file system call. The stages of execution of this interpretation process (as given in Figure 4), is explained in the following.

  

  Figure 4: Stages in execution of Command Interpreter. (Object Data.)

  • Verification: At first, the Interpreter verifies whether the command has right to access and execute on the object.

  • Pre-processing: Every command will either get or set the attribute of the target object. The pre-processing stage is to check the type of command from a group and decide whether to get or set attribute [10].

  • Execution: Actual processing of command on the target object takes place in this stage. The object command is translated to storage-system specific system call. For example, if an OSD command is OSD WRITE and the underlying storage system is a file, then the command is translated to write() system call [10].

  • Post-processing: This is the stage where final update process takes place and all attributes related to the command are changed. One simple example is during post-processing of write operation, the timestamp of the file related to that object is updated.

3 Taxonomy of Big Data Storage System

for tree= rounded corners, draw, align=center, top color=white, bottom color=blue!20, edge+=-¿, l sep’+=1pt, , baseline, qtree [Taxonomy of Big Data Storage System [Distributed File Systems (DFS)] [Clustered File Systems (CFS)] [Cloud Storage] [Archive Storage] [Object Storage Systems (OSS)] ]

Figure 5: Taxonomy of Big Data Storage System.

3.1 Cloud Storage

As briefly introduced, the cloud storage provides a platform which abstracts the object storage architecture at back-end. There are many cloud providers, but few utilize object-storage services. The data can be accessed via graphical user interface or REST API [23]. Also various JAVA libraries can be utilized to access the objects in the cloud. Amazon S3 [4, 5, 16], Mezeo Cloud storage [35], EMC Atmos [27, 42], EMC ECS [12, 11, 21] and Eucalyptus Walrus [41] belong to this category (Table 1).

Table 1: Summaries of some Cloud Storage type of Object-based storage.

Name of object storage	Summarized architectural components
Amazon S3 [4, 5, 16]	Here Buckets are created which are containers of data. Every object is stored in a bucket. Object= Metadata + Object Data. The metadata comprises name and corresponding value that describes the object.
Mezeo Cloud [35]	The Mezeo Cloud architecture comprises 3 layers: Top Layer (Requests are handled by this layer via API’s and REST Web Services), Middle Layer (Provides services like allowing data to be stored in 1 or more files) and Lower Layer (It is the actual storage of data) and the cloud platform
EMC Atmos [27, 42]	EMC Atmos is a cloud platform by EMC Corporation which also gives support file-based services like Network File System (NFS) to access data. An important feature of Atmos is multi-tenancy which implies to the system being hosted in one or more user site which are accessed by a number of tenants or users.
EMC ECS [12, 11, 21]	Buckets are the actual containers of data, which are distributed across sites. Important feature is providing mechanism of geo-caching which identifies the location of the site requiring maximum access to data and caching there.
Eucalyptus Walrus [41]	Walrus is the object storage service of Eucalyptus in which users store data in buckets and objects. Walrus provides an interface for manipulation of buckets.

3.1.1 Amazon Simple Storage System (S3)

Amazon S3 is a simple implementation of Web Service [5] which allows data storage in form of objects [16]. Buckets are created which are containers of data. Proper authentication and access control mechanisms are also present for container and data management. Components of Amazon S3 (as depicted in figure 6) include the following.

Figure 6: Operation of Amazon S3.

• Bucket: Every object is stored in a bucket. For example if the object is abc.jpg which belongs to alpha bucket, the image is addressable using URL http://alpha.s3.amazonaws.com/abc.jpg. Functions of bucket include defining the main storage and the corresponding path that leads to an object. Buckets also provide mechanism for access control. It can be configured to a specified region. When an object is added, a unique version ID is associated with it.

• Object: For Amazon S3, object is the basic unit of storage. Object = Metadata + Object Data. The Metadata comprises name and corresponding value that describes the object. Standard information like last data modified are present as default Metadata. An object is identified within a bucket by one key name and unique version ID.

• Key: Key name is unique for an object. Key is used to identify an object in basket. Bucket + key +version ID = identifier of the object.

• Region: Regions to store buckets are chosen to improve performance and minimize cost. For instance, some object data is of specific usage to only banks in Japan. Hence, data is stored in Asia Pacific Region to minimize network latency. Amazon S3 supports some region like Asia Pacific-Tokyo Region and EU-Frankfurt region. Objects stored in a region cannot move or exit from a region unless explicitly transferred.

• REST API: REST API uses standard HTTP request for storing, retrieval or deleting bucket or object. We can use any tool that supports HTTP, or even browsers to access objects.

• SOAP API: New Amazon Features cannot be supported for SOAP API. But it can be still used with HTTPS.

Amazon S3 has been utilized to provide web services and in media and health care industry as well.

3.1.2 Mezeo Cloud Storage

Mezeo Cloud storage provides a platform to transform any traditional storage to scalable cloud storage. Mezeo File Client ensures availability of data. It provides solution to the current data management problems, by means of Mezeo Cloud and Mezeo File Solution [35]. Mezeo Cloud is deployed on LINUX based server and it follows a scalable Metadata storage. The Metadata is assigned to every node in the cluster. Hence, fast access to the data is possible without actually disturbing the storage. Along with the object or file, the Metadata is created. The information about the file is maintained in a distributed file catalog [35]. To ensure more availability, file catalog and Metadata should be separated from actual file location. The Mezeo cloud architecture comprises 3 layers [35] as follows.

• Top Layer: Top Layer is the access layer. Requests can be handled by this layer via API’s and REST Web Services.

• Middle Layer: It provides services like allowing data to be stored in 1 or more files. It helps in data replication. The Storage connector provides advance features in addition to forming a bridge to the storage. Encryption of data can be done by Mezeo or at the storage level. Multiple connector modules are present which are supported by single cloud. They allow organizations to convert any type of storage to object store.

• Lower Layer: It is the actual storage of data. Data is stored in containers.

• Mezeo Cloud Storage Platform: Provides easier access to Mezeo Cloud Storage and provides a platform for development of mobile applications. API is built primarily on three concepts: Resource, Representation, Method. The resources are actually data objects which are uniquely identified by URL. The object may be represented in a format such as XML/JSON. The representation is in standard form which is identifiable by storage. Methods include the creation or update or deletion of an object.

Mezeo Cloud storage provides various features which makes it popular [35]. The features are listed as follows.

Authentication: Mezeo Cloud incorporates authentication systems and they can utilize the advantages of CDMI (Cloud Data Management Interface) for proper functioning of the environment. CDMI refers to the interface that can be used for management of data in the cloud. The client can also utilize the interface to manage containers and data

Multi-tenancy: Each vendor uses separate storage providing physical separation of data. Mezeo Cloud hosts various storage by means of consolidated interface.

Flexible Integration Points: Mezeo Interoperability API (IOP) provides access to applications used for storage systems like Amazon S3. IOP provides the combination of backup facilities like Asigra , Commvault [40].

Rapid Scalability: The stateless architecture of Mezeo Cloud ensures scalability and accounts, data can be stored dynamically.

User Services: The services provided include uploading files of any type, offering real time access to it. Also, permission is granted only to specified role. The access to shared data expires with time for security. Mezeo Cloud storage can provide archival solutions. It can also be utilized for web services.

3.1.3 EMC Atmos

EMC Atmos is a cloud storage service platform providing object based storage for storing Peta-bytes of data [27]. Combining the advantages of Atmos, third generation of storage called EMC Elastic cloud storage has been developed, which will be discussed later.

EMC Atmos is a cloud platform by EMC Corporation which provides unstructured data storage in the form of objects [27]. The objects can be accessed via web services like REST API. It also gives support to file-based services like Network File System (NFS) to access data.

An important feature of Atmos is Multi-tenancy which implies the system is hosted in one or more user sites which are accessed by a number of tenants or users. These users may in turn provide services to sub-tenants to continue the chain. Whatever the depth of the chain, each user is identified by user_id. The storage system is also highly scalable i.e., additional nodes may be added to the existing rack or new rack may be introduced to add new nodes.

Data protection in Atmos is maintained by a number of mechanisms. For instance, a decision is given whether an acknowledgment will be sent after a successful write is done. This is known as Geo Mirror technique of data protection [27]. Another procedure is the decision to split the data into code and place data fragments at different distributed nodes. Accumulation of all fragments will give the resultant data. This technique is known as Geo Parity techniques [27]. The policy manager is in charge of these data management. EMC Atmos is widely used in health care industry.

3.1.4 EMC Elastic Cloud Storage (ECS)

EMC ECS provides software defined storage in accordance with the huge scaling requirement of the cloud [21]. ECS provides low cost access to public cloud with minimum risk. The features include the following [21].

• Global Repository: ECS urges the vendors to consolidate multiple storage system into a single global repository on which numerous applications run. It can be accessed anywhere. Buckets are the actual containers of data, which are distributed across sites. The architecture ensures proper replication and availability. There are certain consistency mechanisms provided by ECS which ensures consistency among replicas. Important feature is providing mechanism of Geo-caching which identifies the location of the site requiring maximum access to data and caching there.

• Maintenance of Data Lake: Presence of centralized data lake which allows data to be stored in original format by maintaining semantics. Storage and processing is managed by Hadoop Architecture .

• Multi-protocol API accessible Storage: Allows organizations to store huge amount of unstructured data as objects. ECS supports standard API’s like REST API.

• Support for Multi-tenant Platform: Ideal to support multi-tenancy. Reporting is enabled to provide object count, object creation and storage utilization. Quota features ensure locking of buckets. The creation and deletion of bucket is transparent to administrators via REST API.

• IoT Cloud Storage: Provides storage to huge volume of data coming from intelligent devices. The end-to-end view of processing of storage system and Geo-caching scheme makes it suitable for IoT platforms as well as storage.

• Cloud Scale Storage: Refers to dynamic scaling feature to incorporate huge storage. It is ideal to support cloud and Big Data applications for storing mass data.

ECS Cloud platform provides various software services [11]. As shown in Figure 7 for instance. Portal Services provide interface to storage resources like GUI (browser-based; called portal) or REST API (which can be utilized to design own portal) or CLI (which functions similar to GUI). Provisioning services are responsible for user administration, authentication, management of resources and multi-tenancy. Fabric Service monitors the health of nodes, disk, cluster. Infrastructure service provides LINUX OS on nodes and manages hardware tools. Storage service layer forms the heart of ECS Cloud Storage platform [11]. The unstructured storage Engine (USE) [11], performs main functions like managing transaction and maintenance of object namespace. USE performs all write operations on containers called chunks. However, data that is written is not updated and is written in append-mode. Thus, locking mechanism is not required. For tracking object location, an index is maintained which records object name, chunk id and offset id. Storage engine also ensures erasure coding and recovery for protection of data. EMC ECS is used for archival purposes in health care industry particularly in partnership with EMC Isilon [12].

for tree=rounded corners, draw, align=center, top color=white, bottom color=blue!20, edge+=-¿, l sep’+=1pt, , baseline, qtree [ ECS Platform [ Portal Service] [ Storage Service] [ Provisioning Service] [ Fabric Service] [ Infrastructure Service] ]

Figure 7: ECS Platform Services.

3.1.5 Eucalyptus Walrus Object Storage

Walrus is the object storage service of Eucalyptus in which users store data in buckets [41]. Walrus provides an interface for manipulation of buckets and objects. The function of buckets are [41] as follows.

1. Buckets are used to store and manage Eucalyptus Machine Images (EMI). An image is actually collection of various application software and system software which is uploaded to Eucalyptus cloud.

2. They are also used to store and manage user data.

During Eucalyptus installation, two types of accounts are created: Admin Account and User account. Third party tools like S3 curl [4] is used for interaction with Walrus. For storing, retrieval of object or creation/deletion of bucket, users use these CLI or GUI interfaces provided by third party tools. Even S3 buckets can be accessed as local directories.

Walrus uses ACL (Access Control List) for giving restrictions on users access to bucket/object. Secret Key and Access Key are required to authenticate the user. Once authenticated, permissions may be granted.

3.2 Archive Storage

In archive type storage, only focus is management of huge volume of data for an indefinite period of time. Since time-scale is invariant, hence the hardware setup and architecture is such that it will incur as much less cost as possible. Its performance metrics is based on ease of scalability, retention efficiency and cost, replication mechanism, security policy and many others. The two examples that have been discussed are: HCP [30] and Storiant [37] in Table 2.

Table 2: Examples of Archive Storage type of Object-based storage.

Object Storage System	Summarized architectural components
Hitachi Content Platform (HCP) [30]	Each object mainly comprises 2 components Fixed-content data which is the actual copy of the file contents and associated metadata which contains additional information like time of creation and Access Control List (ACL) defining set of permission given to users.
Storiant [37]	Storiant is one of the leaders in provider of cold data storage where data is being stored for a long period at a cost which is lower than the traditional system. Storiant developed object storage by consolidating various open source technologies like Open ZFS file, Cassandra NOSQL for storing object metadata and for programmable interface.

3.2.1 Hitachi Data System (HDS) - Hitachi Content Platform (HCP)

The exponential growth pattern and complexity of handling voluminous data is a great concern. HDS provides a solution to these problems through HCP. HCP is a multipurpose distributed object-based storage system [30] that helps in the management of unstructured data by protecting and retrieving data over a single platform. The features of HCP [30] are as follows.

Object Based Storage: Unstructured data files are stored as an object. Object is a container which contains file data as well as Metadata. Each object is an independent unit which is the target for manipulation of data. The Metadata can be used to perform functions like storage tiering or allocation of object. Each object is therefore intelligent to understand the advance storage and this ensures proper distribution. HCP Architecture ensures abstraction of object data. Internally, data is present in object container. Externally, they are represented as set of files in a directory or by URL.

Object Structure: Each object mainly comprises 2 components as shown in Figure 8: Fixed Content data and associated Metadata. The metadata consists of System Metadata, Custom Metadata (optional) and Access Control List (ACL). Fixed Content Data is an actual copy of the file contents. After the file is stored, it becomes immutable. If versioning is permitted, multiple copies may be present. The Metadata describes content of data. System Metadata may include the time the object was stored and modified (i.e., HCP specific Metadata) as well as Metadata like user id and group id. Custom Metadata contains user supplied information about data object usually in XML Format. ACL contains a set of permission given to users and a group of users to manipulate an object.

Figure 8: Object Structure in HCP Architecture.

HCP also has open architecture that abstracts the data from technology enhancements. This ensures proper archival where data can be accessed by users even after a long time period. Hence, HCP provides storage as well as enables access via several interface like HTTP REST. Hitachi Adaptable Modular Storage (HAMS) [36] is introduction to extend the functionalities of HCP. It is capable of handling more complex tasks mostly suitable in high-end storage system. HCP has applications in health care industry for storing medical records.

3.2.2 Storiant Object Storage

Storiant is one of the leaders in provider of cold data storage. Cold data storage actually refers to the mechanism where data is being stored for a long period at a cost which is lower than the storage cost of traditional system [37]. Storiant developed object storage by consolidating various open source technologies like Open ZFS file [31] and Cassandra NOSQL [22] for storing object Metadata and Swiftstack [38, 8, 33] for programmable interface. Here, object is stored in logical containers. Open ZFS is an example of distributed file system which is highly scalable with features like replication, compression and data security [31] running on LINUX operating systems. Cassandra NOSQL is an example of Key-Value Store. In Key-Value store, data is indexed on the basis of key and the value is associated with it. ZFS is actually used for high reliability purposes. Cassandra is used for proper storage. Storiant uses MAID (Massive Array of Idle Disks) technology [9] for physical storage purposes. MAID technology comprises disk drives that are constantly spinning. Whenever a drive is out of order, it shifts data to other disks. MAID together with Open ZFS and Cassandra makes Storiant ideal for storing unstructured cold data [37].

3.3 Object Storage System (OSS)

OSS came into picture with a promise to deliver the same good performance and efficiency as that of other cloud storage providers. OSS is actually a software solution deployed by organizations within their enterprise or in platforms belonging to various cloud providers. Various examples of OSS and their architectural components are listed in Table 3.

Table 3: Examples of Object Storage System.

Name of Object Storage	Summarized Architectural Components
Scality RING [26]	The ring forms the heart of the system. Two basic processing components are present i.e Access Layer to receive data request from application server and Storage Layer to provide interface with physical storage device.
IBM Cleversafe [13]	Major components include Cleversafe Manager for performance monitoring, Cleversafe Accesser for providing interface for manipulation and encryption of data and Cleversafe Slicestor which is the storehouse of slices of data.
DDN WOS [25]	Major components include WOS building blocks, managed by WOS Core Software and WOS Interface.
Amplidata Himalaya [2, 3]	Against the architecture followed by previous storage systems of Amplidata example Amplistor, Himalaya incorporates a second layer called Scaler layer which contains reverse proxy layer to ensure protection to storage pool.
Cloudian Hyperstore [18]	The components of Cloudian HyperStore include HyperStore Geo Cluster for storage and HyperStore service components like S3 Service, Hyperstore Service which manages Hyperstore storage system, Redis DBservice for managing user information
Caringo Swarm [34]	It is an example of distributed scale out type object storage system that supports different API and allows file, block level access of data stored in Clusters. The Swarm Manager is responsible for accepting instructions coming from Swarm Cluster.
StorageGRID [15]	The unique feature of StorageGRID is the use of Dynamic Policy Engine that is used in place of containers. The Dynamic Policy determines the location and the physical storage device to store data, the degree of replication.
Huawei OceanStor UDS [24]	Main feature is the use of Sea of Drives (SoD) Architecture which comprises Access Cluster which processes client requests and Storage Cluster containing Smart Disks to store data. Access Layer processes client requests and storage cluster.
Swift [38, 8, 33]	Swift is an open source object storage system comprising of Swift Cluster and Process Layer and the ring. Cluster is an aggregation of nodes which run on Swift Server and Services. The swift server processes running in Swift Cluster are referred to as Swift Server Process Layer. The functionality of Ring is map** of names of entities to their physical location
GridBank [1]	Its components are GridBank File System (which ensures that the object storage utilizes any of media storage like tape or cloud as need), Data Management (concerned with data security , integrity, retention of data) and Metadatabase (management of metadata).
Lattus [29]	Comprises Controller Nodes (Distributes Object Data), Storage Nodes (Storage of Data) and Access Nodes.

3.3.1 Scality RING Object Storage System

The ring forms the heart of Scality Object Storage System [26]. It follows scale-out system storage and shared-nothing architecture. It was designed primarily with three goal.

• Store data in the order of exabytes.

• Provide security and enhance durability.

• Manage data efficiently.

The Distributed Ring Architecture is based on three principles:

1. Divide and Conquer: This principle implies balancing of load across the distributed access points. There is no single point of failure.

2. Divide and Store: The method refers to the distributed storage of data where all components work independent of each other and possibly in a parallel manner.

3. Divide and Serve: This principle describes the replication of data across nodes thereby increasing storage performance and reliability.

To provide high performance and scalability, Scality provides two basic processing components i.e., Access Layer and Storage Layer which are actually called Connectors and Storage Nodes [26] as shown in Figure 9.

Figure 9: Scality RING Architecture.

Connector: The connector receives data request from application server and manipulates access to RING. It serves as an entrance to the RING and exposes the API or implemented protocol to the outside world to map the data request to efficient routing mechanism for storing data. Scality supports wide range of connectors including REST API and Scality FUSE.

Storage Nodes: These are logical, virtual servers present in the RING and are distinctly separated from physical entity. Main functions are read, write and retrieval of data. Also, they provide interface and manages communication between the system and actual physical storage devices.

Every storage node is an independent LINUX/UNIX process. Their main task is to locate data and serve object request. Each server, by default, has 6 storage nodes. These nodes are responsible for a separate segment of storage in global RING. So, if there are 6 servers each having 6 nodes, 1 node is responsible for 1/36^th of Key Space of Distributed Hash Table (DHT) that is present for locating object data. Hence, if 1 node is non-functional, the corresponding 6 non-sequential portion of key space will be allocated to the remaining servers. Scality is utilized for archival and analytics purposes by media broadcasters, in research labs as well as government services to manage big data in an efficient manner.

3.3.2 IBM Cleversafe Object Storage

It is a software defined object storage solution that solves the problem of storing massive amount of data in order of petabytes. It uses REST API for retrieval of large amount of object-based data that is stored in flat address container. It used Information Dispersal Algorithm [13] to allocate data that are split into slices. The components [13], as shown in Figure 10 are as follows.

Figure 10: Cleversafe Software Solution.

• Cleversafe Manager: It is a web-based manager for configuration, performance monitoring.

• Cleversafe Accesser: This provides REST API for manipulation of data. It also provides encryption of data. The data slices from the Accesser are sent to the third component i.e., Slicestor.

• Cleversafe Slicestor: It is the storage of slices and performs integration by removing differences in data.

Cleversafe is utilized immensely for analytics in R&D Industry along with Hortonworks. It is also used in health care industry for archival purposes.

3.3.3 DDN’s Web Object Scalar (WOS)

DDN’s chief importance is designing efficient storage system without making the structure more complicated. WOS is an example of storage which strips down the architecture into basic units [25]. It comprises three components as shown in Figure 12: WOS Building Blocks, WOS Core Software and Interfaces.

WOS Building Blocks: The unit of building block is WOS Storage node. The storage nodes comprise 4U rack servers [45] filled with SATA (Serial Advanced Technology Attachment) disk. SATA is a new standard for connecting hard drive to computer system [17]. A WOS block by default has 3 nodes and also has flexible scalability.

WOS Core Software: Good storage needs to be managed by smart software. WOS Core has efficient management functionality. It even has self-healing capacities [25], which reduces cost of maintenance.

WOS Interface: WOS provides decent choices of interface like S3 REST, file access interface.

WOS is designed as a true object storage platform following flat address space for storage. The data is stored as contiguous blocks; hence, disk access is minimized and performance is enhanced. It is mainly used as persistent storage in financial services and media and entertainment industry where replication factor, consistency are also very important.

3.3.4 Amplidata’s Himalaya

Traditional object storage architecture describes objects which are self-contained with metadata. Himalaya takes up the challenge by achieve unlimited scalability by attaching the Metadata to the physical layer of nodes [2]. Amplidata’s previous object storage called Amplistore enhanced the speed of access to data by caching the object Metadata at controller nodes. Hence, the entire storage pool need not be searched for the object. Himalaya incorporates a second layer called Scaler layer. The scaler layer contains reverse proxy layer to ensure protection to storage pool.

Figure 11: Components of WOS.

Figure 12: Himalaya Architectural Overview.

Amplidata’s architecture has 2 components [3] as shown in Figure 12. Scaler accessing devices and object store protected by erasure coding. Erasure coding implies splitting of data into fragments, encoding with additional redundant bits and distributing across locations. Scalers include highly scalable global name-space with access layer. Reverse proxy also helps users to access and read data. Himalaya’s service is widely used in media and entertainment industry and as an archive in health-care industry.

3.3.5 Cloudian HyperStore

It is a multi-tenant supported object storage system [18], which supports Amazon S3 services. It incorporates NoSQL storage layer for flexible and scalable storage of increasing upsurge of data. It provides platform that deploys S3 services-compliant storage system. HyperStore is again another perfect example of horizontally scalable object storage supporting increased storage of data by simply adding node in a flat address space. It has got fully distributed, peer-to-peer architecture eliminating single point of failure. Its various features [18] include:

• Amazon S3 Compliant: All customers’ existing S3 REST API based applications will work with Cloudian Service. S3 tools can also be used for develo** HyperStore client applications.

• Multi-tenancy: Implies the supporting multiple user access to single infrastructure in parallel. Each user’s data is logically separated and permission needs to be granted explicitly for access.

• Group Support: A single HyperStore account can be shared by a group or an organization. Each group member supervised by an admin has a storage space allocated.

• Quality of Service (QoS) Controls: Cloudian system administrator can control usage rate per member of group or per user. Based on these quotas, reporting mechanisms ensure proper billing of group or individual users.

• Access Control: Read and write access are supported per bucket, per object. Objects are accessible to external environment via URL.

The components of Cloudian HyperStore include HyperStore GeoCluster for storage and services like S3 Services, HyperStore Service, Redis DB Service and Cloudian Management Console (CMC).

Cloudian Hyperstore GeoCluster: As shown in Figure 13, the entire hyperstore system can be deployed in several geographical locations. Each region contains a number of data centers. 1 or more nodes are present in the data centers. Each node contains a number of VNodes which store object for a particular id range. Set of all nodes in a single region forms a GeoCluster. Different regions store different datasets and there is no replication across regions.

Service Components: As shown in Figure 14, it supports several type of services as follows.

1. S3 Service: Responsible for handling S3 REST requests coming from client applications.

2. HyperStore Service and HyperStore File System (HSFS): HyperStore storage system uses hybrid storage system where Cassandra can be used for storing small S3 data objects and LINUX file system resting on Cassandra can be used for large S3 data objects. The portion of LINUX File system where object data is stored is called Cloudian HyperStore File System (HSFS). Casssandra defines key-space of object based storage.

3. Redis DB Services: These are used to store user-level and group level QoS settings that is set by system administrator and to count user requests.

4. Cloudian Management Console (CMC): It is a web-based UI (User Interface) for system administrator, group administrator, users for QoS control and manages billing plan, generates bill and views data stores as object.

Figure 13: HyperStore GeoCluster.

Figure 14: HyperStore Service Components.

HyperStore is popularly used in media & Entertainment industry as well as in R & D labs for analytics purposes.

3.3.6 Caringo Swarm 7

Caringo Swarm 7 is an example of distributed scale out type object storage system that supports different API and allows file and block level access of data [34]. It is an object based software solution that can run on hardware platform which is compatible with Linux or virtual machines. New nodes can be added dynamically in an existing cluster as and when needed. Swarm does not have a separate Metadata management. Objects (i.e., Data + Metadata + Policies) are directly stored on disk. This ensures block level access in addition to file system (NFS) and object API access methods (REST API). Copy of index pointing to Metadata is stored in RAM for faster access. Swarm enables different gateways like Cloud Scalar and File Scalar, each of which provides platforms to traditional interfaces as well as S3 API. These interfaces are selected by users.

Swarm Manager [34], is responsible for accepting instructions coming from Swarm Cluster. It also schedules the proper resources against a particular cluster. The failure of Swarm Manager leads to the failure of cluster operations. The actual processing is done by primary Swarm Manager. The Secondary Swarm manager forwards the same to primary manager. Swarm is used for storing huge volume of unstructured data on platform for Big Data Analysis.

3.3.7 NetApp StorageGRID Webscale

StorageGRID Webscale is a storage platform built for object-based data storage technique [15]. It has massive scalability to store billions of objects in distributed nodes running under single global namespace. The unique feature of StorageGRID is the use of Dynamic Policy Engine [15] that is used in place of containers (a unique manager of data for object storages). The Dynamic Policy determines the location and the physical storage device to store data, the degree of replication (number of copies of data to be stored) and the Metadata stores information about the location of data and the number of fields. StorageGRID also provides S3 API support which is the standard API for object storage. CDMI support is also given. The architecture of StorageGRID Webscale as shown in Figure 15. It comprises 4 nodes.

Figure 15: NetApp StorageGRID Webscale Architecture.

• Admin Nodes: It is responsible for services like configuration, audit, logging and other management services.

• Storage Node: It controls object storage. It also ensures proper replication of object.

• API Gateway Node: It provides an API through which applications connect to StorageGRID.

• Archive Node: It provides interface to physical archival medium like tapes [15].

3.3.8 Huawei OceanStor UDS

Based on big data market demand, Huawei released UDS, a massive storage system that follows scale out architecture and appropriate data protection mechanism [24]. UDS comprises Access Nodes (A Node) and Distributed Nodes (UDSNs). A Nodes are primarily used for data scheduling i.e., assignment of data requests to particular UDSNs. UDSNs are used for data storage. The architecture followed is known as Sea of Drives (SoD) Architecture.

Figure 16: UDS Sea of Drives(SoD) Architecture and I/O Process.

Sea of Drives (SoD) Architecture: Sea of Drives is an architecture suited for manipulation of huge amount of unstructured data that is used primarily for more read operations than write. SoD architecture comprises Access Cluster and Storage Cluster as shown in Figure 16.

• Access Cluster: Access Layer comprises A Nodes that processes client requests. It controls routing of a client request to a proper UDSN. A Nodes are accumulated to form clusters. When the number of requests increase, the nodes can be added dynamically to reduce traffic.

• Storage Cluster: Storage Cluster comprises UDSNs which contain Smart Disks. Smart Disk is the basic unit of physical storage. It contains disk drive together with CPU, memory and Ethernet Port. Each disk is designated with dedicated IP address and connected to switch and other disks. UDS capacity is increased by adding more disks.

I/O Process: I/O process implies writing of object data and reading of object data. Following steps are performed for writing data takes place in the Figure 16.

• A client communicates with an A-node and transmits data to the A Node.

• A Node determines data storage policy (which involves determining multiple copy-for each data, use of erasure Code mechanism for data protection and so on).

• If data size is large, A Node splits the data into partitions.

• Then, A Node writes data slices into storage device based on DHT (Distributed Hash Table).

As shown in Figure 16, steps for reading data are as follows.

• A client sets up connection with A Node. Specifies the data request.

• A Node determines the partition and retrieves address of the smart disk.

• A Nodes aggregate the data slices.

3.3.9 OpenStack Swift

Swift [8, 33] is an open-source object storage system that stores unstructured data [38] like files or videos in large quantities. Object is the unit of data storage and data is replicated across various systems. The applications access data via REST API. An architectural overview of SwiftStack components and processes as shown in Figure 17 includes the following.

Figure 17: SwiftStack Architecture.

• Swift Cluster: It follows the distributed object storage system concept. It is an aggregation of nodes which run on Swift Server and Services.

• Swift Server Processes: Four Swift Server processes are proxy, account, container and object.

  • Proxy: Responsible for connecting the rest of Swift architecture. For each request [8, 19], the proxy looks up for the location or account or object in the ring. All these components will be discussed in the following.

  • Object: It is a simple BLOB storage system that involves storing, removing and retrieving objects on a local device [8]. Objects are stored and identified by using path name.

  • Container: Primary function is to list the objects [8, 19]. It does not know the location of the object.

  • Account: It lists the containers that are present [8, 19]. Account should not be confused with user account. All accounts and containers are considered as places of storage.

• Swift Server Process Layer: The swift server processes which are running in Swift Cluster are referred to as Swift Server Process Layer [19]. There are four process layers.

  • Proxy Layer: Directly interfaces with the client [38]. It is the first layer to handle API. It uses standard HTTP request and response.

  • Account Layer: Requests regarding Metadata [38] of individual account or container are handled here.

  • Container Layer: Requests regarding Metadata of object [38] are handled here.

  • Object Layer: Responsible for actual storage of data on drive [38]. Object are stored in binary file using path name.

• Ring: Basic functionality of Ring is map** of names of entities to their physical location [28]. Separate ring is maintained for account and container. For one object, one ring is available. Whenever a component needs to perform an operation on an object or container or account, it needs to consult with the ring to determine the location in the cluster.

Every data or object in SwiftStack clusters is represented by means of URL. A Storage URL in Swift is shown in the Figure 18.

Figure 18: A Storage URL in Swift for an object.

The URL comprises two parts; Cluster Location and Storage Location.

• Cluster Location: It determines target cluster address where the request will be sent. Say the storage URL is https://pic.com/b1/account/container/object. https://pic.com/b1 portion indicates the cluster location within which the object storage must be identified.

• Storage Location: It defines where request is to be processed within a cluster . A storage location can be defined in terms of account or container or object.

  • /account: It is a storage area which lists the Metadata about the account and its corresponding containers.

  • /account/container: Metadata about container and corresponding objects that are stored.

  • /account/container/object: Storage comprising object $data$ and its Metadata.

3.3.10 Tarmin GridBank

Tarmin GridBank provides object storage solution for proper storage and management of object as data [1]. GridBank is mostly popular as a data repository in Oil & Gas Industry. Three components of GridBank actually utilizes the advantages of Data Defined Storage (consolidating data into an integrated storage where the users and application actually accesses the Metadata repository, thereby maintaining full abstraction about data). The main components of Tarmin Data Defined Storage are GridBank File System (GBFS), GridBank Data Management, GridBank Metadatabase. There are additional components too.

• GridBank File System (GBFS): The file system ensures that the object storage utilizes any of media storage like tape or cloud as per organizational demand [1]. The File system integrates all such storage and externally it gives a view of consolidated distributed data storage via the global namespace. There is another utility, compresses huge volumes of data with the help of optimization layer. GBFS also has mechanism for removing duplication.

• GridBank Data Management: This layer is concerned with data security (encryption), integrity, retention of data and replication of data across various sites [1].

• GridBank Meta Database: Concerned with management of Metadata. It is actually a Metadata repository that contains 3 kinds of Metadata indexing from raw data.

  1. File property and attribute

  2. Full content Metadata

  3. Arbitrary Key-Value Metadata

  The distributed store integrates the data coming form different datasets and forms consolidated Metadata access to data [1].

• Policy Engine: That is concerned with automation of migration of data and management of data coming from all sources.

• Global Namespace: Global namespace is maintained which provides a view to client with various protocols.

• Multiple Protocol Access: Data can be accessed via HTTP and FTP. GridBank also supports REST API for management of data.

GridBank is used extensively as storage medium of data in Oil & Gas Industry.

3.3.11 Quantum Lattus

Existing traditional as well as current technologies face the issue of archiving data, as well analyzing or reusing it in future with a major focus on its availability [29]. Lattus is an example of scale-out architecture which eradicates this issue by providing online storage and its inherent nature of high scalability and durability. It actually comprises three components [29].

Controller Nodes: It stores data as object and distributes them into storage nodes. The data can be accessed via REST API.

Storage Nodes: It provides scalable object storage and can automatically check data integrity and perform bit error correction.

Access Nodes: Access nodes improve performance by the mechanism of in-memory caching and disk caching [29]. In-memory cache refers to kee** frequently accessed data in memory to reduce the latency of user query. Disk Cache is a process which aims to reduce access time of hard disk. It may be a part of hard disk or a portion of RAM which holds recently used data. Lattus provides the following advantageous features which make it a strong contender for object storage [29].

• Scalability: Scalability is achieved to thousands of petabytes of data without migration to other storages. Even automatic replication process takes place and proper distribution of data is ensured.

• Self Healing: There are error correcting algorithms which ensure that data is present even when there is a failure in node or device. Lattus continuously performs bit checking for errors and rectifies them. When the drives are replaced, data is again redistributed across network to ensure undisturbed availability of data.

• Reduces Cost: Since a great amount of replicated data is always available, backup cost is minimized.

3.4 Distributed File System (DFS)

Distributed File System creates an illusion of a common file system by distributing the data file in different file servers, located in different locations [6]. These file servers are connected with each other via the internet and they are accessed via remote machine call. This is the basic structure where fault tolerance and scalability are resolved. Modern technologies are involved in distributed file systems for storing big data. The architecture of ceph [43, 44] distributed file system is described in given below.

3.4.1 RedHat Ceph

Ceph is an example of distributed file system which offers high performance and scalability. A unique feature of Ceph is separation of data and Metadata operations [43]. Metadata Cluster is only concerned with opening and renaming file (Metadata operation). Storage cluster is concerned with retrieval of object data. There is a complete removal of overhead caused by maintaining allocation list. Instead, it forms an allocation function called ’CRUSH’ [43] which determines the location of the object in its partition at an OSD. Hence, in order to access object, every time functional computation is performed by client instead of lookup for object and its location. As the Metadata cluster need not be communicated for lookup, load is reduced. Ceph has got the following architectural components [44] as shown in the Figure 19. Major components are as follows.

• Metadata Server (MDS): The main functions of MDS are controlling Metadata operation like file management, ensuring security, and proper authentication.

• Client Interface: Provides API to manage storage system. It also provides the platform between applications and storage system. Performs allocation mechanism called CRUSH to locate the object slice.

• Storage Manager: It manages Object Storage, allocation of free spaces in OSD and scheduling management in each OSD.

When application wants to access data, it first communicates with MDS cluster and then authentication takes place. If authenticated, computation of object storage location is performed. Then, data transfer takes place directly with OSDs after consulting Storage manager.

3.5 Clustered File System (CFS)

In clustered file system, a common file is distributed among multiple storage servers and all the servers are treated as a single file system. Here, storage space is shared among the servers, which increases the system performance with respect to better utilization of storage space. OASIS [20, 32], Panasas [39] and Lustre File System [7] are discussed below as CFS.

3.5.1 OASIS

It is an example of clustered file system utilizing OSD in a file system environment [20]. OASIS is designed with the aim of attaining a highly scalable structure. The functioning of OASIS is shown in Figure 20. OASIS uses OSD as their storage device, and this also provides object based interface to the file data. Each OSD is responsible for the optimization of storage and automatic storing of objects. The Metadata cluster is responsible for global namespace management of object data [20]. It also helps in file to object map** as well as maintains authentication and security of data. There are primarily three components [20].

• OASIS OSD Target (OASIS/OST): It is concerned with the management of object data in an OSD command interface. OST functions as an independent storage device which itself contains processor, RAM (Random Access Memory), the Gigabit Ethernet network interface and disks. OST actually comprises OSD manager and storage device driver. OSD manager implements T10 OSD storage protocol (T10 actually standardizes storage interfaces. It can be seen in reference [32]) and uses SCSI transport protocol for manipulation of object data. Storage device driver is for supporting data storage in the file systems.

• OASIS Metadata Server: Concerned with operations like file creation, lookup and attribute assignment. It also is responsible for sharing file data.

• OASIS/FS File System: It provides POSIX API platform to client and is built on OASIS MDS and OST. It is the gateway to OASIS file and directory. For accessing data directly communicates with OSD.

Figure 19: RedHat Ceph Storage System.

Figure 20: Flowchart of Functioning of OASIS.

3.5.2 Panasas

Panasas Storage Cluster architecture is based on the concept of separation of Metadata from actual path leading to data [39]. Panasas storage devices are actually OSD, with which the client directly communicates. For proper distribution of data, Panasas Active File System (Pan FS) stripes the data into number of objects and one object is allocated per OSD. There are actually 5 components of Panasas as shown in Figure 21.

1. Object: Object is the unit of storage. Each object contains self-containing information which helps in proper management of data all by itself. Object, together with its attributes communicates with storage system regarding management of data. This is in contrast to a file system where the storage system monitors all the attributes of data.

2. OSD: These are intelligent devices which structurally comprises disk drives, RAM, processor and provides three basic functionalities.

   • Data Storage: Object data are present along tracks and sectors on storage devices. Clients request object data by querying with object ID, and offset that determines length of data requested. No block level access is permitted.

   • Intelligent Layout: Provides a mechanism for efficient access to data. For example, since object Metadata states the length of data to be written, contiguous blocks can be determined. By the help of write-behind cache, huge amount of data is written in less number of passes.

   • Object Management: OSD manages Metadata that is associated with the object. Hence, it reduces the load on MDS.

3. Panasas File System (Pan FS): For reading and writing object from OSD, PanFS kernel module is loaded on the client. There are mainly 4 functions.

   • POSIX File System Interface: Pan FS provides a POSIX interface to applications performing file operations (read() and write()).

   • Caching: There are 3 kinds of caching facilities in PanFS. The first one provides caching in computation node for incoming data. The second one is write-data caching that accumulates a stipulated number of writes for efficient transmission of data. Third caching allows clients to issue secured commands to access data on OSD.

   • Strip**: Strips a file across multiple OSD where one object is allocated to one OSD.

   • iSCSI: Uses iSCSI protocol to support OSD SCSI commands and transmit data payload over Ethernet.

4. MDS: It is the bridge between clients and objects. MDS determines the map containing the outline of each object and thereby enables clients to access objects directly. It is concerned with the management of file strip**, ensuring secured access to object data. Secured access is done with the help of capability, which is a secure token determining the client’s authorization and token encoding object ID. Expiration of capability leads to rejection in client access. MDS performs reconstruction of lost data and cache consistency (notifies the client when changes in cache file are going to change the client’s cache file).

5. Network Fabric: Network provides efficient communication between OSD, MDS, PanFS and uses TCP/IP as protocol over Ethernet. iSCSI protocol is used for propagation of command and data from/to OSD. It has a facility of RPC (Remote Procedure Call) to provide fast communication between client and MDS.

3.5.3 Lustre

It is an example of open-source parallel distributed cluster file system using OSD as its storage device [7]. It is generally used for large-scale computing. It comprises the following components as shown in Figure 22.

Figure 21: Panasas Architecture.

Figure 22: Lustre Architecture.

1. Client: Absolute abstraction regarding the data storage is maintained from the client. The client sees a distributed file system supporting standard POSIX commands. A global namespace provides an abstracted view of available stored data. However, special applications can directly communicate with OSD without intervention of client.

2. Metadata Control System: Concerned with management of file system Metadata as well as directly communicates with storage target for information about Metadata.

3. Storage Target: It is the physical storage location of object. Objects can be an entire file, or strips of file data. The files are represented by container objects on MDS and data object in storage target.

4. Resource and Security Database: Concerned with resource management and security services in databases.

4 Conclusion

The big data storage system is a service model where internet technologies are used to store big data. The data storage servers are placed in offsite locations and this data is accessed via internet service. Fault tolerance, accessibility, reliability, availability, scalability, and data security, all are managed by the big data storage system. For big data managing purposes, different data storage service providers consider different approaches, such as distributed file system, clustered file system, cloud storage, archive storage, and hybrid storage solution.

In this survey article, our aim is to present a brief overview of the architectural details of different big data storage systems. These big data storage systems are developed by corporate organizations from a business perspective. Different storage service provider has different intention to design storage system. Some service provider serves as the data dump station and some support the data analytic-related job. Therefore, the architectures are also varied from one to another. These architectural variations are manipulated through the detailed discussion of the architecture of different storage systems.

Statements & Declarations

Competing Interests and Funding

The authors have no relevant financial or non-financial interests to disclose.

Conflict of Interest

The authors declare that they have no conflict of interest.