Hadoop Distributed Storage layer (HDSL) is a distributed, replicated block layer that allows separation of block layer and namespace for HDFS.
HDSL allows different kind of storage systems to be built on an HDFS cluster. Ozone is a distributed key-value store built using HDSL
store built by leveraging primitives present in HDSL. Ozone supports REST API for accessing the store.
Ozone is a work in progress and currently in alpha state. To test, you need to build if from the source code or use Hadoop version higher than 3.1.
Getting started
To run an Ozone cluster you have multiple options:
- Build a new Ozone cluster from the source code and start a cluster with docker (also useful for development)
- Build new Ozone cluster from the source code and start it with the startup scripts without docker.
Additional information
To follow the progress of Ozone development you can check the JIRA dashboard
All the discussions/subtasks/original design documents are available from the feature branch jira:
- A detailed discussion and work on how HDSL can be used in HDFS is tracked in Building HDFS on top of new storage layer
- A detailed discussion of Ozone file system and design docs can be found here.
- A detailed and work items for the security of Hadoop Distributed Storage Layer is tracked in this JIRA
The documentation is part of the source code, but you can check out directly from github: https://github.com/apache/hadoop/tree/HDFS-7240/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown See the markdown files with Ozone prefix (Or you can generate a new site from the feature branch)