Git branch: https://github.com/apache/ozone/tree/HDDS-3630
Currently there will be one RocksDB for each Container on datanode, which leads to hundreds of thousands of RocksDB instances on one datanode. It's very challenge challenging to manage this amount of RocksDB instances in one JVM. Please refer to the "problem statement" section of the design document[1] for detail challenges. Different than challenge details. Unlike the current approach, Datanode RocksDB merge feature will use only one RocksDB for each data volume. With far less fewer RocksDB instance instances to manage, the write path performance and DN stability is improvementare improved, Refer to the Micro Benchmark Data section of the design document[1].
For more, please check out the full documentation. The doc has feature overview, setup guide, CLI guide and access control guide (best viewed locally rendered using hugo serve
command under ./hadoop-hdds/docs/
, as it is not published to the website yet).
To enable the feature, the following configs need to be added to Ozone Manager's ozone-site.xml
.
...
2. Documentation
Documentation has been is being added since by
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
3. Design, attached the docs
...
5. Docker-compose / acceptance tests
New acceptance test is being added by Jira:
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
6. Support of containers / Kubernetes:
...