...

<configuration>
  <property>
    <name>ozone.om.enable.filesystem.paths</name>
    <value>true</value>
  </property>
 
  <property>
    <name>ozone.om.metadata.layout</name>
    <value>prefix</value>
  </property>
 
</configuration>

Related documents

• Design Document
• Internal design - metadata format
• Performance Test Report

Branch merge checklist

...

Described feature in Apache ozone page via HDDS-5067.

• hadoop-hdds/docs/content/feature/PrefixFSO.md has the feature details and related configurations.

3. design, attached the docs

All the Following design docs are linked from the documentation as part of present in HDDS-2939 TODO: create a link from the documentation page.jira

• Design Document
• Internal design - metadata format
• hadoop-hdds/docs/content/feature/PrefixFSO.md has the feature details and related configurations. 

4. s3 compatibility

There are no incompatibilities with respect to S3. This feature can be enabled only together with ozone.om.enable.filesystem.paths. When file system-style path handling is enabled, 100 % s3 compatibility could not be guaranteed. FS compatible s3 key names supposed to be working well, but non-fs compatible, extra key names (like 'a/../b1 or real file with the name `key1/` might be normalized or rejected by the implementation of ozone.om.enable.filesystem.paths.)

TODONote: Added S3 acceptance test when with feature is turned on ?- PREFIX layout.

5. docker-compose / acceptance tests

...

There is no significant difference between local build time.

Recent master build

Recent HDDS-2939 branch build

Image Removed

TODO: Image Added

• test time of acceptance unsecure is increased with 20 ~3 minutes
• integration test is increased with 30 ~4 mins

9. possible incompatible changes/used feature flag: 

For using this feature, "ozone.om.metadata.layout" config needs to be set to be true in ozone-site.xml

The new metadata layout version can be turned on at any cluster as the flag is stored is supported only in a fresh cluster and the layout detail is stored in per-bucket. Old buckets will use the old metadata version (simple) new buckets will use new layout. All existing keys can be read and no additional migration is required (TODO: confirm)Presently, both old and new metadata layout buckets can't co-exists in the same cluster. User can't start OM in new layout(prefix) if there are existing old layout buckets(simple) and vice-versa. Work is in progress to support the existing old buckets to be available in new layout, this will be supported in the next development phase.

10. third party dependencies/licence changes:

...

Done testing to evaluate the performance of delete, rename operations in feature branch vs master code base. Following charts capturing the directory delete and rename operations execution time shows that, feature branch has a very significant performance gain compared to the master.

Ran freon 'dtsg' dfs tree generator benchmark test in a single node cluster. V0 represents master code(simple) and V1 represents feature branch(prefix). Please refer to the Jira document for more details.

...