Page History

...

The prefix based FileSystem optimization idea allows to perform performing rename, delete of any directory in a deterministic/constant time atomically. Now, ozone performs rename/delete operations in a single RPC call by sending only the given directory to OM. It will finish rename, delete operations with O(1) complexity. Also, makes it possible to support atomic rename/delete of any directory at any level in the namespace.

...

Following are the set of configurations to be configured in 'ozone-default.xml' to enable this feature. By default, the feature will be turned OFF.

...

2. documentation

Described feature in apache Apache ozone page via HDDS-5067.

...

All the design docs are linked from the documentation as part of HDDS-2939

TODO: create a link from the documentation page.

4. s3 compatibility

There are no incompatibilities with respect to S3. This feature can be enabled only together with ozone.om.enable.filesystem.paths. When file system-style path handling is enabled, 100 % s3 compatibility could not be guaranteed. FS compatible s3 key names supposed to be working well, but non-fs compatible, extra key names (like 'a/../b1 or real file with the name `key1/` might be normalized or rejected by the implementation of ozone.om.enable.filesystem.paths.)

TODO: S3 acceptance test when feature is turned on?

5. docker-compose / acceptance tests

Added docker test cases.The `compose/ozone` cluster is modified with testing `ozonefs/ozonefs.robot` with or without turning on the new feature. (both ofs and o3fs and linked and unlinked bucket are tested...)

6. support of containers / Kubernetes:

...

7. coverage/code quality:

Sonar master branch
Sonar HDDS-2939 branch.

The branch has better coverage than master (73.5% vs 742.2%) but two new Sonar bugs are introduced (169 vs 171)

8. build time

There is no significant difference between local build time.

Recent master build

Image Added

Recent HDDS-2939 branch build

Image Added

TODO:

• test time of acceptance unsecure is increased with 20 minutes
• integration test is increased with 30 mins

9. possible incompatible changes/used feature flag: 

For using this feature, "ozone.om.metadata.layout" config needs to be set to be true in ozone-site.xml

The new metadata layout version can be turned on at any cluster as the flag is stored per-bucket. Old buckets will use the old metadata version (simple) new buckets will use new layout. All existing keys can be read and no additional migration is required (TODO: confirm)

10. third party dependencies/licence changes:

No new dependencies are added.

11. performance

Done testing to evaluate the performance of delete, rename operations in feature branch vs master code base. Following charts capturing the directory delete and rename operations execution time shows that, feature branch has a very significant performance gain compared to the master.

...