Versions Compared

Old Version 1

changes.mady.by.user Tsz-wo Sze

Saved on

compared with

New Version 2

changes.mady.by.user Tsz-wo Sze

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Documentation is being added by 

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyHDDS-67907425
. 

3. Design, attached the docs

...

New acceptance test is being added by Jira:

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyHDDS-67917426

6. Support of containers / Kubernetes:

...

A global enable/disable switch for the this feature is added in

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyHDDS-65415480
.

10. Third party dependencies/license changes:

There is no third party dependencies introduced by this feature.

11. Performance

We have tested major datanode activities which require RocksDB operation, include container create & close & delete, and block put & get.  Except that container delete performance drops because container metadata KV need to be deleted from RocksDB, other four major activities all have performance improved. 

The throughput of the new Streaming Pipeline is roughly 2x and 3x of the throughput of the existing Write Pipeline for the single client case and the 3-client case, respectively.

Single client case: 20220702_Single Client - Benchmarks for Ozone Streaming.pdf

3-client case: 20220702_3 Clients - Benchmarks for Ozone Streaming.pdfImage Removed