Versions Compared

Old Version 1

changes.mady.by.user Wei-Chiu Chuang

Saved on

compared with

New Version 2

changes.mady.by.user Wei-Chiu Chuang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Configuration protocol for SCM 
    1. https://docs.google.com/document/d/11Okhp0r9YnOpvTwenMtgPO5i19-BWUyBGBNxKCOUaaQ/edit
    2. https://docs.google.com/document/d/1Bh0SKDsMFLGB4k8BVWE9lozapn8CeGFPKLKqldi1L0Q/edit 
    3. Size : Medium
  2. S3 Load balancing - This is a work in progress based on the ideas from Craig Conduit from Target. I need to write down these ideas, but they have a successful load balancing pattern they use inside their cluster. We will borrow those ideas for S3 load balancing for Ozone. Size : Medium
  3. Trash - Matt from target is already driving this project. We can sync up and post patches with in this project. Size: Medium
  4. Old Config to new config translations - https://docs.google.com/document/d/1LYcir2-0K16-I4ZKNRrQLrz1GRLmMG8GTNEQ3OcpJcM/edit  --
      1. Chris Teoh is working with me on this -- but there are so many to convert this is a gold mine of patches. Easy, Size: Medium.
    1. SSD aware datanode -- if there are SSDs on the data node, we would like to use SSD as a cache and as the Ratis Log location. We also need a background thread that can move older files to spinning disks. Size : Medium
    2. Security architecture for the containers - we need to add some metadata and checking on containers to create them securely. This would mean that container creation will need to move to a client pipeline. Size: Small
    3. Certificate Server - Revoke Certificate - apurohit is probably going to work on this- Size: small
    4. Certificate Server - Manual approval of Data node certificates - Size: Medium
    5. Generic Token Service - Size: Small/Mediumhttps://docs.google.com/document/d/1_iUBKnbodPoQCzUuhlrTaXD4W_mmySeby660v60ibis/edit
    6. Modification of Block Size -- Size: Small
    7. Modification of Blocks to Support Multiple SCMs. - Size: Small/Medium
    8. SCM to have the ability to support multiple OMs groups - depend on the container security (6) - Small
    9. Ozone to Ozone upgrade - Size:Large ??
      1. rolling upgrade
    10. Object Version Support : Medium
    11. Ozone in-place upgrades - XXL -HDDS-1266
    12. Fix the open key cleanup thread - Size:small
    13. Translate docs to Chinese -- Size:Small
    14. Add Object ID and Change ID to Buckets : Size: very small
    15. Add Object ID and Change ID to Keys : Size: very small
    16. Add modification time for Volume/Bucket (HDDS-426) : Size: very small
    17. Track tests that fail -- build an historical report for Ozone. Size:Small
    18. Add notion of ServiceName or Domain to OM certificates - part of generic token design - apurohit. Size: very small
    19. Create an NIO based Ozone client. - Size: Small/Medium. -reference HDFS
    20. Rename S3Token to something like S3AuthInfo.- Size: Very small.
    21. Ozone Client(Python) for Data Science Notebook such as Jupyter - Size: Large
      1. -  PyArrow: https://pypi.org/project/pyarrow/
      2. - Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API Impala uses                      libhdfs
      3. How Jupyter iPython work: https://jupyter.readthedocs.io/en/latest/architecture/how_jupyter_ipython_work.html
      4. Eco, Architecture:https://ipython-books.github.io/chapter-3-mastering-the-jupyter-notebook/