THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
stable builds/intermittent test failures: There are no intermittent failures specific to the HDDS-2823 branch as of now. During the development , it was ensured all the CI checks are clean prior to every commit merge .The plan is to run repeated CI checks on the merge commit to master.
documentation: All the docs are available in the doc section DOCS.
design, attached the docs: All the design docs are also available in the doc section DOCS.
s3 compatibility: There are no incompatibilities with respect to s3. S3 is not changed at all (except that the OmUtils rename affected it)
support of containers / kubernetes: (Will be updated soon)
Tested in Kubernetes and everythign worked well. Example files are uploaded in the patch to:
coverage/code quality: (Will be updated soon)
Sonar master branch.
Sonar SCM-HA branch
SCM-HA has 20 more issues but 2.3 better code coverage.
build time:
Recent master builder: https://github.com/apache/ozone/actions/runs/638668950
Recent SCM-HA build: https://github.com/apache/ozone/actions/runs/638165382
SCM-HA branch didn't introduce any significant slowness (2-3 minutes plus to the existing integrations test and acceptance tests which are already close to 1h runtime).
There is no significant difference between build time. In a linux with the below configuration,
hadoop@9 ~/glengeng$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 1
Core(s) per socket: 16
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 94
Model name: Intel(R) Xeon(R) Gold 61xx CPU
Stepping: 3
CPU MHz: 2494.138
BogoMIPS: 4988.27
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 4096K
NUMA node0 CPU(s): 0-15
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap xsaveopt xsavec xgetbv1 arat
hadoop@9 ~/glengeng$ free -g
total used free shared buff/cache available
Mem: 125 4 98 0 22 120
Swap: 0 0 0
#data for master
INFO BUILD SUCCESS
INFO ---------------------------------------------
INFO Total time: 08:10 min
INFO Finished at: 2021-03-10T16:54:42+08:00
INFO ---------------------------------------------
#data for HDDS-2823
INFO BUILD SUCCESS
INFO --------------------------------------------
INFO Total time: 07:09 min
INFO Finished at: 2021-03-10T17:06:42+08:00
possible incompatible changes/used feature flag: For using the SCM HA feature, "ozone.scm.ratis.enable" config needs to be set to be true in ozone-site.xml.
third party dependencies/licence changes: None