You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

Upgrade Branch Merge Checklist

1. stable builds/intermittent test failures

  • The HDDS-3698-nonrolling-upgrade branch has no intermittent test failures.
  • Most intermittent failures specific to the HDDS-3698-nonrolling-upgrade branch were tracked and resolved in HDDS-4833.
  • Other intermittent failures have been resolved in HDDS-5109 and HDDS-5336.

2. documentation

  • hadoop-hdds/docs/feature/design/how-to-do-a-nonrolling-upgrade.md contains instructions for users to upgrade a cluster using the framework.

  • Documentation will be refined in coming weeks, before it is needed in the 1.2.0 release.

3. design, attached the docs

  • hadoop-hdds/docs/content/design/upgrade-dev-primer.md contains instructions for developers who need to add a feature using the upgrade framework.
  • hadoop-hdds/docs/content/design/nonrolling-upgrade.md contains links to the main design document and presentation.
  • hadoop-hdds/docs/content/design/omprepare.md contains links and summary for OM preparation design document.

4. s3 compatibility

  • There are no S3 incompatibilities. This code was not changed.

5. docker-compose / acceptance tests

  • Upgrades can be done in docker-compose to/from any Ozone version which has a docker image, or to the current code.
  • See hadoop-ozone/dist/src/main/compose/upgrade/README.md for more information.

6. support of containers / Kubernetes:

  • The upgrade framework will not effect Kubernetes deployments, since it does not require additional configuration for startup.

7. coverage/code quality:

  • Sonar for master

    • Code coverage: 74.8%

  • Sonar for upgrades

    • Code coverage: 73.3%

  • OM Prepare and general upgrade flow are tested through new acceptance tests which are not reflected in the code coverage.

8. build time

  1. New upgrade acceptance tests, which walk through a cluster upgrade from the last release's docker image to the current build, in order to catch backwards incompatible changes.

    • Although we previously had upgrade tests on master, the added tests use the upgrade framework which uses more commands and tests, and therefore takes longer to run.

  2. New upgrade integration tests in TestHDDSUpgrade, which test various failure combinations that could occur during the upgrade.

    • Only a subset of these tests are currently being run to save time.

  3. New integration tests in TestOzoneManagerPrepare, which tests OM preparations under various failure and request combinations.

9. possible incompatible changes/used feature flag:

There are no incompatible changes and no feature flags. The upgrade framework will only be used during an upgrade.

10. third party dependencies/license changes:

  • The following dependencies have been added:

    • aspectjrt-1.8.9.jar
    • aspectjweaver-1.8.9.jar
    • reflections-0.9.12.jar

  • All new libraries have compatible licenses. (License file update: HDDS-5137)

11. performance

  • The majority of the upgrade framework is not used when an upgrade is not being performed, so it is not expected to have a performance impact.

  •  LayoutVersionInstanceFactory was added to potentially handle client requests within OzoneManagerRatisUtils, although this is not being used yet to accommodate HDDS-2939 (filesystem optimizations).

  •  Changes do not seem to affect performance based on simple freon testing (3 runs per branch):

ozone freon rk --num-of-keys=100 --num-of-buckets=10 --num-of-volumes=1 --replication-type=RATIS --factor=THREE

HDDS-3698-nonrolling-upgrade:

***************************************************
Status: Success
Git Base Revision: 7a3bc90b05f257c8ace2f76d74264906f0f7a932
Number of Volumes created: 1
Number of Buckets created: 10
Number of Keys added: 1000
Ratis replication factor: THREE
Ratis replication type: RATIS
Average Time spent in volume creation: 00:00:00,013
Average Time spent in bucket creation: 00:00:00,050
Average Time spent in key creation: 00:00:02,493
Average Time spent in key write: 00:00:01,163
Total bytes written: 10240000
Total Execution time: 00:00:20,679
***************************************************

***************************************************
Status: Success
Git Base Revision: 7a3bc90b05f257c8ace2f76d74264906f0f7a932
Number of Volumes created: 1
Number of Buckets created: 10
Number of Keys added: 1000
Ratis replication factor: THREE
Ratis replication type: RATIS
Average Time spent in volume creation: 00:00:00,007
Average Time spent in bucket creation: 00:00:00,065
Average Time spent in key creation: 00:00:02,014
Average Time spent in key write: 00:00:01,144
Total bytes written: 10240000
Total Execution time: 00:00:23,661
***************************************************

***************************************************
Status: Success
Git Base Revision: 7a3bc90b05f257c8ace2f76d74264906f0f7a932
Number of Volumes created: 1
Number of Buckets created: 10
Number of Keys added: 1000
Ratis replication factor: THREE
Ratis replication type: RATIS
Average Time spent in volume creation: 00:00:00,006
Average Time spent in bucket creation: 00:00:00,089
Average Time spent in key creation: 00:00:02,442
Average Time spent in key write: 00:00:01,076
Total bytes written: 10240000
Total Execution time: 00:00:23,655
***************************************************

master:

***************************************************
Status: Success
Git Base Revision: 7a3bc90b05f257c8ace2f76d74264906f0f7a932
Number of Volumes created: 1
Number of Buckets created: 10
Number of Keys added: 1000
Ratis replication factor: THREE
Ratis replication type: RATIS
Average Time spent in volume creation: 00:00:00,007
Average Time spent in bucket creation: 00:00:00,046
Average Time spent in key creation: 00:00:02,422
Average Time spent in key write: 00:00:01,411
Total bytes written: 10240000
Total Execution time: 00:00:25,699
***************************************************

***************************************************
Status: Success
Git Base Revision: 7a3bc90b05f257c8ace2f76d74264906f0f7a932
Number of Volumes created: 1
Number of Buckets created: 10
Number of Keys added: 1000
Ratis replication factor: THREE
Ratis replication type: RATIS
Average Time spent in volume creation: 00:00:00,006
Average Time spent in bucket creation: 00:00:00,039
Average Time spent in key creation: 00:00:02,762
Average Time spent in key write: 00:00:01,466
Total bytes written: 10240000
Total Execution time: 00:00:23,617
***************************************************

***************************************************
Status: Success
Git Base Revision: 7a3bc90b05f257c8ace2f76d74264906f0f7a932
Number of Volumes created: 1
Number of Buckets created: 10
Number of Keys added: 1000
Ratis replication factor: THREE
Ratis replication type: RATIS
Average Time spent in volume creation: 00:00:00,006
Average Time spent in bucket creation: 00:00:00,029
Average Time spent in key creation: 00:00:01,975
Average Time spent in key write: 00:00:01,040
Total bytes written: 10240000
Total Execution time: 00:00:23,677
***************************************************

12. security considerations

The branch introduced the RPC methods in the ScmAdminProtocol.proto to initialize (and finalize) the upgrade process. They are available only to the admins (HDDS-5138).

  • No labels