Versions Compared

Old Version 16

changes.mady.by.user Andrew Wang

Saved on

compared with

New Version 17

changes.mady.by.user Andrew Wang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

2017-09-22

We've had some late breaking blockers related to Docker support that are delaying the release. We're on a day-by-day slip at this point.

 

Highlights:

  • I did a successful test create-release earlier this week.

Red flags:

  • Docker work resulted in some last minute blockers

Previously tracked beta1 blockers that have been resolved or dropped:

  • HADOOP-14771 (hadoop-client does not include hadoop-yarn-client): Dropped this from the blocker list as it's mainly for documentation purposes
  • HDFS-12247 (Rename AddECPolicyResponse to AddErasureCodingPolicyResponse) was committed.
beta1 blockers:
  • YARN-6623 (Add support to turn off launching privileged containers in the container-executor): This is a newly escalated blocker related to the Docker work in YARN. Patch is up but we're still waiting on a commit.
  • HADOOP-14897 (Loosen compatibility guidelines for native dependencies): Raised by Chris Douglas, Daniel will post a patch soon.

beta1 features:

  • Erasure coding
    • Resolved last must-do for beta1!
    • People are looking more at the flaky tests and nice-to-haves
    • Eddy continues to make improvements to block reconstruction codepaths
  • Addressing incompatible changes (YARN-6142 and HDFS-11096)
    • Ray has gone through almost all the YARN protos and thinks we're okay to move forwards.
    • I think we'll move forward without this committed, given that Sean has run it successfully.
  • Classpath isolation (HADOOP-11656)
    • HADOOP-13917 (Ensure nightly builds run the integration tests for the shaded client): Sean wants to get this in before beta1 if there's time, it's already catching issues. Relies on YETUS-543 which I reviewed, waiting on Allen.
    • HADOOP-14771 might be squeezed in if there's time.
  • Compat guide (HADOOP-13714)
    • HADOOP-14897 Above mentioned blocker filed by Chris Douglas.
  • TSv2 alpha 2
    • This was merged, no problems thus far (smile)

GA features:

  • Resource profiles (Wangda Tan)
    • Merge vote was sent out. Since branch-3.0 has been cut, this can be merged to trunk (3.1.0) and then backported once we've completed testing.
  • HDFS router-based federation (Chris Douglas)
    • This is like YARN federation, very separate and doesn't add new APIs, run in production at MSFT.
    • If it passes Cloudera internal integration testing, I'm fine putting this in for GA.
  • API-based scheduler configuration (Jonathan Hung)
    • Jonathan mentioned that his main goal is to get this in for 2.9.0, which seems likely to go out after 3.0.0 GA since there hasn't been any serious release planning yet. Jonathan said that delaying this until 3.1.0 is fine.
  • YARN native services
    • Still not 100% clear when this will land.

2017-09-19

Sorry for the late update. We're down to one blocker and one EC must do! Made great progress over the last week and a bit.

...