You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

2017-02-21

alpha3 features:

  • Erasure coding
    • Not much movement on blockers since last update, moved out planned alpha3 date as a result.
    • Some discussion around configuration of enabled EC policies.
    • Manoj and Andrew are working on EC CLI improvements (HDFS-11405, HDFS-11426, HDFS-11427, HDFS-11428)
    • Manoj continues to work on OIV improvements (HDFS-10983)
    • Kai Sasaki revving patch on HDFS-8196 for webUI additions after reviews from Takanobu Asanuma and Andrew
  • Tomcat to Jetty conversions
    • HDFS-10860 (HttpFS): This was committed! Will remove section for next update.

beta1 features:

  • Addressing incompatible changes
    • YARN-6142 (compatibility with 2.x): Tracking JIRA filed and assigned to Ray
    • HDFS-11096 (compatibility with 2.x): Nothing further pending on this JIRA, but would like automated test runs setup to catch future breakages.
  • Classpath isolation (HADOOP-11656)
    • Andrew has asked Sean for an ETA on completing the remaining subtasks, ideally by alpha3 but latest beta1

2017-02-03

3.0.0-alpha2 was released last week, onward to alpha3!

Notable release blockers:

  • Since alpha3 is planned as the last alpha before beta, the goal is to be feature complete by alpha3. beta1 can be used to stage any remaining incompatible changes, or nice-to-have feature-related work.
  • No other release blockers are filed.

alpha3 features:

  • Erasure coding
    • All must-do JIRAs are now marked as blockers for alpha3, goal is for EC to be feature complete and API stable by alpha3.
    • Light development activity. Andrew did some reviews, Manoj is working on OIV improvements.
  • Tomcat to Jetty conversions
    • HDFS-10860 (HttpFS): Patch has gone through multiple revs and has a +1 pending from Xiao

beta1 features:

  • Addressing incompatible changes
    • YARN-6142 (compatibility with 2.x): Tracking JIRA filed and assigned to Ray
    • HDFS-11096 (compatibility with 2.x): Nothing further pending on this JIRA, but would like automated test runs setup to catch future breakages.
  • Classpath isolation (HADOOP-11656)
    • Andrew has asked Sean for an ETA on completing the remaining subtasks, ideally by alpha3 but latest beta1

2017-01-19

branch-3.0.0-alpha2 has been created in git. Still on track for planned alpha2 release at the end of January.

Notable release blockers:

  • YARN-5667 (Move HBase backend code in ATS v2 into its separate module): One subtask left at YARN-5928, looks close to commit.
  • Also resolved blockers like HADOOP-13996, YARN-6071, YARN-5646, YARN-6072, HADOOP-13961, HDFS-11312, YARN-6068, HADOOP-13780 since last update. Wow!

Development activity

  • Addressing incompatible changes
    • HDFS-11096 (compatibility with 2.x): Sean filed and fixed HDFS-11312, which was a PB incompatibility. Seems like this work is complete on the HDFS side, but we still need YARN and MR help.
  • Erasure coding
    • Andrew went through and re-triaged all the JIRAs on HDFS-8031, using the "hdfs-ec-3.0-must-do" and "hdfs-ec-3.0-nice-to-have" labels. HDFS-8031 more closely reflects what we want to get done for 3.0, and the labels are now tracked on the JIRA dashboard.
  • Classpath isolation (HADOOP-11565)
    • Arun added a Maven variable to skip shading for faster build times. No progress on subtasks required for beta1.
  • Tomcat to Jetty conversions
    • HADOOP-13597 (KMS): this patch went in
    • HDFS-10860 (HttpFS): Patch up and gone through initial review

2017-01-04

Notable release blockers:

Down to just four release blockers, three of which have patches posted and seem to be close.

  • HADOOP-13780 (L&N updates): Patch is posted and +1'd, likely will be committed soon.
  • HADOOP-13896 (distribution tarball missing jars): Tiny patch is posted, pending review and precommit.
  • YARN-6022 (Revert changes of AbstractResourceRequest): Patch is posted and been reviewed, looks close.
  • YARN-5667 (Move HBase backend code in ATS v2 into its separate module): One subtask left at YARN-5928, PA but no progress recently.

Development activity

  • Addressing incompatible changes
    • MAPREDUCE-6704: resolved!
    • YARN-5882 (issuing YARN delegation tokens even in insecure deployments): Jian reverted YARN-4126 pending further discussion, so no longer a blocker.
    • HDFS-11096 (compatibility with 2.x): Sean reviewed the JACC output and posted a big comment on possible source/binary incompatibilities.
  • Erasure coding
    • Still slowly burning down must-dos from the JIRA query.
  • Classpath isolation (HADOOP-11565)
    • HADOOP-11804 (shaded client jars): Initial patch has been committed, Sean also filed follow-on JIRAs for related work we need to clean up before beta1.
  • Tomcat to Jetty conversions
    • HADOOP-13597 (KMS): +1 pending a few nits
    • HDFS-10860 (HttpFS): Patch up and gone through initial review

2016-12-09

Notable release blockers:

  • HADOOP-13780: L&N updates from Jetty bump. Xiao has picked this up, no patch yet though.

Development activity

  • Addressing incompatible changes
    • MAPREDUCE-6704: still seems close after examination of Docker issue
    • YARN-5882 (issuing YARN delegation tokens even in insecure deployments): Andrew proposed we revert YARN-4126 to unblock the release while discussion continues
    • YARN-5559: Resolved!
    • YARN-5184: Resolved!
    • HDFS-11096 (compatibility with 2.x): Discussion still underway. Kihwal and Eric worked to resolve HDFS-11207 which reverted some incompatible DN/NN changes that broke rolling upgrade.
  • Erasure coding
    • Still slowly burning down must-dos from the JIRA query.
  • Classpath isolation (HADOOP-11565)
    • HADOOP-11804 (shaded client jars): Sean added integration tests for WebHDFS and things seem close.
  • Tomcat to Jetty conversion (HDFS-11096, HDFS-10860)
    • HDFS-11096 (KMS): patch is under review
    • HDFS-10860 (HttpFS): no patch yet

2016-11-28

Notable release blockers:

  • org.json:json: legal-discuss has allowed grandfathering in releases until April 30, 2017 for usages of org.json:json. HADOOP-13050 bumping the problematic AWS SDK dependency has been committed. We're in the clear.
  • HADOOP-13780: L&N updates from Jetty bump. Still unassigned.

Development activity

  • Addressing incompatible changes
    • MAPREDUCE-6704: still seems close after examination of Docker issue
    • YARN-5882: issuing YARN delegation tokens even in insecure deployments, discussion underway
    • YARN-5559: found via JDiff analysis, fix some accidental incompats
    • YARN-5184: found via downstream testing. New abstract methods added to public class.
    • HDFS-11096 (compatibility with 2.x): Summary of a discussion between Andrew and Karthik was posted, seeking additional community feedback.
  • Erasure coding
    • Slowly burning down the must-dos from the JIRA query.
  • Classpath isolation (HADOOP-11565)
    • HADOOP-11804 (shaded client jars): POC testing with Avro and HBase seem to work, waiting on additional testing and reviews.
  • Tomcat to Jetty conversion (HDFS-11096, HDFS-10860)
    • Initial patch posted on HDFS-11096 for KMS conversion.

2016-11-16

Notable release blockers:

  • HADOOP-11694: Need to upgrade AWS SDK dependency, org.json:json fallout
  • HADOOP-13780: L&N updates from Jetty bump

Development activity

  • Addressing incompatible changes
    • MAPREDUCE-6704: make the default pseudo-cluster work without additional env var configuration, seems close?
    • YARN-5882: issuing YARN delegation tokens even in insecure deployments, discussion underway
    • YARN-5559: found via JDiff analysis, fix some accidental incompats
    • YARN-5184: found via downstream testing. New abstract methods added to public class.
  • Erasure coding
    • Work continues on API refinements and supportability-related JIRAs.
  • Classpath isolation (HADOOP-11565)
    • HADOOP-11804 (shaded client jars): Multiple revs have been posted based on review feedback.

 




  • No labels