2017-01-19
branch-3.0.0-alpha2 has been created in git. Still on track for planned alpha2 release at the end of January.
Notable release blockers:
- YARN-5667 (Move HBase backend code in ATS v2 into its separate module): One subtask left at YARN-5928, looks close to commit.
- Also resolved blockers like HADOOP-13996, YARN-6071, YARN-5646, YARN-6072, HADOOP-13961, HDFS-11312, YARN-6068, HADOOP-13780 since last update. Wow!
Development activity
- Addressing incompatible changes
- HDFS-11096 (compatibility with 2.x): Sean filed and fixed HDFS-11312, which was a PB incompatibility. Seems like this work is complete on the HDFS side, but we still need YARN and MR help.
- Erasure coding
- Andrew went through and re-triaged all the JIRAs on HDFS-8031, using the "hdfs-ec-3.0-must-do" and "hdfs-ec-3.0-nice-to-have" labels. HDFS-8031 more closely reflects what we want to get done for 3.0, and the labels are now tracked on the JIRA dashboard.
- Classpath isolation (HADOOP-11565)
- Arun added a Maven variable to skip shading for faster build times. No progress on subtasks required for beta1.
- Tomcat to Jetty conversions
- HADOOP-13597 (KMS): this patch went in
- HDFS-10860 (HttpFS): Patch up and gone through initial review
2017-01-04
Notable release blockers:
Down to just four release blockers, three of which have patches posted and seem to be close.
- HADOOP-13780 (L&N updates): Patch is posted and +1'd, likely will be committed soon.
- HADOOP-13896 (distribution tarball missing jars): Tiny patch is posted, pending review and precommit.
- YARN-6022 (Revert changes of AbstractResourceRequest): Patch is posted and been reviewed, looks close.
- YARN-5667 (Move HBase backend code in ATS v2 into its separate module): One subtask left at YARN-5928, PA but no progress recently.
Development activity
- Addressing incompatible changes
- MAPREDUCE-6704: resolved!
- YARN-5882 (issuing YARN delegation tokens even in insecure deployments): Jian reverted YARN-4126 pending further discussion, so no longer a blocker.
- HDFS-11096 (compatibility with 2.x): Sean reviewed the JACC output and posted a big comment on possible source/binary incompatibilities.
- Erasure coding
- Still slowly burning down must-dos from the JIRA query.
- Classpath isolation (HADOOP-11565)
- HADOOP-11804 (shaded client jars): Initial patch has been committed, Sean also filed follow-on JIRAs for related work we need to clean up before beta1.
- Tomcat to Jetty conversions
- HADOOP-13597 (KMS): +1 pending a few nits
- HDFS-10860 (HttpFS): Patch up and gone through initial review
2016-12-09
Notable release blockers:
- HADOOP-13780: L&N updates from Jetty bump. Xiao has picked this up, no patch yet though.
Development activity
- Addressing incompatible changes
- MAPREDUCE-6704: still seems close after examination of Docker issue
- YARN-5882 (issuing YARN delegation tokens even in insecure deployments): Andrew proposed we revert YARN-4126 to unblock the release while discussion continues
- YARN-5559: Resolved!
- YARN-5184: Resolved!
- HDFS-11096 (compatibility with 2.x): Discussion still underway. Kihwal and Eric worked to resolve HDFS-11207 which reverted some incompatible DN/NN changes that broke rolling upgrade.
- Erasure coding
- Still slowly burning down must-dos from the JIRA query.
- Classpath isolation (HADOOP-11565)
- HADOOP-11804 (shaded client jars): Sean added integration tests for WebHDFS and things seem close.
- Tomcat to Jetty conversion (HDFS-11096, HDFS-10860)
- HDFS-11096 (KMS): patch is under review
- HDFS-10860 (HttpFS): no patch yet
2016-11-28
Notable release blockers:
- org.json:json: legal-discuss has allowed grandfathering in releases until April 30, 2017 for usages of org.json:json. HADOOP-13050 bumping the problematic AWS SDK dependency has been committed. We're in the clear.
- HADOOP-13780: L&N updates from Jetty bump. Still unassigned.
Development activity
- Addressing incompatible changes
- MAPREDUCE-6704: still seems close after examination of Docker issue
- YARN-5882: issuing YARN delegation tokens even in insecure deployments, discussion underway
- YARN-5559: found via JDiff analysis, fix some accidental incompats
- YARN-5184: found via downstream testing. New abstract methods added to public class.
- HDFS-11096 (compatibility with 2.x): Summary of a discussion between Andrew and Karthik was posted, seeking additional community feedback.
- Erasure coding
- Slowly burning down the must-dos from the JIRA query.
- Classpath isolation (HADOOP-11565)
- HADOOP-11804 (shaded client jars): POC testing with Avro and HBase seem to work, waiting on additional testing and reviews.
- Tomcat to Jetty conversion (HDFS-11096, HDFS-10860)
- Initial patch posted on HDFS-11096 for KMS conversion.
2016-11-16
Notable release blockers:
- HADOOP-11694: Need to upgrade AWS SDK dependency, org.json:json fallout
- HADOOP-13780: L&N updates from Jetty bump
Development activity
- Addressing incompatible changes
- MAPREDUCE-6704: make the default pseudo-cluster work without additional env var configuration, seems close?
- YARN-5882: issuing YARN delegation tokens even in insecure deployments, discussion underway
- YARN-5559: found via JDiff analysis, fix some accidental incompats
- YARN-5184: found via downstream testing. New abstract methods added to public class.
- Erasure coding
- Work continues on API refinements and supportability-related JIRAs.
- Classpath isolation (HADOOP-11565)
- HADOOP-11804 (shaded client jars): Multiple revs have been posted based on review feedback.