Page History

...

This document tracks on-going efforts to upgrade from Hadoop 2.x to Hadoop 3.x  - Refer Umbrella Jira HADOOP-15501 for current status on this.

Upgrade Tests for HDFS/YARN

The following scenarios were tested while upgrading from Hadoop 2.8.4 to Hadoop 3.1.0

...

Type	Component	Scenario	Issues Found	WorkaroundResolution	Overall Status
EXPRESS/ROLLING UPGRADE	HDFS	Starting 3.1.0 NameNode/DataNode with custom MetricsPlugin configured in hadoop2-metrics.properties	Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key HADOOP-15502	Workaround is applicable only for EXPRESS UPGRADE - Replace MetricsPlugin implementation jars( eg: HadoopTimelineMetricsSink) with recompilked recompiled jars which use package "org.apache.commons.configuration2"
EXPRESS UPGRADE	YARN	Starting Hadoop 3.1.0 YARN daemons
ROLLING UPGRADE	HDFS	3.1.0 NN is started with rollingUpgrade with default policy configured for Erasure coding	Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key HDFS-13596	Workaround Not known
ROLLING UPGRADE	YARN	Start 3.1.0 NM in batches after starting RM.	Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key YARN-8346	Fixed
EXPRESS/ROLLING UPGRADE	YARN	RM started with recovery enabled	Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key YARN-8068	Fixed

Upgrade Tests for YARN

Application Type	Upgrade Type	Issues Found	Overall Status
MR	EXPRESS/ROLLING UPGRADE	Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key YARN-8346
TEZ

SPARK